Air-ground robot action prediction model training and application method, device and medium

Abstract

The application discloses an air-ground robot action prediction model training and application method, equipment and medium, relates to the field of embodied intelligence, and the training method comprises the following steps: acquiring an artificial operation data set of an air-ground robot; the artificial operation data set comprises a plurality of artificial operation samples; the artificial operation sample is input into a visual language action model, a sample hidden state sequence output by the last layer of the Transformer of the LM is obtained, a sample fixed-length multi-modal representation is obtained after preprocessing, and a sample conditional embedding is obtained after the sample air-ground robot state and the sample environment state are fused; an intermediate action state block is determined according to a source end noise vector and a fixed-length action block; the student and teacher network are trained by taking the sample conditional embedding and the intermediate action state block at the current moment as input, and the application improves the air-ground robot action prediction efficiency and precision.

Description

Technical Field

[0001] This application relates to the field of embodied intelligence, and in particular to a method, device and medium for training and applying a motion prediction model for air and ground robots. Background Technology

[0002] With the continuous development of mechanical engineering, materials science, computer science, and automation technology, robots are increasingly replacing humans in performing high-risk, highly repetitive, and high-precision tasks. To further enhance the functionality of robots and expand their operating space, air-ground robots capable of performing tasks in both air and land environments have received widespread attention.

[0003] However, the richer motion modalities of air-to-ground robots also make their perception, planning, and control systems more complex. Compared to traditional robots such as wheeled robots, legged robots, and rotary-wing drones, the planning and control systems of air-to-ground robots not only need to complete multimodal motion planning and control, but also need to make decisions on motion mode switching. In some tasks with high precision requirements, the mode switching process needs to be additionally controlled. In addition, different motion modes usually correspond to significantly different dynamic characteristics, further increasing the complexity of the control system. Currently, research and engineering applications related to air-to-ground robots still largely rely on manual operation, and their level of intelligence is significantly lower than that of traditional robot platforms, limiting the practical value of air-to-ground robots.

[0004] With the development of artificial intelligence technology, especially the advancement of large language model-related technologies, VLA-based intelligent robot systems have attracted widespread attention, providing a new technical path for the design of intelligent systems for air and ground robots. However, most existing VLA systems based on autoregressive generation of action sequences have long inference times and large response delays, making them difficult to directly apply to air and ground robot systems that have high requirements for real-time performance and fast closed-loop response. Summary of the Invention

[0005] The purpose of this application is to provide a method, device, and medium for training and applying motion prediction models for air and ground robots, which can improve the efficiency and accuracy of motion prediction for air and ground robots.

[0006] To achieve the above objectives, this application provides the following solution:

[0007] Firstly, this application provides a method for training an air-to-ground robot motion prediction model, including:

[0008] A dataset of human-operated data for an air-to-ground robot is obtained. The dataset includes several human-operated samples. The human-operated samples include sample airborne RGB image sequences, sample air-to-ground robot state sequences, sample environmental state sequences, action sequence data, and sample air-to-ground navigation language commands. The human-operated samples are collected when the air-to-ground robot is manually controlled for air-to-ground motion navigation.

[0009] When training the visual language action model using the aforementioned human operation dataset, the human operation samples are input into the visual language action model to obtain the sample hidden state sequence output by the last layer of the Transformer in the visual language action model.

[0010] The hidden state sequence of the sample is preprocessed to obtain a fixed-length multimodal representation of the sample;

[0011] The sample air-ground robot state sequence, the sample environment state sequence, and the sample fixed-length multimodal representation are fused to obtain the sample conditional embedding at the current moment.

[0012] The intermediate action state blocks of the student network and teacher network at the current moment are determined based on the source noise vector and fixed-length action blocks; the fixed-length action blocks include actions at all moments within a set time period;

[0013] The student network is trained using the sample conditional embedding and the intermediate action state block of the student network at the current moment, and the teacher network is trained using the sample conditional embedding and the intermediate action state block of the teacher network at the current moment. The action sequence data at the current moment is used as the label to train the student and teacher networks, thereby obtaining a trained air-ground robot action prediction model.

[0014] Secondly, this application provides a method for applying an air-to-ground robot motion prediction model, including:

[0015] Acquire target control data for the air-to-ground robot; the target control data includes the target airborne RGB image, the target air-to-ground robot status, the target environment status, and the target air-to-ground navigation language commands;

[0016] The target manipulation data is input into the visual language action model to obtain the target hidden state sequence output by the last layer of the Transformer in the visual language action model.

[0017] The target hidden state sequence is preprocessed to obtain a fixed-length multimodal representation of the target;

[0018] The target air-to-ground robot state, the target environment state, and the target fixed-length multimodal representation are fused to obtain the target conditional embedding at the current moment;

[0019] The intermediate action state block at the current moment is determined based on the source noise vector and the fixed-length action block;

[0020] The target conditions at the current moment and the intermediate action state block are input into the trained air-ground robot action prediction model to obtain the predicted action at the next moment; the predicted action at the next moment is used to control the air-ground robot; the trained air-ground robot action prediction model is a model trained using the air-ground robot action prediction model training method described in the first aspect.

[0021] Thirdly, this application provides a computer device, including: a memory, a processor, and a computer program stored in the memory and capable of running on the processor, wherein the processor executes the computer program to implement the steps of the above-described air-to-ground robot motion prediction model training method or the above-described air-to-ground robot motion prediction model application method.

[0022] Fourthly, this application provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the above-described air-to-ground robot motion prediction model training method or the above-described air-to-ground robot motion prediction model application method.

[0023] According to the specific embodiments provided in this application, this application has the following technical effects:

[0024] This application provides a method, device, and medium for training and applying a motion prediction model for an air-to-ground robot, which acquires a dataset of human-operated data for the air-to-ground robot. The dataset includes several human-operated samples, comprising sample airborne RGB image sequences, sample air-to-ground robot state sequences, sample environmental state sequences, motion sequence data, and sample air-to-ground navigation language commands. These samples are collected when the air-to-ground robot is manually controlled for air-to-ground motion navigation. When training the visual language action model using the human-operated dataset, the human-operated samples are input into the model to obtain the hidden state sequence of the last layer of the Transformer in the LM (Linguistic Model). The state sequence is preprocessed to obtain a fixed-length multimodal representation of the samples. Feature fusion is performed on the sample air-ground robot state, sample environment state, and sample fixed-length multimodal representation to obtain the sample conditional embedding at the current moment. The intermediate action state blocks of the student network and teacher network at the current moment are determined based on the source noise vector and the fixed-length action block. The fixed-length action block includes actions at all moments within a set time period. The sample conditional embedding and intermediate action state blocks at the current moment are used as inputs to the student and teacher networks to improve the efficiency and accuracy of action prediction. The complete action vector is generated by a single endpoint mapping during the inference stage, reducing the errors and overhead caused by autoregression and discretization, and significantly reducing latency. It can be directly applied to air-ground robot systems with high requirements for real-time performance and fast closed-loop response. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 This is an application environment diagram of a method for training an air-to-ground robot motion prediction model or an application method for an air-to-ground robot motion prediction model according to an embodiment of this application.

[0027] Figure 2 This is a flowchart illustrating a method for training an air-to-ground robot motion prediction model, as provided in Embodiment 1 of this application.

[0028] Figure 3 This is a schematic diagram illustrating the specific process of the air-to-ground robot motion prediction model training method provided in Embodiment 1 of this application.

[0029] Figure 4 This is a schematic diagram comparing the average action step inference time of the method provided in Embodiment 1 of this application with that of TinyVLA.

[0030] Figure 5 This is a flowchart illustrating an application method for an air-to-ground robot motion prediction model provided in Embodiment 2 of this application.

[0031] Figure 6 This is a schematic diagram of the structure of a computer device provided in Embodiment 3 of this application. Detailed Implementation

[0032] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0033] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0034] Example 1

[0035] The air-to-ground robot motion prediction model training method provided in this application embodiment can be applied to, for example... Figure 1In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be set up independently, integrated into server 104, or placed in the cloud or on another server. Terminal 102 can send the manual operation dataset to server 104. After receiving the dataset, server 104 inputs the manual operation samples into the visual language action model (VM), obtaining the hidden state sequence of the last layer of the Transformer in the VM. The hidden state sequence is preprocessed to obtain a fixed-length multimodal representation. Feature fusion is performed on the robot state, environment state, and fixed-length multimodal representation to obtain the current conditional embedding. Based on the source noise vector and the fixed-length action block, the intermediate action state blocks of the student and teacher networks are determined. The current conditional embedding and the intermediate action state blocks of the student and teacher networks are used as inputs to the student and teacher networks, respectively. Using the current action sequence data as labels, the student and teacher networks are trained to obtain a trained robot action prediction model. Server 104 can then feed back the trained robot action prediction model to terminal 102. In addition, in some embodiments, the training method for the air-to-ground robot motion prediction model can also be implemented by the server 104 or the terminal 102 separately. For example, the terminal 102 can directly train the model on the human operation dataset, or the server 104 can obtain the human operation dataset from the data storage system and train the model on the human operation dataset.

[0036] The terminal 102 can be, but is not limited to, various desktop computers and laptops. The server 104 can be implemented using a standalone server or a server cluster consisting of multiple servers, or it can be a cloud server.

[0037] In one exemplary embodiment, such as Figure 2 As shown, a method for training a motion prediction model for an air-to-ground robot is provided. This method is executed by a computer device, specifically by a terminal or server alone, or by both a terminal and a server. In this embodiment, the method is applied to... Figure 1 Taking server 104 as an example, the explanation includes the following steps 201 to 206.

[0038] Step 201: Obtain the human operation dataset of the air-to-ground robot; the human operation dataset includes several human operation samples; the human operation samples include sample airborne RGB image sequences, sample air-to-ground robot state sequences, sample environmental state sequences, action sequence data, and sample air-to-ground navigation language commands; the human operation samples are collected when the air-to-ground robot is manually controlled to perform air-to-ground motion navigation.

[0039] Step 202: When training the visual language action model using the manual operation dataset, the manual operation samples are input into the visual language action model to obtain the sample hidden state sequence output by the last layer of the Transformer in the visual language action model.

[0040] Step 203: Preprocess the hidden state sequence of the sample to obtain a fixed-length multimodal representation of the sample.

[0041] Step 204: Perform feature fusion on the sample air-ground robot state sequence, the sample environment state sequence, and the sample fixed-length multimodal representation to obtain the sample condition embedding at the current moment.

[0042] Step 205: Determine the intermediate action state blocks of the student network and teacher network at the current moment based on the source noise vector and the fixed-length action block; the fixed-length action block includes the actions at all moments within the set time period.

[0043] Step 206: Using the sample conditional embedding and the intermediate action state block of the student network at the current moment as the input of the student network, and the sample conditional embedding and the intermediate action state block of the teacher network at the current moment as the input of the teacher network, and using the action sequence data at the current moment as the label, train the student and teacher networks to obtain a trained air-ground robot action prediction model.

[0044] Implementing steps 201 to 206 above, using the current conditional embedding and intermediate action state blocks as inputs to the student and teacher networks improves action prediction efficiency and accuracy. During the inference phase, the current conditional embedding and intermediate action state blocks are used as inputs to the trained air-to-ground robot action prediction model. A single endpoint mapping generates a complete action vector during inference, reducing errors and overhead caused by autoregression and discretization, significantly reducing latency. This approach can be directly applied to air-to-ground robot systems with high requirements for real-time performance and fast closed-loop response. Air-to-ground mode switching decisions and air-to-ground hybrid control characteristics are incorporated into a lightweight VLA framework. The VLA is fine-tuned based on data from manually manipulated air-to-ground robots, and a flow policy is introduced, utilizing consistent flow matching instead of autoregression to generate action sequences, thereby enhancing the VLA's thrust speed. Effectively integrating the unique planning and control requirements of air-to-ground robots into the VLA framework and improving VLA inference speed and response capabilities has significant research and application value for enhancing the autonomous intelligence level of air-to-ground robots.

[0045] like Figure 3 As shown, the training process includes the following steps 1 to 12.

[0046] Step 1: First, based on the pre-trained language model... and visual encoder A pre-trained visual-language model (VLM) is obtained through joint visual-language training. This application The system uses EleutherAI's Pythia; the visual encoder used is OpenAI's CLIP Vision Encoder.

[0047] Before joint training, the language model and visual encoder A feature mapping module is added between them to be responsible for... Output visual features mapped to The word embedding space; the vision-language joint training adopts the LLaVA training paradigm and dataset.

[0048] The feature mapping module can be a multilayer perceptron, or a transformer, linear projection, or other models; no specific limitation is made here.

[0049] Step 2: Manually control the air-to-ground robot to perform air-to-ground navigation in single airspace, single land area, and mixed scenarios of entering airspace from land area and entering land area from airspace, and collect air-to-ground operation data of the air-to-ground robot, which is recorded as the manual operation dataset.

[0050] The dataset is ,in For the number of episodes, For the first An episode of air-to-ground navigation, i.e., a manually operated sample, can be represented as: ,in The number of steps in the episode; That is, 50 seconds; the sampling period is Determined by experience, it can be between 0.001 and 1 second, with 0.1 seconds being a possible value in a specific example; , , and The first The episode from the beginning to The sample includes airborne RGB image sequences, sample air-to-ground robot state sequences, sample environmental state sequences, action sequence data, and sample air-to-ground navigation language commands; among them... , , 3 and 3 represent the height, width, and 3 channels of the sample airborne RGB image, respectively; for Time of the first The sample air-to-ground robot state in a single manually operated sample includes both direct IMU measurements and estimates obtained after fusing the IMU with other sensors. , , Acceleration, angular velocity, position, and pitch angle in three directions Yaw angle Roll angle and sports modes Where 1 represents flight mode, -1 represents ground mode, -2 represents the transition from ground mode to flight mode, and 2 represents the transition from flight mode to ground mode; the boundaries of each mode vary depending on the structure and are defined empirically. for Time of the first The environmental conditions of the sample in the manual operation sample include the wind speed measured by the sensor and the Laplacian variance of the RGB image. The wind speed measured by the sensor is used to reflect the flight conditions, and the Laplacian variance of the RGB image is used to reflect the ground surface conditions. The larger the variance, the worse the road surface smoothness. ( (This can assist VLM in further understanding the runtime environment and action decisions) as the first In the episode The 6D motion sequence data of the air-to-ground robot at any given moment includes the air-to-ground robot's... velocity in direction , velocity in direction , velocity in direction Pitch angular velocity yaw rate and roll rate ; For the first Each episode contains one sample air-to-ground navigation instruction in natural language form.

[0051] Step 3: To In the attention layer of the Transformer, a low-rank adaptation (LoRA) module is inserted at the linear projection weight matrix of the query vector Q, key vector K, and value vector V.

[0052] The visual language action model in step 202 above is an improved visual language model. This improved visual language model, compared to existing visual language models, [is an improvement upon the previous one]. A feature mapping module is introduced between the visual encoder and the visual encoder. In the attention layer of the Transformer, a low-rank adaptation module is inserted at the linear projection weight matrix of the query vector Q, key vector K, and value vector V; the feature mapping module is used to map the output of the visual encoder to the word embedding space.

[0053] Step 4: Freeze and The main parameters are minimized by using a dataset based on manual operations. The weighted next-word prediction cross-entropy loss is used, and backpropagation is used to update only the model parameters of the LoRA module (including the upper projection matrix and the lower projection matrix). Training is completed when the training epochs or the loss meets the preset conditions (the training epochs reach 500 epochs or the relative change rate of the weighted next-word prediction cross-entropy loss is less than 0.1% within 5 consecutive epochs).

[0054] When training the visual language action model using the aforementioned human-operated dataset, before the parameters begin to be updated... In It is necessary to use a visual encoder to extract features from the sample airborne RGB images in the manually operated dataset to obtain visual features.

[0055] The visual features are mapped to using the feature mapping module. The word embedding space is used to obtain visual tokens; the sample empty ground robot state sequence is used. Sample environment state sequence Action sequence data and sample air-to-ground navigation language instructions The visual token and the enhanced text training sample are concatenated to obtain the language token, which is then used as the input to the language model.

[0056] Before calculating the weighted next-word prediction cross-entropy loss, language tokens are divided into key tokens and remaining tokens for air-to-ground motion navigation of interest. The weighted next-word prediction cross-entropy loss is then calculated based on these two types of tokens. Specifically, the weighted next-word prediction cross-entropy is calculated separately for each type of token and then weighted. The weighted next-word prediction cross-entropy loss is used to update the upper and lower projection matrix parameters of the low-rank adaptation module. The formula for calculating the weighted next-word prediction cross-entropy loss is as follows:

[0057] (1);

[0058] in, This represents the set of upper and lower projection matrix parameters in the LoRA module, including upper projection matrix parameters and lower projection matrix parameters; Represents the loss function. For the gradient operator with respect to the set of parameters of the upper and lower projection matrices in the LoRA module; and These represent the key token index set and the remaining token index set, respectively. The key token index set includes seven types of key tokens: "takeoff", "landing", "flight", "ground movement", "airspace to land", "land to airspace", and "switching". Indicates the position of the key token in the key token index set; Indicates the position of the remaining tokens in the remaining token index set; and Assigned to respectively and The gradient weight coefficients of the token. In a specific example, it is possible to take and .

[0059] Step 5: The sequence of hidden states of the samples output from the last layer of the Transformer at time LM is used as a multimodal token. Its size is simultaneously with Visual token embeddings and language instructions output by encoder MLP1 The total number of token embeddings needs to be unified. The length.

[0060] Step 5.1: Extract the hidden state sequence of the sample along the token dimension. Perform adaptive pooling to obtain a fixed-length vector .

[0061] Step 5.2, Multimodal Feature Normalization: Further, for fixed-length vectors... Layer normalization is performed to obtain a fixed-length multimodal representation. The lengths were standardized.

[0062] Step 6: Sequence of sample open-air robot states Sample environment sequence With fixed-length multimodal representation of samples The input is fed into the DMLP (Deep Multilayer Perceptron Branch), and the sample conditional embeddings are obtained through encoding and fusion within the DMLP. This serves as the conditional input for subsequent policy generation networks (student network and teacher network).

[0063] The sample air-to-ground robot state sequence, the sample environment state sequence, and the sample fixed-length multimodal representation are fused to obtain the sample conditional embedding at the current moment. Specifically, this includes: using a deep multilayer perceptron branch to fuse the features of the sample air-to-ground robot state sequence, the sample environment state sequence, and the sample fixed-length multimodal representation; the deep multilayer perceptron branch includes a first encoder, a second encoder, a third encoder, a stitching unit, and a fourth encoder; the first encoder, second encoder, third encoder, and fourth encoder are multilayer perceptrons; the first encoder is used to encode the sample air-to-ground robot state sequence to obtain a state feature embedding; the second encoder is used to encode the sample environment state sequence to obtain an environment feature embedding; the third encoder is used to encode the sample fixed-length multimodal representation to obtain a control feature embedding; the stitching unit is used to stitch the state feature embedding, the environment feature embedding, and the control feature embedding to obtain a stitched feature; the fourth encoder is used to fuse the stitched feature to obtain the sample conditional embedding at the current moment.

[0064] DMLP first uses three independent encoders (first encoder MLP2, second encoder MLP3, and third encoder MLP4) to... , and Encode them separately to obtain the state feature embedding. Environmental feature embedding and control feature embedding Then splice the unit pairs , and The data is then concatenated and input into the fourth encoder, MLP5, to obtain... Time of the first Sample condition embedding of a manually manipulated sample This process can be represented as:

[0065] (2);

[0066] (3);

[0067] (4);

[0068] (5);

[0069] in, Indicates to , and The features are then concatenated. The feature encoding fusion code is shown in Table 1.

[0070] Table 1 Feature Coding Fusion Code

[0071]

[0072] The process of building the student and teacher network includes steps 7 and 8.

[0073] Step 7: By learning a continuous generation time Define a conditional velocity field function, construct a conditional policy generation network to model action distribution, and use this conditional policy generation network as the student network; (Conditional velocity field: a vector-valued function modulated by conditional variables, used to generate actions in continuous time...) Based on the current intermediate state, predict the direction and magnitude of its evolution toward the target distribution; Conditional policy generation network: essentially a policy generation network, except that the input is conditional.

[0074] The conditional velocity field function is ;in, It represents the normalized generation time in the continuous generation process, used to characterize the generation progress from the initial noise state to the target action distribution; For the network parameters of the student network; for Time of the first The intermediate action state block of a manually operated sample; for Time of the first Sample condition embedding of manually manipulated samples; Representing network parameters The network, at the generation time Intermediate action state block and sample conditional embedding Given a vector, output the vector of changes in the action. For the mapping function of the student network; This represents the dimensions of intermediate action state blocks and predicted actions; Indicates the dimension of the sample conditional embedding; It represents the set of real numbers.

[0075] The conditional velocity field semantically satisfies the consistency constraint: that is, for the same target action distribution, regardless of the time point in the diffusion process... Departure or which intermediate action state block From the outset, the direction of action changes predicted by the network should be semantically consistent and not contradictory.

[0076] Step 8: Define the conditional strategy to generate the network structure.

[0077] Step 8.1: Construct a fixed-length action block Expand the fixed-length action block as the target endpoint action vector. ,in; They are respectively Time of the first The actions of manually manipulating samples; The length of the time domain; This is a vectorized function used to convert fixed-length action blocks. Expand into a vector by columns The 1 in the subscript indicates the number of the vectorization result.

[0078] Step 8.2: Based on the conditional velocity field function and consistency constraints, define the smooth endpoint mapping function of the student network. Responsible for mapping intermediate actions at arbitrary diffusion times to the target action space; the smooth endpoint mapping function of the teacher network is defined as follows: The network structure of the teacher network and The same network parameters for the teacher network Depend on The result is obtained through exponential moving average updates; the consistency constraint is that for the same target action distribution, regardless of the point in time during the diffusion process... Departure or which intermediate action state block Once started, the direction of action change predicted by the network remains semantically consistent;

[0079] Smooth endpoint mapping function in student networks for:

[0080] (6);

[0081] in, The time decay exponent, In a specific example, it is possible to take ; for Time of the first The sample condition embeddings of each manually manipulated sample are shown in Table 2. The code for constructing the student and teacher networks is also shown in Table 2.

[0082] Table 2. Code for constructing student and teacher networks

[0083]

[0084] The student network and the teacher network map the intermediate action state (A_t in Table 2) to the target action space under the guidance of the conditional velocity field (v_t in Table 2) through the smooth endpoint mapping function (f_mu in Table 2), and introduce the time decay exponent (gamma=1.2 in Table 2) to ensure smoothness. The structure of the teacher network is consistent with that of the student network, but the parameters are updated using exponential moving average to maintain the consistency of action prediction and semantic stability.

[0085] Step 9: Construct consistent matching training pairs for FlowPolicy.

[0086] Step 9.1: Sample the source noise vector Source-end noise vector Information noise; source noise vector ,in Represents a multivariate normal distribution; intermediate parameters , Representing the target endpoint action vectors respectively The mean and corresponding covariance matrix over the time dimension; Expressing expectations; Represents covariance; For stability coefficient, the preferred value is... It is acceptable ; express The identity matrix. The code for the noise vector at the sampling source end is shown in Table 3.

[0087] Table 3. Noise vector at the sampling source end

[0088]

[0089] The source noise vector is obtained by calculating the mean and covariance of the target action (target in Table 3), generating a lower triangular matrix through Cholesky decomposition, and then linearly mapping it with Gaussian noise. This provides statistically consistent intermediate action initialization for the construction of student and teacher network training pairs.

[0090] Step 9.2: Based on Constructing intermediate states, the intermediate action state block of the student network at the current moment is: The intermediate action state block of the teacher network at the current moment ,Will and These serve as inputs to the FlowPolicy student network and teacher network, respectively. express The time step can be taken .

[0091] Step 10: Calculate the consistency matching loss in FlowPolicy The consistency matching loss function is expressed as follows:

[0092] (7);

[0093] in, for Time of the first Loss value for each manually operated sample; , These are endpoint consistency loss and speed consistency loss, respectively. For the smooth endpoint mapping function of the student network; For the smooth endpoint mapping function of the teacher network; for Time of the first The intermediate action state block of a manually operated sample student network; Indicates student network; For the network parameters of the student network; Indicates a teacher network; for Time of the first The intermediate action state block of a sample teacher network for manual operation; For network parameters of the teacher network; As a weighting factor, it can be taken as follows: .

[0094] Step 11: Manually manipulate the dataset Calculate the consistency matching loss And update the network parameters of the student network. Gradient descent updates are performed to minimize the cumulative consistency matching loss; network parameters of the teacher network. Updated using exponential moving average; resulting in a well-trained student network. Mapping function with smooth endpoints This serves as a pre-trained motion prediction model for air-to-ground robots.

[0095] In step 206 above, the student network is trained using the sample conditional embedding and the intermediate action state block of the student network at the current moment, and the teacher network is trained using the sample conditional embedding and the intermediate action state block of the teacher network at the current moment, with the action sequence data at the current moment as the label, to obtain a trained air-ground robot action prediction model. Specifically, this includes: using the sample conditional embedding and the intermediate action state block of the student network at the current moment as the input of the student network, using the sample conditional embedding and the intermediate action state block of the teacher network at the current moment as the input of the teacher network, and using the action sequence data at the current moment as the label, training the student network and the teacher network using the consistency matching loss function, updating the network parameters of the student network using gradient descent to minimize the cumulative consistency matching loss, and obtaining a trained air-ground robot action prediction model.

[0096] Step 12: In time ,make Sample Smooth endpoint mapping function input to the trained air-to-ground robot motion prediction model Obtain the predicted endpoint vector Take the first action to control execution and update. Repeat the above process to achieve rolling time-domain optimization until the task is completed.

[0097] Implemented using Python, the libraries used include: python==3.8, torch, torchvision, torchaudio, pytorch3d==0.7.5, diffusers==0.11.1, transformers, accelerate, einops==0.4.1, gym==0.21.0, mujoco-py==2.1.2.14, dm_control, Metaworld, mjrl, mj_envs, FlowPolicy, policy_heads, llava-pythia, and visual. izer, numpy, scipy, numba==0.56.4, zarr==2.12.0, open3d, opencv-python, imageio, av, moviepy, matplotlib, omegaconf, hydra-core==1.2. 0, wandb, dill==0.3.5.1, ipdb, gpustat, termcolor, natsort, plotly, kaleido, setuptools==59.5.0, Cython==0.29.35, patchelf==0.17.2.0.

[0098] It is based on an NVIDIA RTX 4500 Ada graphics card and an AMD Ryzen Threadripper PRO CPU.

[0099] During the experiment, the performance of five air-to-ground navigation language commands was tested in the same indoor environment, as detailed below:

[0100] Instruction 1: "Move forward along the corridor and stop at the foot of the second chair on the right."

[0101] Command 2: "After takeoff, maintain the current altitude, fly forward three meters, and hover directly below the ceiling light."

[0102] Command 3: "Move forward along the ground to an open area, then take off and hover above the center of the room."

[0103] Command 4: "Fly two meters to the left from the current position, then descend and land safely on the ground."

[0104] Command 5: "Move forward one meter along the ground, take off, pass over the obstacle ahead, and land on the other side of the obstacle."

[0105] The trained air-to-ground robot motion prediction model of this application was compared with TinyVLA. Each experiment was repeated 20 times, and the number of successful experiments (the air-to-ground robot completed the corresponding actions according to the above instructions) is recorded in Table 4.

[0106] Table 4 Number of successful experiments

[0107]

[0108] As can be seen from Table 4, this application maintains an advantage in all scenarios from instruction 2 to instruction 5, with instruction 5 showing the most significant advantage. Although instruction 1 lags behind TinyVLA, the gap is relatively small.

[0109] The method described in this application is compared with the average action step inference time of TinyVLA, for example... Figure 4 As shown, from Figure 4 As can be seen, in all instruction scenarios, the inference time per action step in this application is shorter than that of TinyVLA, with a maximum reduction of 25.64% in inference time (instruction 1 scenario). A single action step refers to one action prediction.

[0110] This application constructs a human manipulation dataset based on human manipulation data, trains and fine-tunes the model using this dataset, and perfectly replicates human manipulation habits, thus laying the foundation for air-to-ground robots to move like humans. It has the following advantages:

[0111] (1) By using FlowPolicy to model the conditional distribution in the continuous action space, the complete action vector is generated by a single endpoint mapping during the inference stage, which reduces the error and overhead caused by autoregression and discretization, and significantly reduces the time delay.

[0112] (2) A weighted method for calculating the cross-entropy of the next word prediction is proposed.

[0113] (3) A sliding endpoint mapping function was designed, and a time decay exponent was introduced to avoid endpoint instability;

[0114] (4) Transform the noise vector at the source end into information noise so that the noise is no longer an unstructured random disturbance, but explicitly characterizes the statistical uncertainty of the target endpoint action in the time dimension, thereby improving the stability, consistency and task relevance of the strategy generation process.

[0115] This application also provides an application scenario in which the above-described air-to-ground robot motion prediction model training method is applied. Specifically, the air-to-ground robot motion prediction model training method provided in this embodiment can be applied to an air-to-ground robot motion prediction scenario. The air-to-ground robot motion prediction scenario includes a dataset acquisition stage, a model training chain, and an air-to-ground robot motion prediction stage. The manually operated dataset enters the model training chain from the dataset acquisition stage, and through human-machine collaboration, a trained air-to-ground robot motion prediction model is obtained and then enters the downstream air-to-ground robot motion prediction stage. The air-to-ground robot motion prediction model training method provided in this embodiment belongs to the model training chain. Specifically, in the model training process for the manual operation dataset, the manual operation samples in the dataset can be input into the visual language action model to obtain the sample hidden state sequence output by the last layer of the Transformer in the visual language action model. The sample hidden state sequence is preprocessed to obtain the sample fixed-length multimodal representation. The sample air-ground robot state, sample environment state, and sample fixed-length multimodal representation are fused to obtain the sample conditional embedding at the current time. Based on the source noise vector and the fixed-length action block, the intermediate action state block of the student network and teacher network at the current time is determined. The sample conditional embedding at the current time and the intermediate action state block of the student network are used as the input of the student network, and the sample conditional embedding at the current time and the intermediate action state block of the teacher network are used as the input of the teacher network. The action sequence data at the current time is used as the label to train the student and teacher networks to obtain the trained air-ground robot action prediction model.

[0116] Example 2

[0117] The air-to-ground robot motion prediction model application method provided in this application embodiment can be applied to, for example... Figure 1In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be set up independently, integrated into server 104, or placed in the cloud or on another server. Terminal 102 can send target manipulation data to server 104. After receiving the target manipulation data, server 104 inputs the target manipulation data into the visual language action model to obtain the target hidden state sequence output from the last layer of the Transformer in the LM (Linguistic Action Model). The target hidden state sequence is preprocessed to obtain a fixed-length multimodal representation of the target. Feature fusion is performed on the target air-ground robot state, the target environment state, and the target fixed-length multimodal representation to obtain the target conditional embedding at the current moment. The intermediate action state block at the current moment is determined based on the source noise vector and the fixed-length action block. The target conditional embedding and the intermediate action state block at the current moment are input into the trained air-ground robot action prediction model to obtain the predicted action for the next moment. Server 104 can feed back the obtained predicted action for the next moment based on the target manipulation data to terminal 102. In addition, in some embodiments, the training method for the air-to-ground robot motion prediction model can also be implemented by the server 104 or the terminal 102 separately. For example, the terminal 102 can directly perform motion prediction based on the target control data, or the server 104 can obtain the target control data from the data storage system and perform motion prediction based on the target control data.

[0118] The terminal 102 can be, but is not limited to, various desktop computers and laptops. The server 104 can be implemented using a standalone server or a server cluster consisting of multiple servers, or it can be a cloud server.

[0119] In one exemplary embodiment, such as Figure 5 As shown, a method for applying a motion prediction model for an air-to-ground robot is provided. This method is executed by a computer device, specifically by a terminal or server alone, or by both a terminal and a server. In this embodiment, the method is applied to... Figure 1 Taking server 104 as an example, the explanation includes the following steps 301 to 306.

[0120] Step 301: Obtain target control data for the air-to-ground robot; the target control data includes the target airborne RGB image, the target air-to-ground robot status, the target environment status, and the target air-to-ground navigation language commands. Details are as described above and will not be repeated here.

[0121] Step 302: Input the target manipulation data into the visual language action model to obtain the target hidden state sequence output by the last layer of the Transformer in the visual language action model.

[0122] The visual language action model here is an improved visual language model trained through steps 1-4 above. This improved visual language model, compared to existing visual language models... A feature mapping module is introduced between the visual encoder and the visual encoder. In the attention layer of the Transformer, a low-rank adaptation module is inserted at the linear projection weight matrix of the query vector Q, key vector K, and value vector V; the feature mapping module is used to map the output of the visual encoder to the word embedding space.

[0123] Step 303: Preprocess the target hidden state sequence to obtain a fixed-length multimodal representation of the target.

[0124] After obtaining the target hidden state sequence output by the last layer of the Transformer of the LM, adaptive pooling is performed on the target hidden state sequence in the token dimension to obtain a target fixed-length vector. The target fixed-length vector is then layer normalized to obtain a target fixed-length multimodal representation to unify the length.

[0125] Step 304: Perform feature fusion on the target air-ground robot state, the target environment state, and the target fixed-length multimodal representation to obtain the target condition embedding at the current moment.

[0126] The deep multilayer perceptron branch is used to perform feature fusion on the target air-ground robot state, the target environment state, and the target fixed-length multimodal representation. Specifically, the first encoder, the second encoder, and the third encoder encode the target air-ground robot state, the target environment state, and the target fixed-length multimodal representation, respectively, to obtain state feature embedding, environment feature embedding, and control feature embedding. The splicing unit splices the state feature embedding, environment feature embedding, and control feature embedding to obtain spliced ​​features. The spliced ​​features are then input into the fourth encoder to obtain the target conditional embedding at the current time.

[0127] Step 305: Determine the intermediate action state block at the current moment based on the source noise vector and the fixed-length action block.

[0128] Expand the fixed-length action block as the target endpoint action vector, based on the source noise vector. and target endpoint action vector, based on Construct intermediate action state blocks.

[0129] Step 306: The target conditions and intermediate action state blocks of the current moment are embedded into the trained air-ground robot action prediction model to obtain the predicted action of the next moment; the predicted action of the next moment is used to control the air-ground robot; the trained air-ground robot action prediction model is a model trained using the air-ground robot action prediction model training method described in Example 1.

[0130] The target conditions embedded at the current moment and the intermediate action state block are input into the smooth endpoint mapping function of the trained air-to-ground robot action prediction model. Obtain the predicted endpoint vector The first action is taken as the predicted action for the next moment, which is used to control the air-to-ground robot to execute actions and update the current moment. Repeat the above process to achieve rolling time-domain optimization until the task is completed.

[0131] This application also provides an application scenario in which the above-described air-to-ground robot motion prediction model application method is applied. Specifically, the air-to-ground robot motion prediction model application method provided in this embodiment can be applied in an air-to-ground robot control scenario. The air-to-ground robot control scenario includes a data acquisition stage, an air-to-ground robot motion prediction link, and an air-to-ground robot control stage. Target manipulation data enters the air-to-ground robot motion prediction link from the data acquisition stage, and through human-machine collaboration, the predicted motion for the next moment is obtained, and then enters the downstream air-to-ground robot control stage. The air-to-ground robot motion prediction model application method provided in this embodiment belongs to the air-to-ground robot motion prediction link. Specifically, in the air-to-ground robot motion prediction process based on target manipulation data, the target manipulation data can be input into the visual language action model to obtain the target hidden state sequence output by the last layer of the Transformer in the visual language action model. The target hidden state sequence is preprocessed to obtain a fixed-length multimodal representation of the target. Feature fusion is performed on the target air-to-ground robot state, the target environment state, and the target fixed-length multimodal representation to obtain the target conditional embedding at the current moment. The intermediate action state block at the current moment is determined based on the source noise vector and the fixed-length action block. The target conditional embedding at the current moment and the intermediate action state block are input into the trained air-to-ground robot motion prediction model to obtain the predicted action at the next moment.

[0132] Example 3

[0133] In one exemplary embodiment, a computer device is provided, which may be a server or a terminal, and its internal structure diagram may be as follows. Figure 6As shown, the computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores data related to the motion prediction of the air-to-ground robot. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements a method for training an air-to-ground robot motion prediction model or a method for applying the air-to-ground robot motion prediction model.

[0134] Those skilled in the art will understand that Figure 6 The structures shown are merely block diagrams of some structures related to the present application and do not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than shown in the figures, or combine certain components, or have different component arrangements. In an exemplary embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0135] Example 4

[0136] In one exemplary embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0137] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Moreover, the collection, use and processing of the relevant data are carried out in compliance with the relevant data protection laws and policies of the country where the location is located, and with the authorization granted by the owner of the corresponding device.

[0138] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0139] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0140] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0141] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for training a motion prediction model for an air-to-ground robot, characterized in that, The training method for the air-to-ground robot motion prediction model includes: A dataset of human-operated data for an air-to-ground robot is obtained. The dataset includes several human-operated samples. The human-operated samples include sample airborne RGB image sequences, sample air-to-ground robot state sequences, sample environmental state sequences, action sequence data, and sample air-to-ground navigation language commands. The human-operated samples are collected when the air-to-ground robot is manually controlled for air-to-ground motion navigation. When training the visual language action model using the aforementioned human operation dataset, the human operation samples are input into the visual language action model to obtain the sample hidden state sequence output by the last layer of the Transformer in the visual language action model. The hidden state sequence of the sample is preprocessed to obtain a fixed-length multimodal representation of the sample; The sample air-ground robot state sequence, the sample environment state sequence, and the sample fixed-length multimodal representation are fused to obtain the sample conditional embedding at the current moment. The intermediate action state blocks of the student network and teacher network at the current moment are determined based on the source noise vector and fixed-length action blocks; the fixed-length action blocks include actions at all moments within a set time period; Using the current sample conditional embedding and the intermediate action state block of the student network as input to the student network, and the current sample conditional embedding and the intermediate action state block of the teacher network as input to the teacher network, with the current action sequence data as labels, the student and teacher networks are trained using the consistency matching loss function. The network parameters of the student network are updated using gradient descent to minimize the cumulative consistency matching loss, resulting in a trained air-to-ground robot action prediction model. The consistency matching loss function is expressed as follows: ； in, for Time of the first Loss value for each manually operated sample; , These are endpoint consistency loss and speed consistency loss, respectively. For the smooth endpoint mapping function of the student network; For the smooth endpoint mapping function of the teacher network; for Time of the first The intermediate action state block of a manually operated sample; For time t, the first Sample condition embedding of manually manipulated samples; express Time step; Indicates student network; For the network parameters of the student network; Indicates a teacher network; For network parameters of the teacher network; This is the weighting factor.

2. The method for training an air-to-ground robot motion prediction model according to claim 1, characterized in that, The process of constructing the student and teacher network is as follows: In continuous generation time A conditional velocity field function is defined above, and a conditional policy generation network is constructed to model the action distribution. This conditional policy generation network is then used as the student network. The conditional velocity field function is... ;in, It represents the normalized generation time in the continuous generation process, used to characterize the generation progress from the initial noise state to the target action distribution; For the network parameters of the student network; for Time of the first The intermediate action state block of a manually operated sample; for Time of the first Sample condition embedding of manually manipulated samples; Represents the student network, at the time of generation. Intermediate action state block and sample conditional embedding Given a vector, output the vector of changes in the action. For the mapping function of the student network; This represents the dimensions of intermediate action state blocks and predicted actions; Indicates the dimension of the sample conditional embedding; Represents the set of real numbers; Construct a fixed-length action block Expand the fixed-length action block as the target endpoint action vector. ,in; They are respectively Time of the first The actions of manually manipulating samples; The length of the time domain; This is a vectorized function used to convert fixed-length action blocks. Expand into a vector by columns The 1 in the subscript indicates the number of the vectorization result; Based on the aforementioned conditional velocity field function and consistency constraints, a smooth endpoint mapping function for the student network is defined. Responsible for mapping intermediate actions at arbitrary diffusion times to the target action space; the smooth endpoint mapping function of the teacher network is defined as follows: The network structure of the teacher network and Similarly, the network parameters of the teacher network Depend on The result is obtained through exponential moving average updates; the consistency constraint is that for the same target action distribution, regardless of the point in time during the diffusion process... Departure or which intermediate action state block Once started, the direction of action change predicted by the network remains semantically consistent; The smooth endpoint mapping function in the student network for: in, This is the time decay exponent.

3. The method for training an air-to-ground robot motion prediction model according to claim 2, characterized in that, The intermediate action state block of the student network at the current moment is The intermediate action state block of the teacher network at the current moment Among them, the source noise vector ,in Represents a multivariate normal distribution; , Representing the target endpoint action vectors respectively The mean and corresponding covariance matrix over the time dimension; Expressing expectations; Represents covariance; The stability coefficient; express The identity matrix; express The time step.

4. The method for training an air-to-ground robot motion prediction model according to claim 1, characterized in that, Feature fusion is performed on the sample air-to-ground robot state sequence, the sample environment state sequence, and the sample fixed-length multimodal representation to obtain the sample conditional embedding at the current moment, specifically including: The deep multilayer perceptron branch is used to perform feature fusion on the sample air-ground robot state sequence, the sample environment state sequence, and the sample fixed-length multimodal representation; the deep multilayer perceptron branch includes a first encoder, a second encoder, a third encoder, a stitching unit, and a fourth encoder; the first encoder, the second encoder, the third encoder, and the fourth encoder are multilayer perceptrons. The first encoder is used to: encode the sample air-to-ground robot state sequence to obtain state feature embedding; The second encoder is used to: encode the sample environment state sequence to obtain an environmental feature embedding; The third encoder is used to: encode the fixed-length multimodal representation of the sample to obtain the control feature embedding; The splicing unit is used to splice the state feature embedding, the environment feature embedding, and the control feature embedding to obtain spliced ​​features; The fourth encoder is used to fuse the spliced ​​features to obtain the sample condition embedding at the current time.

5. The method for training an air-to-ground robot motion prediction model according to claim 1, characterized in that, The visual language action model is an improved visual language model. This improved visual language model, compared to existing visual language models, [is an improvement upon the previous one]. A feature mapping module is introduced between the visual encoder and the visual encoder. In the attention layer of the Transformer, a low-rank adaptation module is inserted at the linear projection weight matrix of the query vector Q, key vector K, and value vector V; the feature mapping module is used to map the output of the visual encoder to the word embedding space.

6. The method for training an air-to-ground robot motion prediction model according to claim 5, characterized in that, The training method for the air-to-ground robot motion prediction model also includes: When training the visual language action model using the aforementioned manual operation dataset, a visual encoder is used to extract features from the sample airborne RGB images in the manual operation dataset to obtain visual features. The visual features are mapped to using the feature mapping module. The word embedding space yields the visual token; The enhanced text training samples are obtained by concatenating the sample air-ground robot state sequence, sample environment state sequence, action sequence data and sample air-ground navigation language commands. Visual tokens and enhanced text training samples are concatenated at the token level to obtain language tokens, which are then used as input to the language model. The language tokens are divided into key tokens and remaining tokens for air-to-ground motion navigation, and the weighted cross-entropy loss value for next word prediction is calculated based on the key tokens and remaining tokens. The model parameters of the low-rank adaptation module are updated using the weighted next-word prediction cross-entropy loss value; the formula for calculating the weighted next-word prediction cross-entropy loss value is as follows: ； in, This represents the set of upper and lower projection matrix parameters in the LoRA module, including upper projection matrix parameters and lower projection matrix parameters; Represents the loss function. For the gradient operator with respect to the set of parameters of the upper and lower projection matrices in the LoRA module; and These represent the key token index set and the remaining token index set, respectively. Indicates the position of the key token in the key token index set; Indicates the position of the remaining tokens in the remaining token index set; and Assigned to respectively and The gradient weight coefficients of the token. .

7. A method for applying a motion prediction model for air-to-ground robots, characterized in that, The application method of the air-to-ground robot motion prediction model includes: Acquire target control data for the air-to-ground robot; the target control data includes the target airborne RGB image, the target air-to-ground robot status, the target environment status, and the target air-to-ground navigation language commands; The target manipulation data is input into the visual language action model to obtain the target hidden state sequence output by the last layer of the Transformer in the visual language action model. The target hidden state sequence is preprocessed to obtain a fixed-length multimodal representation of the target; The target air-to-ground robot state, the target environment state, and the target fixed-length multimodal representation are fused to obtain the target conditional embedding at the current moment; The intermediate action state block at the current moment is determined based on the source noise vector and the fixed-length action block; The target conditions at the current moment and the intermediate action state block are input into the trained air-ground robot action prediction model to obtain the predicted action at the next moment; the predicted action at the next moment is used to control the air-ground robot; the trained air-ground robot action prediction model is a model trained using the air-ground robot action prediction model training method according to any one of claims 1-6.

8. A computer device, comprising: A memory, a processor, and a computer program stored in the memory and capable of running on the processor, characterized in that the processor executes the computer program to implement the air-to-ground robot motion prediction model training method according to any one of claims 1-6 or the air-to-ground robot motion prediction model application method according to claim 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the air-to-ground robot motion prediction model training method according to any one of claims 1-6 or the air-to-ground robot motion prediction model application method according to claim 7.

Images