Robot action prediction method and device, computer equipment and storage medium

Abstract

The invention relates to the technical field of robots, and discloses a robot action prediction method and device, computer equipment and a storage medium, and the method comprises the steps: generating an input observation sequence according to a position code and a current feature vector, and predicting an action sequence through Transform and the input observation sequence; and exponential decay weighted average processing is carried out on the predicted action sequence, and a target action is determined. Through the above mode, the multi-modal observation data is converted into the unified feature vector through the feature extraction network, key information in the observation data is reserved, the time sequence dependency relationship and context information in the observation data are fully captured by using the Transform decoder, and the robot action sequence is accurately predicted. And exponential decay weighted average processing is performed on the predicted action sequence, so that the action sequence is further smoothed, the prediction instability is reduced, the finally determined target action is more accurate, and the performance and success rate of the robot during task execution are improved.

Description

Technical Field

[0001] This application relates to the field of robotics technology, and in particular to a method, apparatus, computer equipment, and storage medium for predicting robot motions. Background Technology

[0002] In the field of robotics, imitation learning and behavior cloning are widely considered efficient ways to enable robots to quickly master complex operational skills. The basic idea is to train a policy network by collecting demonstration data from human experts performing specific tasks (e.g., visual images, proprioceptive states, and corresponding continuous actions), enabling the robot to reproduce the expert's behavior in similar environments. In recent years, with the development of deep learning, the industry has successively proposed various advanced models that perform well in both simulation and real-world scenarios, such as ACT (Action Chunking with Transformers).

[0003] When facing practical deployments, the aforementioned existing technologies generally suffer from the following defects and shortcomings: For example, their architectures are overly complex. To cope with the multimodal distribution of actions, many state-of-the-art (SOTA) models integrate variational autoencoders, diffusion models, or vector quantization modules in addition to the backbone network. While these generative components help capture multimodal action distributions during training, in the inference phase, robots typically only need to output a single, definite action trajectory, rendering the generative modules redundant and causing unnecessary computational overhead and latency. Furthermore, the models have a massive number of parameters; existing high-performance models generally have 33 million to 270 million or more parameters, while the robot's action space typically has only 10 to 20 consecutive values. This mismatch between parameters and tasks results in significant storage and computational waste, making it difficult to run in real-time on CPUs, edge computing devices, or GPU embedded platforms. Finally, the training process is cumbersome; some models employ a two-stage training process of "discretization followed by regression," requiring additional clustering or quantization steps, increasing training complexity and time costs.

[0004] Therefore, how to improve the efficiency of robot imitation learning through simplified architecture, optimized training process and efficient inference mechanism has become an urgent technical problem to be solved. Summary of the Invention

[0005] This application provides a robot motion prediction method, apparatus, computer equipment, and storage medium to improve the efficiency of robot imitation learning.

[0006] In a first aspect, this application provides a robot motion prediction method, the method comprising:

[0007] The current feature vector is generated using a pre-defined feature extraction network and the robot's current observation data.

[0008] Based on the preset position encoding and the current feature vector, an input observation sequence is generated, and based on the pre-trained Transformer decoder and the input observation sequence, the robot's predicted action sequence within a preset time period is predicted.

[0009] The predicted action sequence is subjected to exponential decay weighted averaging to determine the robot's target action within the preset time period.

[0010] Secondly, this application also provides a robot motion prediction device, the device comprising:

[0011] The current feature vector generation module generates the current feature vector using a preset feature extraction network and the robot's current observation data;

[0012] The action sequence prediction module generates an input observation sequence based on a preset position encoding and the current feature vector, and predicts the robot's action sequence within a preset time period based on a pre-trained Transformer decoder and the input observation sequence.

[0013] The target action prediction module determines the robot's target action within the preset time period by performing an exponentially decaying weighted average on the predicted action sequence.

[0014] Thirdly, this application also provides a computer device, the computer device including a memory and a processor; the memory is used to store a computer program; the processor is used to execute the computer program and, when executing the computer program, implement the robot motion prediction method as described above.

[0015] Fourthly, this application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, causes the processor to implement the robot motion prediction method described above.

[0016] This application discloses a robot motion prediction method, apparatus, computer device, and storage medium. The robot motion prediction method includes generating a current feature vector using a preset feature extraction network and the robot's current observation data; generating an input observation sequence based on a preset position code and the current feature vector; predicting a sequence of robot motions within a preset time period using a pre-trained Transformer decoder and the input observation sequence; and performing exponentially decaying weighted averaging on the predicted motion sequence to determine the robot's target motion within the preset time period. Through this method, this application transforms the robot's multimodal observation data into a unified feature vector using a preset feature extraction network, preserving key information in the observation data. The pre-trained Transformer decoder fully captures the temporal dependencies and contextual information in the observation data, accurately predicting the robot's motion sequence within the preset time period. The exponentially decaying weighted averaging on the predicted motion sequence further smooths the motion sequence, reduces prediction instability, and makes the final determined target motion more accurate, thus helping to improve the robot's performance and success rate when performing tasks. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a schematic flowchart of a robot motion prediction method provided in an embodiment of this application;

[0019] Figure 2 A schematic block diagram of a robot motion prediction device provided for an embodiment of this application;

[0020] Figure 3 A schematic block diagram of the structure of a computer device provided for an embodiment of this application. Detailed Implementation

[0021] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0022] The flowchart shown in the attached diagram is for illustrative purposes only and does not necessarily include all content and operations / steps, nor does it necessarily have to be performed in the described order. For example, some operations / steps can be broken down, combined, or partially merged, so the actual execution order may change depending on the actual situation.

[0023] It should be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of the application. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0024] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0025] This application provides a robot motion prediction method, apparatus, computer device, and storage medium. The robot motion prediction method can be applied to robots. Through a pre-set feature extraction network, it transforms the robot's multimodal observation data into a unified feature vector, preserving key information from the observation data. A pre-trained Transformer decoder fully captures the temporal dependencies and contextual information in the observation data, accurately predicting the robot's motion sequence within a preset time period. An exponentially decaying weighted average is applied to the predicted motion sequence, further smoothing the sequence, reducing prediction instability, and making the final determined target motion more accurate. This helps improve the robot's performance and success rate when performing tasks.

[0026] The following detailed description of some embodiments of this application is provided in conjunction with the accompanying drawings. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0027] Please see Figure 1 , Figure 1 This is a schematic flowchart illustrating a robot motion prediction method provided in an embodiment of this application. This robot motion prediction method can be applied to robots. Through a pre-set feature extraction network, the robot's multimodal observation data is transformed into a unified feature vector, preserving key information in the observation data. A pre-trained Transformer decoder is used to fully capture the temporal dependencies and contextual information in the observation data, accurately predicting the robot's motion sequence within a preset time period. An exponentially decaying weighted average is applied to the predicted motion sequence, further smoothing the sequence and reducing prediction instability, resulting in more accurate final target motions and improving the robot's performance and success rate when performing tasks.

[0028] like Figure 1 As shown, the robot motion prediction method specifically includes steps S10 to S30.

[0029] Step S10: Generate the current feature vector using the preset feature extraction network and the robot's current observation data;

[0030] Specifically, robot observation data typically includes image data and state data. Image data comes from multiple cameras, while state data includes low-dimensional state information such as joint angles and velocities. Image data includes current and historical image data acquired from the robot's cameras, while state data includes current and historical low-dimensional state data, such as joint angles and velocities, acquired from the robot's sensors.

[0031] Image data preprocessing, including resolution adjustment and normalization, ensures that the image data meets the input requirements of the predefined feature extraction network, scaling all images to the same resolution. For example, if ResNet18 is used as the visual backbone network, images can be scaled to 224x224 pixels. Image data normalization is typically performed using the mean and standard deviation to improve the training stability and performance of the model.

[0032] If the robot is equipped with multiple cameras, after scaling all the images to the same resolution, they are stitched together along the channel dimension. Assuming that the image from each camera is scaled to a dimension of C×H×W, if there are N cameras, the dimension of the stitched image will be N×C×H×W.

[0033] Step S20: Generate an input observation sequence based on the preset position encoding and the current feature vector, and predict the robot's predicted action sequence within a preset time period based on the pre-trained Transformer decoder and the input observation sequence;

[0034] Specifically, positional encoding is a vector used to provide the model with the temporal or spatial location information of each element in the sequence. Positional encoding is usually generated using sine and cosine functions, and its dimension is the same as that of the feature vector. The positional encoding matrix is ​​generated according to the preset time step T and feature dimension.

[0035] Create an empty input sequence to store feature vectors with positional encoding, and then fill the input sequence with these positional-encoded feature vectors in chronological order. If historical observation data needs to be considered, the corresponding feature vectors can be obtained from the historical observation queue and added to the input sequence in chronological order. Ensure that the Transformer decoder has been pre-trained and loaded into the system. The pre-trained decoder can process sequences of feature vectors with positional encoding and predict future action sequences. Input the constructed input observation sequence into the Transformer decoder for forward propagation. The decoder will predict the robot's action sequence within a preset time period in the future based on the feature vectors and positional information in the input sequence.

[0036] The predicted action sequence is extracted from the output of the Transformer decoder. The predicted action sequence contains the robot's action predictions for multiple future time steps. The predicted action sequence is then subjected to necessary post-processing, such as checking whether the action conforms to physical constraints and whether it is within the robot's action range.

[0037] Step S30: Perform exponential decay weighted averaging on the predicted action sequence to determine the robot's target action within the preset time period.

[0038] Specifically, select a decay factor α (0 < α < 1), which determines the decay rate of the weights over time. A smaller α value will result in higher weights for recent predictions, while a larger α value will result in a more uniform weight distribution. Confirm the time window length T of the predicted action sequence, which is the number of time steps contained in the predicted action sequence.

[0039] For each time step t (from 0 to T-1) in the predicted action sequence, calculate its corresponding weight W. t The weight calculation formula is: W t =α t Thus, the closer a time step is to the current time, the higher its weight. For each motion component in the predicted motion sequence (e.g., the angle or velocity of each joint), a weighted average is calculated using the corresponding weights.

[0040] All calculated target motion components are combined into the final target motion vector. The target motion is then checked to see if it conforms to the robot's physical constraints and task requirements. If necessary, adjustments or restrictions are made.

[0041] In this specific embodiment, two parameters are provided to optimize performance during actual deployment:

[0042] 1. `predict_every_n`: The model can perform a complete inference only once every n time steps. In the middle n-1 steps, the model reuses the previous prediction and shifts the time steps. This can improve inference speed by several times, especially suitable for scenarios with limited hardware resources.

[0043] 2. `return_every_n`: The model does not return the first action in the predicted sequence, but instead returns the nth action. This makes the robot move faster, sacrificing some accuracy for higher task completion efficiency, and is suitable for tasks that are sensitive to completion time.

[0044] During the training phase, a dataset containing multimodal demonstration data is loaded. Random cropping is applied to the images, Gaussian noise is superimposed on the state vectors, and the historical observation steps are randomly jittered to achieve temporal augmentation. The visual stream is processed through a ResNet18 with LoRA, and the state stream through a linear layer. Both types of data are projected onto a 128-dimensional feature space and concatenated into a sequence. This sequence is fed into an 8-head, 8-layer Transformer containing only a decoder, which outputs consecutive actions for the next few steps at once. The sequence is then aligned with expert actions using an exponentially decaying weighted L1 loss. The AdamW optimizer and Cosine learning rate scheduler are used to update the LoRA and decoder parameters, iterating until the verification loss converges, completing a single-stage end-to-end training.

[0045] During the inference phase, the robot first collects real-time observation data, including images and state information, through its sensors (such as cameras and encoders). After preprocessing, the system maintains a fixed-length historical observation queue and constructs an input observation sequence based on a preset hybrid temporal strategy (combining recent observations and sparse historical observations). This input observation sequence is fed into a trained model for forward propagation, outputting a predicted action sequence for the next time window. The model performs frame-skipping inference based on the action sequence prediction parameters to improve inference speed. After obtaining the predicted action sequence, it undergoes exponential decay weighted averaging to generate the final action command. Finally, the robot selects the action to be executed based on the `predict_every_n` parameter and sends the action command to the robot controller to drive the robot to perform the corresponding action. The entire inference process aims to generate and execute action commands quickly and efficiently to achieve real-time robot control and task execution.

[0046] This embodiment discloses a robot motion prediction method, apparatus, computer device, and storage medium. The robot motion prediction method includes generating a current feature vector using a preset feature extraction network and the robot's current observation data; generating an input observation sequence based on a preset position code and the current feature vector; predicting a sequence of predicted robot motions within a preset time period using a pre-trained Transformer decoder and the input observation sequence; and performing exponentially decaying weighted averaging on the predicted motion sequence to determine the robot's target motion within the preset time period. Through this method, this application transforms the robot's multimodal observation data into a unified feature vector using a preset feature extraction network, preserving key information in the observation data. The pre-trained Transformer decoder fully captures the temporal dependencies and contextual information in the observation data, accurately predicting the robot's motion sequence within the preset time period. The exponentially decaying weighted averaging on the predicted motion sequence further smooths the motion sequence, reduces prediction instability, and makes the final determined target motion more accurate, thus helping to improve the robot's performance and success rate when performing tasks.

[0047] based on Figure 1 In the illustrated embodiment, step S10 includes:

[0048] Collect the robot's current image data and current state data;

[0049] Specifically, the robot's camera collects current image data in real time. It ensures that the resolution and format of the image data meet the requirements of the preset feature extraction network. For example, if ResNet18 is used as the visual backbone network, the image data should be adjusted to 224x224 pixels.

[0050] The robot's sensors (such as encoders and IMUs) collect real-time status data, including joint angles, velocity, and acceleration. This ensures the completeness and accuracy of the status data.

[0051] The current image features of the current image data are extracted through the visual backbone network built into the preset feature extraction network;

[0052] Specifically, a pre-trained visual backbone network (such as ResNet18) is loaded, and some layers are fine-tuned or frozen as needed. The acquired image data is then pre-processed, including operations such as resolution adjustment and normalization. For example, the image is scaled to 224x224 pixels and normalized. The pre-processed image data is then input into the visual backbone network, and the output of its fully connected layers is used as the current image features. For example, the output dimension of the fully connected layers in ResNet18 is 512.

[0053] The current state features of the current state data are extracted through the fully connected layer built into the preset feature extraction network;

[0054] Specifically, a fully connected layer is set up, with its input dimension matching the dimension of the state data and its output dimension being a preset dimension. The collected state data is standardized to ensure that its numerical range is consistent with that of the training phase. The standardized state data is then input into the fully connected layer, and the output is obtained as the current state feature.

[0055] The current image features and the current state features are projected onto a preset dimension, and then concatenated to generate the current feature vector.

[0056] Specifically, the extracted current image features and current state features are projected onto preset dimensions. If the output dimension of the visual backbone network is already the preset dimension, no additional projection is needed; otherwise, a linear layer is used for projection.

[0057] The projected current image features and current state features are concatenated along the feature dimension to generate the final current feature vector. The concatenation operation can be a simple vector join or a weighted summation operation.

[0058] In a specific embodiment, the robot's camera and sensors collect current image and state data in real time. The collected image data needs to be preprocessed, such as adjusting resolution and normalizing, to meet the input requirements of the visual backbone network (e.g., ResNet18). The preprocessed image data is then input into the visual backbone network to extract current image features. Simultaneously, the collected state data, after standardization, is input into a fully connected layer to extract current state features. The extracted image and state features are projected onto preset dimensions to ensure they have the same dimensionality. The projected image and state features are then concatenated along the feature dimension to generate the current feature vector.

[0059] based on Figure 1 In the illustrated embodiment, the steps preceding step S20 include:

[0060] Read a preset number of frames of sparse historical observation data from the robot's cache queue;

[0061] Specifically, the robot maintains a fixed-length historical observation data cache queue, which stores the most recent observation data, including image and status data. Based on task requirements, parameters for sparse historical observations are set, including the sparse interval `lookback_steps` and the sparse quantity `num_sparse_samples`. A preset number of historical observation data, `num_sparse_samples`, are read from the cache queue at the set sparse interval `lookback_steps`.

[0062] Based on the task requirements, determine the total length T of the time series. The total length T includes the number of time steps for the current observation data and the sparse historical observation data. Create a time series array to store the location information of the time steps. The array has a length of T, and each element represents the location index of a time step.

[0063] The preset location code is determined based on the preset time series, the current observation data, and the sparse historical observation data.

[0064] Specifically, positional encoding is typically generated using sine and cosine functions, with the same dimension as the feature vector. The positional encoding matrix is ​​generated based on the length T of the time series and the feature dimension.

[0065] The current observation data's position index in the time series is typically 0. Its position index is determined based on the time steps of the sparse historical observation data. For example, if the time steps of the sparse historical observation data are t1, t2, ..., tnum_sparse_samples, then their position indices are t1, t2, ..., tnum_sparse_samples respectively.

[0066] Based on the location indices of current observation data and sparse historical observation data, corresponding location coding vectors are extracted from the location coding matrix. These location coding vectors are then arranged in chronological order to form the final location coding sequence.

[0067] Specifically, a preset number of sparse historical observation data frames are read from the robot's buffer queue. Based on the set sparse interval `lookback_steps` and sparse quantity `num_sparse_samples`, historical observation data is read from the buffer queue at intervals. The total length `T` of the time series is determined according to task requirements, and a time series array is initialized to store the position information of each time step. A position encoding matrix is ​​generated using sine and cosine functions. Based on the time steps of the current observation data and the sparse historical observation data, the corresponding position encoding vectors are extracted from the position encoding matrix and arranged in chronological order to form the final position encoding sequence.

[0068] In a specific embodiment, determining the preset location code based on the preset time series, the current observation data, and the sparse historical observation data includes:

[0069] The historical image data and the current image data are fused to generate fused image features;

[0070] Specifically, read historical images and status data from the cache queue and retain their original timestamps; add the latest timestamp to the most recently collected current images and status data to ensure that all data are comparable under the same time base; if the resolution of historical images is different from that of current images, scale them to the same size; if the dimension of the status vector changes with version upgrades, fill the dimension with zeros or linear mapping.

[0071] The historical state data and the current state data are fused to generate a fused state feature;

[0072] Specifically, historical frames are sequentially fed into the same visual backbone network (ResNet18 with LoRA) as the training phase to obtain the historical image feature sequence, and the latest image frame is fed into the same network to obtain the current image features.

[0073] The historical state vector is projected through a fully connected layer to obtain the historical state feature sequence, and the latest state vector is passed through the same fully connected layer to obtain the current state feature.

[0074] The fused image features and the fused state features are concatenated according to the preset time series to generate a fused feature vector sequence;

[0075] Specifically, a weighted summation method is used to fuse historical and current image features, as well as to fuse historical and current state features.

[0076] Assign position indices to each fused feature in the fused feature vector sequence to determine the preset position code.

[0077] Specifically, the fused image features and state features are concatenated along a preset feature dimension to obtain a fused feature vector. After serializing multiple fused feature vectors, a fused feature vector sequence is generated. Each fused feature in the fused feature vector sequence is assigned a position index according to its actual time step in a preset time series.

[0078] In a specific embodiment, after aligning and unifying the format by timestamp from the cache queue, historical image / state data and the latest observation are retrieved. The same feature extraction network is used to extract historical and current image features and state features respectively. The historical and current features are fused by weighted summation to obtain more robust fused image and fused state features. These two types of fused features are then concatenated into a fused feature vector sequence in chronological order. Finally, according to their true position index in the preset time series, a position code is generated using sine and cosine functions and added to the corresponding feature. The final output is a fused feature vector sequence that contains both spatial-state semantics and temporal position information, which is then used by the Transformer decoder for action prediction.

[0079] In a specific embodiment, step S20 includes:

[0080] The input observation sequence is used as the memory input of the pre-trained Transformer decoder, wherein the pre-trained Transformer decoder includes only at least one decoder structure.

[0081] Specifically, the fused feature vector sequence carrying position encoding is arranged into a tensor by batch and sequence dimension. In the Transformer containing only the decoder structure, this tensor is directly used as memory input for subsequent self-attention modules to call.

[0082] The pre-trained Transformer decoder extracts the temporal features of the memory input and outputs the continuous action sequence within the preset time period in parallel according to each temporal feature.

[0083] Specifically, a linear projection is performed on the memory input to obtain the query, key, and value. Long-range temporal dependencies are captured by scaling dot product attention. After multi-head parallel computation, the data are concatenated and linearly mapped to obtain temporal features.

[0084] Each of the continuous action sequences is subjected to feature transformation to generate a linear output layer mapping, and the linear output layer is mapped to the predicted action sequence.

[0085] Specifically, the fused feature vector sequence containing location information is directly used as the memory input of the Transformer, which only contains the decoder. The decoder extracts the temporal features of the entire sequence at once through multi-head self-attention, and then refines them through a multi-layer feedforward network. The high-order temporal features output by the last layer are used to map each time step to the action space in parallel to obtain the original action tensor. The feature transformation is completed through a linear output layer, outputting a smooth and continuous predicted action sequence for subsequent exponentially decaying weighted averaging or direct execution. This process has no encoder, no loops, and is completely parallel.

[0086] based on Figure 1In the illustrated embodiment, step S30 includes:

[0087] A preset attenuation coefficient is assigned to the predicted action corresponding to each time step within the preset time period, wherein the predicted action sequence includes each predicted action, and the preset attenuation coefficient is 0-1;

[0088] The target action is determined by performing a weighted average of each predicted action based on the preset attenuation coefficient.

[0089] Specifically, for the predicted action sequence at several future time steps, a decay coefficient between 0 and 1 is assigned to each time step, with smaller weights for later steps. These coefficients are then normalized so that their sum equals 1.

[0090] Each predicted action is weighted and averaged according to its corresponding weight, and the gradually weakening contributions over time are integrated into a target action that is both smooth and responsive. Optional smoothing filtering and physical limiting are applied to the result to ensure that the target action not only conforms to robot dynamics constraints but also has good execution stability.

[0091] Based on any of the above embodiments, in this embodiment, step S10 is followed by:

[0092] If the resolution of the current image data is different, the resolution of each current image data will be converted to a preset resolution.

[0093] Specifically, the robot captures the latest frame image in real time through the interfaces of each camera, records its original resolution and channel order, and prepares scaling parameters according to the preset resolution (such as 224×224 or 256×256).

[0094] When the resolution of the images captured by the robot's various cameras is inconsistent, all raw frames are read, and the corresponding scaling ratio is immediately calculated with a preset resolution (e.g., 224×224) as the target. Each image is then scaled without distortion using bilinear interpolation, and channel order adjustments are performed when necessary to normalize pixel values ​​to the same range as during training. The images with unified resolution are then packaged into tensors.

[0095] Please see Figure 2 , Figure 2 This is a schematic block diagram of a robot motion prediction device provided in an embodiment of this application. The robot motion prediction device is used to execute the aforementioned robot motion prediction method. The robot motion prediction device can be configured on a server.

[0096] like Figure 2 As shown, the robot motion prediction device 400 includes:

[0097] The current feature vector generation module 410 is used to generate the current feature vector through a preset feature extraction network and the robot's current observation data;

[0098] The action sequence prediction module 420 is used to generate an input observation sequence based on a preset position encoding and the current feature vector, and to predict the robot's predicted action sequence within a preset time period based on a pre-trained Transformer decoder and the input observation sequence.

[0099] The target action prediction module 430 is used to determine the target action of the robot within the preset time period by performing exponential decay weighted averaging on the predicted action sequence.

[0100] Furthermore, the current feature vector generation module 410 includes:

[0101] The current data acquisition submodule is used to acquire the current image data and the current state data of the robot;

[0102] The current image feature extraction submodule is used to extract the current image features of the current image data through the visual backbone network built into the preset feature extraction network;

[0103] The current state feature extraction submodule is used to extract the current state features of the current state data through the fully connected layer built into the preset feature extraction network;

[0104] The current feature vector generation submodule is used to project the current image features and the current state features onto a preset dimension, and to concatenate the current image features and the current state features to generate the current feature vector.

[0105] Furthermore, the robot motion prediction device 400 includes:

[0106] The sparse historical observation data reading module is used to read a preset number of frames of sparse historical observation data from the robot's cache queue;

[0107] The preset location code determination module is used to determine the preset location code based on the preset time series, the current observation data, and the sparse historical observation data.

[0108] Furthermore, the preset position encoding determination module includes:

[0109] The image feature generation submodule is used to fuse the historical image data and the current image data to generate fused image features;

[0110] The fusion state feature generation submodule is used to fuse the historical state data and the current state data to generate fusion state features;

[0111] The fusion feature vector generation submodule is used to sequentially concatenate the fused image features and the fused state features according to the preset time series to generate a fused feature vector sequence;

[0112] The preset position encoding determination submodule is used to assign position indices to each fused feature in the fused feature vector sequence and determine the preset position encoding.

[0113] Furthermore, the action sequence prediction module 420 includes:

[0114] The memory input determination submodule is used to use the input observation sequence as the memory input of the pre-trained Transformer decoder, wherein the pre-trained Transformer decoder includes only at least one decoder structure.

[0115] The continuous action sequence output submodule is used to extract the temporal features of the memory input through the pre-trained Transformer decoder, and output the continuous action sequence within the preset time period in parallel according to each temporal feature;

[0116] The predicted action sequence mapping submodule is used to perform feature transformation on each of the continuous action sequences, generate a linear output layer mapping, and map the linear output layer to the predicted action sequence.

[0117] Furthermore, the target action prediction module 430 includes:

[0118] A preset attenuation coefficient allocation submodule is used to allocate a preset attenuation coefficient to the predicted action corresponding to each time step within the preset time period, wherein the predicted action sequence includes each predicted action, and the preset attenuation coefficient is 0-1;

[0119] The target action determination submodule is used to perform a weighted average of each predicted action according to the preset attenuation coefficient to determine the target action.

[0120] Furthermore, the robot motion prediction device 400 includes:

[0121] The resolution conversion module is used to convert the resolution of each current image data to a preset resolution if the resolutions of the current image data are different.

[0122] It should be noted that those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the above-described apparatus and modules can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0123] The aforementioned device can be implemented as a computer program, which can be used in, for example... Figure 3 It runs on the computer device shown.

[0124] Please see Figure 3 , Figure 3 This is a schematic block diagram illustrating the structure of a computer device according to an embodiment of this application. The computer device may be a server.

[0125] See Figure 3 The computer device includes a processor, memory, and network interface connected via a system bus, wherein the memory may include non-volatile storage media and internal memory.

[0126] Non-volatile storage media can store operating systems and computer programs. These computer programs include program instructions that, when executed, cause the processor to perform any robot motion prediction method.

[0127] The processor provides computing and control capabilities, supporting the operation of the entire computer device.

[0128] Internal memory provides an environment for the execution of computer programs stored in non-volatile storage media. When these computer programs are executed by a processor, the processor can perform any robot motion prediction method.

[0129] This network interface is used for network communication, such as sending assigned tasks. Those skilled in the art will understand that... Figure 3 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0130] It should be understood that the processor can be a Central Processing Unit (CPU), but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among these, a general-purpose processor can be a microprocessor or any conventional processor.

[0131] In one embodiment, the processor is configured to run a computer program stored in memory to perform the following steps:

[0132] The current feature vector is generated using a pre-defined feature extraction network and the robot's current observation data.

[0133] Based on the preset position encoding and the current feature vector, an input observation sequence is generated, and based on the pre-trained Transformer decoder and the input observation sequence, the robot's predicted action sequence within a preset time period is predicted.

[0134] The predicted action sequence is subjected to exponential decay weighted averaging to determine the robot's target action within the preset time period.

[0135] In one embodiment, a current feature vector is generated using a preset feature extraction network and the robot's current observation data, for the purpose of:

[0136] Collect the robot's current image data and current state data;

[0137] The current image features of the current image data are extracted through the visual backbone network built into the preset feature extraction network;

[0138] The current state features of the current state data are extracted through the fully connected layer built into the preset feature extraction network;

[0139] The current image features and the current state features are projected onto a preset dimension, and then concatenated to generate the current feature vector.

[0140] In one embodiment, before generating an input observation sequence based on a preset position encoding and the current feature vector, and predicting the robot's predicted action sequence within a preset time period based on a pre-trained Transformer decoder and the input observation sequence, the following steps are taken:

[0141] Read a preset number of frames of sparse historical observation data from the robot's cache queue;

[0142] The preset location code is determined based on the preset time series, the current observation data, and the sparse historical observation data.

[0143] In one embodiment, the preset location code is determined based on a preset time series, the current observation data, and the sparse historical observation data, for the purpose of:

[0144] The historical image data and the current image data are fused to generate fused image features;

[0145] The historical state data and the current state data are fused to generate a fused state feature;

[0146] The fused image features and the fused state features are concatenated according to the preset time series to generate a fused feature vector sequence;

[0147] Assign position indices to each fused feature in the fused feature vector sequence to determine the preset position code.

[0148] In one embodiment, an input observation sequence is generated based on a preset position code and the current feature vector, and a predicted action sequence of the robot within a preset time period is predicted based on a pre-trained Transformer decoder and the input observation sequence, for the purpose of:

[0149] The input observation sequence is used as the memory input of the pre-trained Transformer decoder, wherein the pre-trained Transformer decoder includes only at least one decoder structure.

[0150] The pre-trained Transformer decoder extracts the temporal features of the memory input and outputs the continuous action sequence within the preset time period in parallel according to each temporal feature.

[0151] Each of the continuous action sequences is subjected to feature transformation to generate a linear output layer mapping, and the linear output layer is mapped to the predicted action sequence.

[0152] In one embodiment, the predicted action sequence is subjected to an exponentially decaying weighted average to determine the robot's target action within the preset time period, for the purpose of:

[0153] A preset attenuation coefficient is assigned to the predicted action corresponding to each time step within the preset time period, wherein the predicted action sequence includes each predicted action, and the preset attenuation coefficient is 0-1;

[0154] The target action is determined by performing a weighted average of each predicted action based on the preset attenuation coefficient.

[0155] In one embodiment, after generating the current feature vector using a preset feature extraction network and the robot's current observation data, it is used to achieve:

[0156] If the resolution of the current image data is different, the resolution of each current image data will be converted to a preset resolution.

[0157] The embodiments of this application also provide a computer-readable storage medium storing a computer program, the computer program including program instructions, and the processor executing the program instructions to implement any of the robot motion prediction methods provided in the embodiments of this application.

[0158] The computer-readable storage medium may be an internal storage unit of the computer device described in the foregoing embodiments, such as the hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, SmartMedia Card (SMC), Secure Digital (SD) card, or Flash Card equipped on the computer device.

[0159] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for predicting robot actions, characterized in that, include: The current feature vector is generated using a pre-defined feature extraction network and the robot's current observation data. Based on the preset position encoding and the current feature vector, an input observation sequence is generated, and based on the pre-trained Transformer decoder and the input observation sequence, the robot's predicted action sequence within a preset time period is predicted. The predicted action sequence is subjected to exponential decay weighted averaging to determine the robot's target action within the preset time period.

2. The robot motion prediction method according to claim 1, characterized in that, The current observation data includes current image data and current state data. The process of generating a current feature vector from the current observation data using a preset feature extraction network and the robot includes: Collect the robot's current image data and current state data; The current image features of the current image data are extracted through the visual backbone network built into the preset feature extraction network; The current state features of the current state data are extracted through the fully connected layer built into the preset feature extraction network; The current image features and the current state features are projected onto a preset dimension, and then concatenated to generate the current feature vector.

3. The robot motion prediction method according to claim 2, characterized in that, Before generating an input observation sequence based on a preset position encoding and the current feature vector, and predicting the robot's predicted action sequence within a preset time period based on a pre-trained Transformer decoder and the input observation sequence, the process includes: Read a preset number of frames of sparse historical observation data from the robot's cache queue; The preset location code is determined based on the preset time series, the current observation data, and the sparse historical observation data.

4. The robot motion prediction method according to claim 3, characterized in that, The sparse historical observation data includes historical image data and historical state data. Determining the preset location code based on the preset time series, the current observation data, and the sparse historical observation data includes: The historical image data and the current image data are fused to generate fused image features; The historical state data and the current state data are fused to generate a fused state feature; The fused image features and the fused state features are concatenated according to the preset time series to generate a fused feature vector sequence; Assign position indices to each fused feature in the fused feature vector sequence to determine the preset position code.

5. The robot motion prediction method according to claim 3, characterized in that, The step of generating an input observation sequence based on a preset position code and the current feature vector, and predicting the robot's predicted action sequence within a preset time period based on a pre-trained Transformer decoder and the input observation sequence, includes: The input observation sequence is used as the memory input of the pre-trained Transformer decoder, wherein the pre-trained Transformer decoder includes only at least one decoder structure. The pre-trained Transformer decoder extracts the temporal features of the memory input and outputs the continuous action sequence within the preset time period in parallel according to each temporal feature. Each of the continuous action sequences is subjected to feature transformation to generate a linear output layer mapping, and the linear output layer is mapped to the predicted action sequence.

6. The robot motion prediction method according to claim 1, characterized in that, The step of performing exponentially decaying weighted averaging on the predicted action sequence to determine the robot's target action within the preset time period includes: A preset attenuation coefficient is assigned to the predicted action corresponding to each time step within the preset time period, wherein the predicted action sequence includes each predicted action, and the preset attenuation coefficient is 0-1; The target action is determined by performing a weighted average of each predicted action based on the preset attenuation coefficient.

7. The robot motion prediction method according to any one of claims 1 to 6, characterized in that, The current observation data includes current image data. After generating the current feature vector using the current observation data from the preset feature extraction network and the robot, the process includes: If the resolution of the current image data is different, the resolution of each current image data will be converted to a preset resolution.

8. A robot motion prediction device, characterized in that, include: The current feature vector generation module is used to generate the current feature vector using a preset feature extraction network and the robot's current observation data; The action sequence prediction module is used to generate an input observation sequence based on a preset position encoding and the current feature vector, and to predict the robot's predicted action sequence within a preset time period based on a pre-trained Transformer decoder and the input observation sequence. The target action prediction module is used to determine the robot's target action within the preset time period by performing exponential decay weighted averaging on the predicted action sequence.

9. A computer device, characterized in that, The computer device includes a memory and a processor; The memory is used to store computer programs; The processor is configured to execute the computer program and, in executing the computer program, implement the robot motion prediction method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, causes the processor to implement the robot motion prediction method as described in any one of claims 1 to 7.

Images