A method and apparatus for generating a robot action

Abstract

The application provides a robot action generation method and device, the generation method comprises: in response to the robot receiving a target task, obtaining image data and text description information associated with the target task; based on the image data and the text description information, using a large language model provided by a supervised training based on a latent action model to determine latent action information and fusion visual representation information; compressing and splicing the image data, the latent action information and the fusion visual representation information to obtain control sequence information; denoising and splicing the state information corresponding to the robot and the control sequence information to generate action sequence information, so that the robot executes corresponding actions based on the action sequence information. Through the above method, the smoothness and continuity of the robot generated action are improved.

Description

Technical Field

[0001] This application relates to the field of robot motion control technology, and in particular to a method and apparatus for generating robot motion. Background Technology

[0002] Visual language action models for robotic applications can receive image and natural language command inputs and directly generate sequences of actions that can be executed by the robot, thereby achieving end-to-end control from robot perception to robot action execution. However, existing visual language action models cannot make good use of demonstration data from multiple robots and humans, resulting in poor cross-entity transfer capabilities of the model.

[0003] Furthermore, existing visual language action models used in robotics still have the following problems in practical applications: On the one hand, some visual language action models use lightweight decoders to generate actions, which reduces computational costs, but reduces the smoothness and coherence of the generated action trajectories when generating long sequences of continuous actions; on the other hand, some visual language action models encode all visual information through a large language model for high-level semantics before using it for action generation, which weakens or loses the fine spatial details and low-level visual features in the original image during the abstraction process, making the model insensitive to changes in visual input and reducing the robustness of the visual language action model to visual changes. Summary of the Invention

[0004] In view of this, the purpose of this application is to provide a method and apparatus for generating robot actions, which has the following beneficial technical effects.

[0005] When training a visual language action model, having the latent action model supervise the training of the large language model can improve the generalization of the large language model while enhancing the cross-entity transfer capability of the visual language action model.

[0006] The original image data, the fused visual representation information and potential action information output by the large language model are compressed and spliced ​​together to reduce the length of image and visual-related sequence information. This improves the sensitivity of the visual language action model to visual information while balancing the proportion of image data and text description information in the visual language action model.

[0007] The robot's state information and the compressed and spliced ​​control sequence information are denoised and spliced ​​to improve the smoothness and coherence of the robot's generated actions.

[0008] This application provides a method for generating robot actions, the method comprising: In response to the robot receiving a target task, it acquires the image data associated with the target task and the text description information of the target task; Based on the image data and the text description information, a pre-trained large language model is used to determine the latent action information and fused visual representation information corresponding to the target task; wherein, the large language model is trained under the supervision provided by the pre-trained latent action model. The image data, the potential action information, and the fused visual representation information are compressed and spliced ​​together to obtain the control sequence information corresponding to the target task. The state information and control sequence information of the robot are denoised and spliced ​​to generate action sequence information of the robot when processing the target task, so that the robot can perform corresponding actions based on the action sequence information.

[0009] Furthermore, the latent action model includes an encoder, a vector quantization module, and a decoder; the latent action model is trained through the following steps: For the training dataset corresponding to the training task, two target training images are selected from the training image set of the training dataset at a preset time interval, and the task instruction information corresponding to the target training images is determined in the training text set of the training dataset. Based on the target training image and the task instruction information, the preset action-independent vectors of the potential action model are learned and trained to determine the target action-independent vectors, so as to obtain the trained irrelevant potential action model. Based on the target training image and the target action-independent vector, the preset potential action vectors of the irrelevant potential action model are learned and trained to determine the target potential action vectors, so as to obtain the trained potential action model.

[0010] Furthermore, the step of learning and training the preset action-independent vectors of the latent action model based on the target training image and the task indication information to determine the target action-independent vectors, so as to obtain the trained action-independent latent action model, includes: The target training image, the task instruction information, and the preset action-independent vector are respectively input into the encoder to obtain the first task-independent action information corresponding to the training task output by the encoder; The first task-independent action information is input into the vector quantization module, and the vector quantization module queries the first task-independent codebook vector corresponding to the first task-independent action information in the preset task-independent potential action codebook. The first embedding vector corresponding to the target training image in the first frame, the text encoding vector corresponding to the task indication information, and the first task-independent codebook vector are respectively input into the decoder to obtain the first target embedding vector reconstructed from the target training image in the second frame by the decoder. Based on the second embedding vector and the first target embedding vector corresponding to the target training image in the second frame, calculate the first loss value corresponding to the action-independent vector; Based on the first loss value, the action-independent vector is iteratively trained to determine the target action-independent vector, so as to obtain the trained action-independent potential action model.

[0011] Furthermore, the step of learning and training the preset potential action vectors of the irrelevant potential action model based on the target training image and the target action-independent vector to determine the target potential action vectors, so as to obtain the trained potential action model, includes: The target training image, the target action-independent vector, and the preset potential action vector are respectively input into the encoder to obtain the task-related action information and the second task-independent action information corresponding to the training task output by the encoder. The task-related action information and the second task-independent action information are respectively input into the vector quantization module. The vector quantization module queries the task-related codebook vector corresponding to the task-related action information in the preset task-related potential action codebook, and queries the second task-independent codebook vector corresponding to the second task-independent action information in the preset task-independent potential action codebook. The first embedding vector, the task-related codebook vector, and the second task-independent codebook vector corresponding to the target training image in the first frame are respectively input into the decoder to obtain the second target embedding vector reconstructed from the target training image in the second frame by the decoder. Based on the second embedding vector and the second target embedding vector corresponding to the target training image in the second frame, calculate the second loss value corresponding to the potential action vector; Based on the second loss value, the potential action vector is iteratively trained to determine the target potential action vector, so as to obtain the trained potential action model.

[0012] Furthermore, the large language model is trained through the following steps based on supervision provided by a pre-trained latent action model: Two frames of the target training images are input into a pre-trained latent action model to obtain the latent action encoding information corresponding to the target training images output by the latent action model. The latent action encoding information is combined with the task instruction information, and the combined information is encoded using a preset text encoder to obtain a text embedding training vector. The target training image of the target frame is encoded using a preset image encoder to obtain an image embedding training vector. The image embedding training vector and the text embedding training vector are concatenated and input into the large language model to be trained to obtain the task-related action position information corresponding to the training task output by the large language model to be trained. Extract the target task-related action location information from the text embedding training vector, and calculate the third loss value based on the task-related action location information and the target task-related action location information; Based on the third loss value, iterative backpropagation is performed on the large language model to be trained to obtain the trained large language model.

[0013] Furthermore, the step of determining the potential action information and fused visual representation information corresponding to the target task based on the image data and the text description information using a pre-trained large language model includes: The text identifier sequence corresponding to the text description information is encoded and converted using a preset text encoder to obtain the text embedding vector corresponding to the text description information. The image data is then encoded and converted using a preset image encoder to obtain the image embedding vector corresponding to the image data. The text embedding vector and the image embedding vector are respectively input into a pre-trained large language model to obtain the latent action information and fused visual representation information corresponding to the target task output by the large language model.

[0014] Furthermore, the step of compressing and concatenating the image data, the latent action information, and the fused visual representation information to obtain the control sequence information corresponding to the target task includes: The image embedding vector corresponding to the image data is concatenated with the fused visual representation information to obtain fused image information, and the fused image information and the potential action information are respectively input into a preset multi-head attention pooler; The multi-head attention pooler compresses the fused image information to obtain visual feature information, and then concatenates the visual feature information with the potential action information to obtain the control sequence information corresponding to the target task.

[0015] Furthermore, the step of denoising and concatenating the state information and control sequence information corresponding to the robot to generate the action sequence information corresponding to the robot when processing the target task includes: Obtain the state information corresponding to the robot, and determine the state embedding information corresponding to the state information; The state embedding information is concatenated with the control sequence information to obtain action state concatenation embedding information, and the action state concatenation embedding information is input into a pre-trained stream matching action decoder. Each of the multiple latent variable transformation modules set by the stream matching action decoder sequentially performs denoising and splicing processing on the action state splicing embedding information according to the setting order corresponding to the latent variable transformation modules, generating action sequence information corresponding to the robot when processing the target task; wherein, the input of each latent variable transformation module is the output of the latent variable transformation module corresponding to the previous setting order of the latent variable transformation module.

[0016] This application also provides a robot motion generation device, the generation device comprising: The data acquisition module is used to acquire image data associated with the target task and text description information corresponding to the target task in response to the robot receiving the target task; The fusion prediction module is used to determine the latent action information and fused visual representation information corresponding to the target task based on the image data and the text description information using a pre-trained large language model; wherein, the large language model is trained under the supervision provided by the pre-trained latent action model; The compression and stitching module is used to compress and stitch together the image data, the potential action information and the fused visual representation information to obtain the control sequence information corresponding to the target task. The motion generation module is used to denoise and splice the state information and control sequence information corresponding to the robot to generate motion sequence information corresponding to the robot when processing the target task, so that the robot can perform corresponding actions based on the motion sequence information.

[0017] Furthermore, when the fusion prediction module is used to determine the latent action information and fused visual representation information corresponding to the target task based on the image data and the text description information using a pre-trained large language model, the fusion prediction module is used to: The text identifier sequence corresponding to the text description information is encoded and converted using a preset text encoder to obtain the text embedding vector corresponding to the text description information. The image data is then encoded and converted using a preset image encoder to obtain the image embedding vector corresponding to the image data. The text embedding vector and the image embedding vector are respectively input into a pre-trained large language model to obtain the latent action information and fused visual representation information corresponding to the target task output by the large language model.

[0018] Furthermore, when the compression and stitching module compresses and stitches the image data, the latent action information, and the fused visual representation information to obtain the control sequence information corresponding to the target task, the compression and stitching module is used to: The image embedding vector corresponding to the image data is concatenated with the fused visual representation information to obtain fused image information, and the fused image information and the potential action information are respectively input into a preset multi-head attention pooler; The multi-head attention pooler compresses the fused image information to obtain visual feature information, and then concatenates the visual feature information with the potential action information to obtain the control sequence information corresponding to the target task.

[0019] Furthermore, when the action generation module performs denoising and splicing processing on the robot's corresponding state information and control sequence information to generate action sequence information corresponding to the robot's processing of the target task, the action generation module is used to: Obtain the state information corresponding to the robot, and determine the state embedding information corresponding to the state information; The state embedding information is concatenated with the control sequence information to obtain action state concatenation embedding information, and the action state concatenation embedding information is input into a pre-trained stream matching action decoder. Each of the multiple latent variable transformation modules set by the stream matching action decoder sequentially performs denoising and splicing processing on the action state splicing embedding information according to the setting order corresponding to the latent variable transformation modules, generating action sequence information corresponding to the robot when processing the target task; wherein, the input of each latent variable transformation module is the output of the latent variable transformation module corresponding to the previous setting order of the latent variable transformation module.

[0020] This application also provides an electronic device, including: a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, the steps of the robot motion generation method described above are performed.

[0021] This application also provides a computer-readable storage medium storing a computer program, which, when run by a processor, performs the steps of the robot motion generation method described above.

[0022] The robot motion generation method and apparatus provided in this application include: in response to the robot receiving a target task, acquiring image data associated with the target task and text description information of the target task; based on the image data and the text description information, determining potential motion information and fused visual representation information corresponding to the target task using a pre-trained large language model; wherein the large language model is trained under supervision provided by a pre-trained potential motion model; compressing and concatenating the image data, the potential motion information, and the fused visual representation information to obtain control sequence information corresponding to the target task; and performing denoising and concatenation processing on the robot's state information and the control sequence information to generate motion sequence information corresponding to the robot when processing the target task, so that the robot performs corresponding actions based on the motion sequence information.

[0023] Compared to existing methods for visual language action models applied in robots, this approach involves using a latent action model to supervise the training of a large language model during visual language action model training. This enhances both the generalization ability of the large language model and the cross-entity transfer capability of the visual language action model. The original image data, the fused visual representation information output by the large language model, and the latent action information are compressed and stitched together to reduce the length of image and visually related sequence information. This improves the sensitivity of the visual language action model to visual information while balancing the proportion of image data and textual description information within the model. Furthermore, denoising and stitching are performed on the robot's state information and the compressed and stitched control sequence information, improving the smoothness and coherence of the robot's generated actions.

[0024] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0025] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1This is a schematic diagram of a module flow for training a visual language action model, provided in an embodiment of this application. Figure 2 A flowchart illustrating a method for generating robot motions provided in an embodiment of this application; Figure 3 A schematic diagram of the structure of a robot motion generation device provided in an embodiment of this application; Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of this application. Based on the embodiments of this application, every other embodiment obtained by those skilled in the art without inventive effort falls within the scope of protection of this application.

[0028] Research has found that visual language action models for robotic applications can receive image and natural language command inputs and directly generate sequences of actions that can be executed by the robot, thereby achieving end-to-end control from robot perception to robot action execution. However, existing visual language action models cannot make good use of demonstration data from multiple robots and humans, resulting in poor cross-entity transfer capabilities of the models.

[0029] For example, GR00T is an open-source visual language action model. The latent action model in this model is used to generate the training target of the action decoder, rather than directly guiding the output of the large language model. This is not conducive to enabling the large model to perform high-level and abstract task planning. Moreover, the large language model and the action decoder are tightly coupled during training, which is not conducive to the transfer of cross-entity capabilities.

[0030] In addition, existing visual language action models used in robots still have the following problems in practical applications: On the one hand, some visual language action models use lightweight decoders to generate actions, which reduces the computational cost, but reduces the smoothness and coherence of the generated action trajectory when generating long sequences of continuous actions.

[0031] For example, the UniVLA model suffers from large model size, high memory consumption, and long inference latency. The UniVLA model uses a lightweight action decoder based on transformer, which has disadvantages in terms of the smoothness and continuity of generated actions.

[0032] On the other hand, some visual language action models encode all visual information into high-level semantics through a large language model before using it for action generation. This weakens or loses the fine spatial details and low-level visual features in the original image during the abstraction process, making the model insensitive to changes in visual input and reducing the robustness of the visual language action model to visual changes.

[0033] For example, the GR00T model does not incorporate the original image embeddings that have not been processed by the large language model into the action decoder. This results in the weakening or loss of fine spatial details and low-level visual features in the original image during the high-level semantic abstraction process. Consequently, the predicted robot actions are not sensitive to visual information. Specifically, when the position or shape of the target object changes, the action trajectory generated by the model may deviate and tend to execute fixed patterns common in pre-training, rather than dynamically adjusting based entirely on the current visual input.

[0034] Furthermore, the Igor model failed to effectively distinguish between task-related actions and task-irrelevant actions, resulting in the potential action vectors it generated being contaminated by irrelevant information such as camera shake, non-subject movement, lighting changes, or the appearance of background objects. This introduced noise and interference into subsequent policy learning, reducing the model's performance and generalization ability.

[0035] Based on this, embodiments of this application provide a method for generating robot actions. When training a visual language action model, a latent action model provides supervision for the training of a large language model. This can improve the generalization of the large language model while enhancing the cross-entity transfer capability of the visual language action model. The original image data, the fused visual representation information output by the large language model, and the latent action information are correspondingly compressed and spliced ​​to reduce the length of image and visually related sequence information. This improves the sensitivity of the visual language action model to visual information while balancing the proportion of image data and text description information in the visual language action model. The robot's state information and the compressed and spliced ​​control sequence information are denoised and spliced, improving the smoothness and coherence of the robot's generated actions.

[0036] In this embodiment of the application, the robot is equipped with a pre-trained visual language action model, which includes a large language model, a multi-head attention pooler, and a stream matching action decoder.

[0037] Here, while large language models excel at high-level semantic understanding and task planning, their attention mechanisms may overlook visual details of the scene, leading to a loss of spatial information. In contrast, raw image data retains complete and low-level visual features, such as precise object edges, positions, and textures. To address this issue, a multi-head attention pooler is employed to fuse the raw image data embeddings and the fused visual representation output by the large language model. This result is then used as a control sequence input to the stream matching action decoder, enabling it to access both high-level semantic information and low-level fine-grained visual information. This provides a dual reference for the action generation process, ensuring that the actions generated by the action decoder better reflect the visual changes within the scene.

[0038] The large language model is trained under supervision provided by a pre-trained latent action model; for example, the large language model may include any one of the large language models such as the Qwen model, the ChatGPT model, and the Deepseek model.

[0039] Here, the latent action model includes an encoder, a vector quantization module, and a decoder; during the training phase of the latent action model, unsupervised learning is performed on the encoder, vector quantization module, and decoder; while during the training phase of the large language model, the decoder is no longer used, but only the encoder and vector quantization module are used to output latent action encoding information for supervised training of the large language model.

[0040] Here, by introducing a latent action model, we can effectively distinguish between task-related actions and task-irrelevant actions. Then, we use the latent action model to guide the training of the large language model, so that the large language model can make full use of Internet-level data and has good cross-entity transfer capabilities.

[0041] Specifically, a two-stage training method is adopted for the latent action model. During the training process, learnable action-independent vectors and latent action vectors are introduced, so that the latent action model can output task-related latent action vectors from training images and videos of different entities. These task-related latent action vectors filter out interference such as camera shake present in the robot.

[0042] Furthermore, a pre-trained latent action model is used to supervise the training of the large language model, enabling the large language model to predict subsequent latent actions from a single video frame, thereby improving the cross-entity capability of the large language model.

[0043] Furthermore, the embodiments of this application employ a flow-matching-based action decoder. The input of the flow-matching action decoder is the control sequence information output by the multi-head attention pooling model, which corresponds to the robot's state information. The output of the flow-matching action decoder is decoded to obtain the action sequence information representing the robot's motor motion angle.

[0044] In one possible implementation of this application, the latent action model is trained through the following steps: S101. For the training dataset corresponding to the training task, select two target training images in the training image set of the training dataset at a preset time interval, and determine the task instruction information corresponding to the target training images in the training text set of the training dataset.

[0045] In this embodiment of the application, a training dataset is pre-constructed, which includes demonstration videos of various entities and their corresponding task instruction texts. The training dataset includes a training image set and a training text set corresponding to each entity.

[0046] Here, two consecutive frames of target training images are selected according to a preset time interval, so as to use the image continuity of the demonstration video to train the latent action model and the large language model.

[0047] S102. Based on the target training image and the task instruction information, the preset action-independent vectors of the potential action model are learned and trained to determine the target action-independent vectors, so as to obtain the trained action-independent potential action model.

[0048] In one possible implementation of this application, step S102 may include: S1021. Input the target training image, the task instruction information, and the preset action-independent vector into the encoder respectively to obtain the first task-independent action information corresponding to the training task output by the encoder.

[0049] Here, the first task-independent action information represents the task-independent action information encoded by the action-independent vectors in the action-independent vector stage of training the potential action model.

[0050] S1022. The first task-independent action information is input into the vector quantization module, and the vector quantization module queries the first task-independent codebook vector corresponding to the first task-independent action information in the preset task-independent potential action codebook.

[0051] Here, task-independent latent actioncodes represent preset vectors that do not depend on or target a low-dimensional, hidden representation space corresponding to a specific task.

[0052] S1023. The first embedding vector corresponding to the target training image of the first frame, the text encoding vector corresponding to the task indication information, and the first task-independent codebook vector are respectively input into the decoder to obtain the first target embedding vector reconstructed from the target training image of the second frame by the decoder.

[0053] In this step, the decoder reconstructs the original training data based on action-independent vectors, combining the processing results of the encoder and vector quantization module with the original first frame target training image and its task indication information.

[0054] S1024. Based on the second embedding vector corresponding to the target training image in the second frame and the first target embedding vector, calculate the first loss value corresponding to the action-independent vector.

[0055] S1025. Based on the first loss value, iteratively learn and train the action-independent vector to determine the target action-independent vector, so as to obtain the trained action-independent potential action model.

[0056] Here, the trained irrelevant latent action model is the irrelevant latent action model obtained by optimizing the learnable action-irrelevant vectors during the first stage of training of the latent action model.

[0057] S103. Based on the target training image and the target action-independent vector, the preset potential action vector of the irrelevant potential action model is trained to determine the target potential action vector, so as to obtain the trained potential action model.

[0058] In one possible implementation of this application, step S103 may include: S1031. Input the target training image, the target action-independent vector, and the preset potential action vector into the encoder respectively to obtain the task-related action information and the second task-independent action information corresponding to the training task output by the encoder.

[0059] Here, task-related action information represents the task-related action information encoded by the potential action vectors in the potential action vector stage of training the potential action model; the second task-independent action information represents the task-independent action information encoded by the action-independent vectors in the potential action vector stage of training the potential action model.

[0060] S1032. Input the task-related action information and the second task-independent action information into the vector quantization module respectively. The vector quantization module queries the task-related codebook vector corresponding to the task-related action information in the preset task-related potential action codebook, and queries the second task-independent codebook vector corresponding to the second task-independent action information in the preset task-independent potential action codebook.

[0061] Here, task-related potential action codes are represented by preset vectors that depend on or target a low-dimensional, hidden representation space corresponding to a specific task.

[0062] S1033. The first embedding vector, the task-related codebook vector, and the second task-independent codebook vector corresponding to the target training image of the first frame are respectively input into the decoder to obtain the second target embedding vector reconstructed from the target training image of the second frame by the decoder.

[0063] In this step, the decoder, based on the processing results of the encoder and vector quantization module, combined with the original first frame target training image and its task indication information, reconstructs the original training data based on target action-independent vectors and potential action vectors.

[0064] S1034. Based on the second embedding vector and the second target embedding vector corresponding to the target training image in the second frame, calculate the second loss value corresponding to the potential action vector.

[0065] S1035. Based on the second loss value, iteratively learn and train the potential action vector to determine the target potential action vector, so as to obtain the trained potential action model.

[0066] Here, the trained latent action model is the latent action model obtained by optimizing both the learnable action-independent vectors and the latent action vectors during the two-stage training process of the latent action model.

[0067] Furthermore, in one possible implementation of this application, the large language model is trained through the following steps based on supervision provided by a pre-trained latent action model: S104. Input the two frames of the target training images into the pre-trained latent action model to obtain the latent action encoding information corresponding to the target training images output by the latent action model.

[0068] Here, during the training of the large language model, the trained latent action model is frozen and used to provide supervision for the training of the large language model. The latent action encoding information output by the latent action model is the learning object of the latent action information predicted by the large language model, and is provided to the large language model as training supervision.

[0069] S105. Combine the potential action encoding information with the task instruction information, and use a preset text encoder to encode the combined information to obtain a text embedding training vector.

[0070] Here, the pre-defined text encoder converts the combined information into a sequence of identifiers, which is then encoded into a text embedding training vector.

[0071] S106. Encode the target training image of the target frame using a preset image encoder to obtain an image embedding training vector, and concatenate the image embedding training vector with the text embedding training vector and input it into the large language model to be trained to obtain the task-related action position information corresponding to the training task output by the large language model to be trained.

[0072] S107. Extract the target task-related action location information from the text embedding training vector, and calculate the third loss value based on the task-related action location information and the target task-related action location information.

[0073] S108. Based on the third loss value, perform iterative backpropagation on the large language model to be trained to obtain the trained large language model.

[0074] In this way, the latent action model is used to supervise the training of the large language model to be trained based on the latent action encoding information corresponding to the target training image. The task-related action position information output by the large language model to be trained is compared with the target task-related action position information in the training dataset to calculate the loss. Then, the large language model is trained by backpropagation based on the loss value.

[0075] Furthermore, in another possible implementation of this application, the step of pre-training the stream matching action decoder may include: During the training of the stream matching action decoder, the training dataset is input into the large language model to obtain the latent action training information and fused visual representation training information output by the large language model. The fused visual representation training information is concatenated with the original training image embedding in the training dataset and fed into a multi-head attention pooler for pooling. The fused image information is concatenated with task-related action information and used as a control sequence input into each latent variable transformation module in the stream matching action decoder to be trained to control action generation. The final generated action is compared with the real action in the training dataset, the loss is calculated and backpropagated, thus completing the training of the stream matching action decoder to be trained.

[0076] Specifically, the training objective of the stream matching action decoder is to learn the velocity field at each moment, thereby gradually transforming from noise to action. In order to learn this velocity field, a reference probability flow needs to be constructed first. This reference probability flow is the theoretically feasible probability flow for transforming from one distribution (such as a noise distribution) to another distribution (such as an action distribution). Thus, the reference velocity field at a certain moment can be obtained based on this reference probability flow. The stream matching action decoder is trained using this reference velocity field. The loss function for training the stream matching action decoder is shown below.

[0077] .

[0078] in, This represents the loss value of the loss function of the stream matching action decoder; Indicates time Uniform sampling is performed on [0,1]. Indicates the flow of probability from the reference in time. Sampling a sample from the distribution of time points ; The velocity field is output by the stream-matching action decoder. This is a preset reference velocity field.

[0079] In this way, after training the stream matching action decoder using a large amount of data in the training dataset, the stream matching action decoder learns how to transform the input from the initial distribution to the target distribution.

[0080] Based on the above training process, the trained visual language action model obtained by training a three-stage training method including a latent action model, a large language model, and a stream matching action decoder can fully utilize the cross-entity knowledge learned from data of different entities, extract the latent actions, eliminate irrelevant actions, and finally apply the knowledge learned by the model to the target robot entity.

[0081] Please see Figure 1 , Figure 1 This is a schematic diagram of a module flow for training a visual language action model, provided as an embodiment of this application. Figure 1 As shown, two consecutive frames of target training images are input into the latent action model S10 to obtain latent action encoding information output by the latent action model S10; the latent action encoding information is combined with task instruction information, and the combined information is encoded using a preset text encoder S20 to obtain a text embedding training vector; the target training image of the target frame is encoded using a preset image encoder S30 to obtain an image embedding training vector, and the image embedding training vector and the text embedding training vector are concatenated and input into the large language model to be trained S40 to obtain the task-related action position information corresponding to the training task output by the large language model to be trained; the original image embedding training vector and the fused visual representation information output by the large language model are fused using a multi-head attention pooler S20 to obtain a control sequence input into each latent variable transformation module S601 of the stream matching action decoder S60; the output of the stream matching action decoder S60 is decoded to obtain action sequence information representing the robot motor action angle.

[0082] Please see Figure 2 , Figure 2 This is a flowchart illustrating a method for generating robot actions according to an embodiment of this application. Figure 2 As shown in the embodiment of this application, the method for generating robot actions includes: S201. In response to the robot receiving the target task, the robot acquires the image data associated with the target task and the text description information of the target task.

[0083] Here, the target task may include the task to be performed by the robot and / or the task to be performed by a human, so that the robot can perform corresponding visual language action processing based on the training of the visual language action model to extract potential actions from different entities in advance.

[0084] The image data is collected by the image acquisition device set up on the robot, and the text description information is the text instructions issued to the robot by the operator controlling the robot.

[0085] S202. Based on the image data and the text description information, determine the potential action information and fused visual representation information corresponding to the target task using a pre-trained large language model.

[0086] The large language model is trained under supervision provided by a pre-trained latent action model.

[0087] In one possible implementation of this application, step S202 may include: S2021. Encode and convert the text identifier sequence corresponding to the text description information using a preset text encoder to obtain the text embedding vector corresponding to the text description information, and encode and convert the image data using a preset image encoder to obtain the image embedding vector corresponding to the image data.

[0088] In this application embodiment, the preset text encoder may include a Qwen text encoder, which includes a text segmenter and a text embedder; the preset image encoder may include a Siglip image encoder.

[0089] In this step, a text segmenter is used to convert the text description information into a corresponding text identifier sequence. Then, a text embedder is used to encode and convert the text identifier sequence to obtain the text embedding vector corresponding to the text description information.

[0090] Furthermore, the image encoder converts the image data into image frame codes and embeds the image frame codes into corresponding image embedding vectors, which capture the semantic information in the image data.

[0091] S2022. Input the text embedding vector and the image embedding vector into the large language model respectively to obtain the potential action information and fused visual representation information corresponding to the target task output by the pre-trained large language model.

[0092] In this step, the text embedding vector and the image embedding vector are respectively input into a large language model trained under the supervision provided by a pre-trained latent action model. The large language model then predicts the latent action information and fused visual representation information corresponding to the target task based on the text embedding vector and the image embedding vector.

[0093] S203. Compress and stitch together the image data, the potential action information and the fused visual representation information to obtain the control sequence information corresponding to the target task.

[0094] In this embodiment, a multi-head attention pooler preset in the visual language action model set by the robot is used to compress and stitch together image data, potential action information and fused visual representation information.

[0095] Among them, Multihead Attention Pooling (MAP) is a feature aggregation module based on the Transformer architecture. Its core structure consists of a set of learnable query vectors that interact with the keys and key-value pairs in the input sequence (e.g., a sequence formed by concatenating the fused visual representation information processed by a large language model with the image embedding vectors corresponding to the original image data) through a cross-attention mechanism.

[0096] During the forward propagation of the multi-head attention pooler, multi-head attention computation is performed in parallel. In this computation process, each attention head independently calculates the correlation weight between the query vector and the fused image information and potential action information of the input, and weights and aggregates the information to capture the semantic information of different subspaces in the input sequence. The outputs of all heads are concatenated, and after residual connection and layer normalization, they are fed into the multilayer perceptron for further nonlinear transformation, and then residual connection and layer normalization operations are performed again.

[0097] Here, the multi-head attention pooler can efficiently compress long sequence image information into a fixed number of compact vector representations, significantly reducing the sequence length while retaining key information, and stably fusing the length relationship between image information and potential action information.

[0098] In this way, the multi-head attention pooling module compresses the vector formed by concatenating the fused visual representation information output by the large language model and the image embedding vector corresponding to the original image data, so that its length is balanced with the potential action information output by the large semantic model. Then, it and the potential action information are input into the stream matching action decoder, thereby introducing the original image information that has not been processed by the large language model into the stream matching action decoder.

[0099] In one possible implementation of this application, step S203 may include: S2031. The image embedding vector corresponding to the image data is concatenated with the fused visual representation information to obtain fused image information, and the fused image information and the potential action information are respectively input into a preset multi-head attention pooler.

[0100] S2032. The multi-head attention pooler compresses the fused image information to obtain visual feature information, and then concatenates the visual feature information with the potential action information to obtain the control sequence information corresponding to the target task.

[0101] In this embodiment, the multi-head attention pooler compresses long sequences of fused image information into a shorter, information-dense feature sequence including visual feature information through a cross-attention mechanism; the control sequence information may include a length-balanced context sequence.

[0102] S204. The state information and control sequence information corresponding to the robot are denoised and spliced ​​to generate the action sequence information corresponding to the robot when processing the target task, so that the robot can perform the corresponding action based on the action sequence information.

[0103] In this embodiment, a pre-trained stream matching action decoder in the visual language action model set by the robot is used to perform denoising and splicing processing on the robot's corresponding state information and control sequence information.

[0104] The stream matching action decoder consists of multiple latent variable transformation modules. Each latent variable transformation module receives control sequence information including visual feature information and latent action information, denoises the noisy actions in the control sequence information, and then generates the final action sequence information by splicing them together.

[0105] Here, the core idea of ​​stream matching in the stream matching action decoder is to use a simple prior distribution. (For example, standard Gaussian noise) can be smoothly "flowed" into complex data distributions through a continuous transformation process. In (for example, the distribution of real robot movements), this transformation is a continuous process that changes with time steps; therefore, it can be viewed as a "flow" of data. This flow can be expressed by the following formula.

[0106] .

[0107] in, express Time conforms to distribution The point in the middle; This indicates that the initial time follows the prior distribution. The point in the middle.

[0108] Furthermore, by differentiating the above formula over time, we can obtain the following formula.

[0109] .

[0110] in, This represents the "velocity field" of the stream matching action decoder.

[0111] Thus, by employing a stream-matching-based action decoder, the smoothness and continuity of generated actions are superior to those of a lightweight action decoder based on transformers.

[0112] In one possible implementation of this application, step S204 may include: S2041. Obtain the state information corresponding to the robot, and determine the state embedding information corresponding to the state information.

[0113] In this embodiment, the input to the action decoder for stream matching is the state embedding information corresponding to the robot's state information.

[0114] S2042. The state embedding information is concatenated with the control sequence information to obtain action state concatenation embedding information, and the action state concatenation embedding information is input into a pre-trained stream matching action decoder.

[0115] In this step, the state embedding information is concatenated with the control sequence information to obtain the action state concatenation embedding information, and the action state concatenation embedding information is input into the pre-trained stream matching action decoder.

[0116] Here, the sequence information input to the stream matching motion decoder also includes preset noise that matches the shape of the robot motion encoding.

[0117] S2043. Each of the multiple latent variable transformation modules set by the stream matching action decoder performs noise reduction and splicing processing on the action state splicing and embedding information in the order of the settings corresponding to the latent variable transformation modules, thereby generating the action sequence information corresponding to the robot when processing the target task.

[0118] The input of each latent variable transformation module is the output of the latent variable transformation module corresponding to the previous setting order of that latent variable transformation module.

[0119] The robot action generation method provided in this application embodiment, during the training of the visual language action model, uses the latent action model to supervise the training of the large language model. This can improve the generalization of the large language model while enhancing the cross-entity transfer capability of the visual language action model. The original image data, the fused visual representation information output by the large language model, and the latent action information are correspondingly compressed and spliced ​​to reduce the length of image and visually related sequence information. While improving the sensitivity of the visual language action model to visual information, the proportion of image data and text description information in the visual language action model is balanced. The robot's state information and the compressed and spliced ​​control sequence information are denoised and spliced, improving the smoothness and coherence of the robot's generated actions.

[0120] Please see Figure 3 , Figure 3 This is a schematic diagram of the structure of a robot motion generation device provided in an embodiment of this application. Figure 3 As shown, the generating apparatus 300 includes: The data acquisition module 310 is used to acquire image data associated with the target task and text description information corresponding to the target task in response to the robot receiving the target task; The fusion prediction module 320 is used to determine the latent action information and fused visual representation information corresponding to the target task based on the image data and the text description information using a pre-trained large language model; wherein, the large language model is trained under the supervision provided by the pre-trained latent action model; The compression and stitching module 330 is used to compress and stitch the image data, the potential action information and the fused visual representation information to obtain the control sequence information corresponding to the target task. The motion generation module 340 is used to denoise and splice the state information and control sequence information corresponding to the robot to generate motion sequence information corresponding to the robot when processing the target task, so that the robot can perform corresponding actions based on the motion sequence information.

[0121] Furthermore, when the fusion prediction module 320 determines the potential action information and fused visual representation information corresponding to the target task based on the image data and the text description information using a pre-trained large language model, the fusion prediction module 320 is used to: The text identifier sequence corresponding to the text description information is encoded and converted using a preset text encoder to obtain the text embedding vector corresponding to the text description information. The image data is then encoded and converted using a preset image encoder to obtain the image embedding vector corresponding to the image data. The text embedding vector and the image embedding vector are respectively input into a pre-trained large language model to obtain the latent action information and fused visual representation information corresponding to the target task output by the large language model.

[0122] Furthermore, when the compression and stitching module 330 compresses and stitches the image data, the latent action information, and the fused visual representation information to obtain the control sequence information corresponding to the target task, the compression and stitching module 330 is used to: The image embedding vector corresponding to the image data is concatenated with the fused visual representation information to obtain fused image information, and the fused image information and the potential action information are respectively input into a preset multi-head attention pooler; The multi-head attention pooler compresses the fused image information to obtain visual feature information, and then concatenates the visual feature information with the potential action information to obtain the control sequence information corresponding to the target task.

[0123] Furthermore, when the action generation module 340 performs noise reduction and splicing processing on the state information and control sequence information corresponding to the robot to generate action sequence information corresponding to the robot when processing the target task, the action generation module 340 is used to: Obtain the state information corresponding to the robot, and determine the state embedding information corresponding to the state information; The state embedding information is concatenated with the control sequence information to obtain action state concatenation embedding information, and the action state concatenation embedding information is input into a pre-trained stream matching action decoder. Each of the multiple latent variable transformation modules set by the stream matching action decoder sequentially performs denoising and splicing processing on the action state splicing embedding information according to the setting order corresponding to the latent variable transformation modules, generating action sequence information corresponding to the robot when processing the target task; wherein, the input of each latent variable transformation module is the output of the latent variable transformation module corresponding to the previous setting order of the latent variable transformation module.

[0124] The robot action generation device provided in this application provides supervision for the training of the large language model by the latent action model during the training of the visual language action model. This can improve the generalization of the large language model while enhancing the cross-entity transfer capability of the visual language action model. The original image data, the fused visual representation information output by the large language model, and the latent action information are correspondingly compressed and spliced ​​to reduce the length of image and visual-related sequence information. This improves the sensitivity of the visual language action model to visual information while balancing the proportion of image data and text description information in the visual language action model. The device also performs denoising and splicing processing on the robot's state information and the compressed and spliced ​​control sequence information, thereby improving the smoothness and coherence of the robot's generated actions.

[0125] Please see Figure 4 , Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 4 As shown, the electronic device 400 includes a processor 410, a memory 420, and a bus 430.

[0126] The memory 420 stores machine-readable instructions executable by the processor 410. When the electronic device 400 is running, the processor 410 communicates with the memory 420 via the bus 430. When the machine-readable instructions are executed by the processor 410, they can perform the operations described above. Figure 2 The steps of the robot motion generation method in the method embodiment shown are described in detail in the method embodiment, and will not be repeated here.

[0127] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, can perform the above-described actions. Figure 2 The steps of the robot motion generation method in the method embodiment shown are described in detail in the method embodiment, and will not be repeated here.

[0128] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0129] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the shown or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0130] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0131] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0132] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0133] Finally, it should be noted that the above-described embodiments are merely specific implementations of this application, used to illustrate the technical solutions of this application, and not to limit them. The scope of protection of this application is not limited thereto. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this application. Such modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for generating robot actions, characterized in that, The generation method includes: In response to the robot receiving a target task, it acquires the image data associated with the target task and the text description information of the target task; Based on the image data and the text description information, a pre-trained large language model is used to determine the latent action information and fused visual representation information corresponding to the target task; wherein, the large language model is trained under the supervision provided by the pre-trained latent action model. The step of determining the latent action information and fused visual representation information corresponding to the target task based on the image data and the text description information using a pre-trained large language model includes: The text identifier sequence corresponding to the text description information is encoded and converted using a preset text encoder to obtain the text embedding vector corresponding to the text description information. The image data is then encoded and converted using a preset image encoder to obtain the image embedding vector corresponding to the image data. The text embedding vector and the image embedding vector are respectively input into a pre-trained large language model to obtain the latent action information and fused visual representation information corresponding to the target task output by the large language model; The image data, the potential action information, and the fused visual representation information are compressed and stitched together to obtain the control sequence information corresponding to the target task. The step of compressing and concatenating the image data, the latent action information, and the fused visual representation information to obtain the control sequence information corresponding to the target task includes: The image embedding vector corresponding to the image data is concatenated with the fused visual representation information to obtain fused image information, and the fused image information and the potential action information are respectively input into a preset multi-head attention pooler; The multi-head attention pooler compresses the fused image information to obtain visual feature information, and then concatenates the visual feature information with the potential action information to obtain the control sequence information corresponding to the target task. The state information and control sequence information of the robot are denoised and spliced ​​to generate action sequence information of the robot when processing the target task, so that the robot can perform corresponding actions based on the action sequence information.

2. The method according to claim 1, characterized in that, The latent action model includes an encoder, a vector quantization module, and a decoder; the latent action model is trained through the following steps: For the training dataset corresponding to the training task, two target training images are selected from the training image set of the training dataset at a preset time interval, and the task instruction information corresponding to the target training images is determined in the training text set of the training dataset. Based on the target training image and the task instruction information, the preset action-independent vectors of the potential action model are learned and trained to determine the target action-independent vectors, so as to obtain the trained irrelevant potential action model. Based on the target training image and the target action-independent vector, the preset potential action vectors of the irrelevant potential action model are learned and trained to determine the target potential action vectors, so as to obtain the trained potential action model.

3. The method according to claim 2, characterized in that, The step of learning and training the preset action-independent vectors of the latent action model based on the target training image and the task indication information, and determining the target action-independent vectors to obtain the trained action-independent latent action model, includes: The target training image, the task instruction information, and the preset action-independent vector are respectively input into the encoder to obtain the first task-independent action information corresponding to the training task output by the encoder; The first task-independent action information is input into the vector quantization module, and the vector quantization module queries the first task-independent codebook vector corresponding to the first task-independent action information in the preset task-independent potential action codebook. The first embedding vector corresponding to the target training image in the first frame, the text encoding vector corresponding to the task indication information, and the first task-independent codebook vector are respectively input into the decoder to obtain the first target embedding vector reconstructed from the target training image in the second frame by the decoder. Based on the second embedding vector and the first target embedding vector corresponding to the target training image in the second frame, calculate the first loss value corresponding to the action-independent vector; Based on the first loss value, iterative learning training is performed on the action-independent vector to determine the target action-independent vector, so as to obtain the trained action-independent potential action model.

4. The method according to claim 2, characterized in that, The step of learning and training the pre-set potential action vectors of the irrelevant potential action model based on the target training image and the target action-independent vectors to determine the target potential action vectors and obtain the trained potential action model includes: The target training image, the target action-independent vector, and the preset potential action vector are respectively input into the encoder to obtain the task-related action information and the second task-independent action information corresponding to the training task output by the encoder. The task-related action information and the second task-independent action information are respectively input into the vector quantization module. The vector quantization module queries the task-related codebook vector corresponding to the task-related action information in the preset task-related potential action codebook, and queries the second task-independent codebook vector corresponding to the second task-independent action information in the preset task-independent potential action codebook. The first embedding vector, the task-related codebook vector, and the second task-independent codebook vector corresponding to the target training image in the first frame are respectively input into the decoder to obtain the second target embedding vector reconstructed from the target training image in the second frame by the decoder. Based on the second embedding vector and the second target embedding vector corresponding to the target training image in the second frame, calculate the second loss value corresponding to the potential action vector; Based on the second loss value, the potential action vector is iteratively trained to determine the target potential action vector, so as to obtain the trained potential action model.

5. The method according to claim 2, characterized in that, The large language model is trained using the following steps based on supervision provided by a pre-trained latent action model: Two frames of the target training images are input into a pre-trained latent action model to obtain the latent action encoding information corresponding to the target training images output by the latent action model. The latent action encoding information is combined with the task instruction information, and the combined information is encoded using a preset text encoder to obtain a text embedding training vector. The target training image of the target frame is encoded using a preset image encoder to obtain an image embedding training vector. The image embedding training vector and the text embedding training vector are concatenated and input into the large language model to be trained to obtain the task-related action position information corresponding to the training task output by the large language model to be trained. Extract the target task-related action location information from the text embedding training vector, and calculate the third loss value based on the task-related action location information and the target task-related action location information; Based on the third loss value, iterative backpropagation is performed on the large language model to be trained to obtain the trained large language model.

6. The method according to claim 1, characterized in that, The step of denoising and concatenating the state information and control sequence information corresponding to the robot to generate the action sequence information corresponding to the robot when processing the target task includes: Obtain the state information corresponding to the robot, and determine the state embedding information corresponding to the state information; The state embedding information is concatenated with the control sequence information to obtain action state concatenation embedding information, and the action state concatenation embedding information is input into a pre-trained stream matching action decoder. Each of the multiple latent variable transformation modules set by the stream matching action decoder sequentially performs denoising and splicing processing on the action state splicing and embedding information according to the setting order corresponding to the latent variable transformation modules, generating action sequence information corresponding to the robot when processing the target task; wherein, the input of each latent variable transformation module is the output of the latent variable transformation module corresponding to the previous setting order of the latent variable transformation module.

7. A device for generating robot motion, characterized in that, The generating apparatus includes: The data acquisition module is used to acquire image data associated with the target task and text description information corresponding to the target task in response to the robot receiving the target task; The fusion prediction module is used to determine the latent action information and fused visual representation information corresponding to the target task based on the image data and the text description information using a pre-trained large language model; wherein, the large language model is trained under the supervision provided by the pre-trained latent action model; The compression and stitching module is used to compress and stitch together the image data, the potential action information and the fused visual representation information to obtain the control sequence information corresponding to the target task. The motion generation module is used to denoise and splice the state information and control sequence information of the robot to generate motion sequence information corresponding to the robot when processing the target task, so that the robot can perform corresponding actions based on the motion sequence information. When the fusion prediction module is used to determine the potential action information and fused visual representation information corresponding to the target task based on the image data and the text description information using a pre-trained large language model, the fusion prediction module is used to: The text identifier sequence corresponding to the text description information is encoded and converted using a preset text encoder to obtain the text embedding vector corresponding to the text description information. The image data is then encoded and converted using a preset image encoder to obtain the image embedding vector corresponding to the image data. The text embedding vector and the image embedding vector are respectively input into a pre-trained large language model to obtain the latent action information and fused visual representation information corresponding to the target task output by the large language model; When the compression and stitching module compresses and stitches the image data, the latent action information, and the fused visual representation information to obtain the control sequence information corresponding to the target task, the compression and stitching module is used for: The image embedding vector corresponding to the image data is concatenated with the fused visual representation information to obtain fused image information, and the fused image information and the potential action information are respectively input into a preset multi-head attention pooler; The multi-head attention pooler compresses the fused image information to obtain visual feature information, and then concatenates the visual feature information with the potential action information to obtain the control sequence information corresponding to the target task.

8. An electronic device, characterized in that, include: The device includes a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus. The machine-readable instructions are executed by the processor to perform the steps of the robot motion generation method as described in any one of claims 1 to 6.

Images