Robot control instruction generation method, device, equipment, medium and program product

Abstract

A robot control instruction generation method, device, equipment, storage medium and program product are disclosed. The method comprises: obtaining a target task of a robot and an observation image, a user instruction and a body state of a current time step of the robot; generating a first modal feature based on the observation image and the body state of the current time step and the observation image and the body state of a historical time step of the target task; generating a second modal feature based on the user instruction of the current time step; sequentially splicing the first modal feature, the second modal feature, a predetermined reasoning feature and a predetermined action feature to obtain input data of the current time step; the reasoning feature and the action feature are determined when the VLA model is iteratively trained; and inputting the input data of the current time step into the VLA model to obtain a control instruction of the current time step of the robot. The scheme solves the problem that the VLA model is difficult to reliably map multi-modal features to control instructions.

Description

Technical Field

[0001] This invention relates to the field of embodied intelligence technology, and in particular to a method, apparatus, electronic device, storage medium, and program product for generating control instructions for a robot. Background Technology

[0002] With the development of embodied AI technology, Vision-Language-Action (VLA) models have gradually become an important research direction for robot control command generation. Existing VLA models usually transform environmental observation images and natural language commands into robot low-level control commands by integrating the perception and understanding capabilities of large models. Existing mapping strategies between multimodal features and control commands mainly include two categories: (1) end-to-end direct mapping based on the "black box" mechanism, that is, directly mapping multimodal features to robot low-level control commands; (2) introducing an explicit text-supervised thought chain between multimodal features and control commands, that is, relying on fine-grained action description text written by humans to supervise the intermediate state training of VLA models, thereby connecting high-level semantics and low-level action trajectories.

[0003] However, due to the inherent limitations of VLA models' understanding capabilities, the end-to-end direct mapping method based on a "black box" mechanism can only support motion control for robots performing simple tasks. For complex tasks with multiple steps, end-to-end direct mapping is difficult to achieve. Text-supervised thought processes require human intervention, and it is difficult for humans to exhaustively capture all reasonable motion trajectories for the robot to perform the target task. This makes it difficult for VLA models to naturally align multimodal features with control commands, and also makes it difficult to perform closed-loop dynamic replanning when motion execution fails. Summary of the Invention

[0004] This invention provides a method, apparatus, device, storage medium, and program product for generating control commands for robots, in order to solve the problem that VLA models have difficulty reliably mapping multimodal features to control commands. By adding pre-determined inference features between the multimodal features and action features of the input data of the VLA model, the semantic reasoning gap between high-level commands and low-level control is bridged, and a reliable mapping from multimodal features to control commands is achieved.

[0005] According to one aspect of the present invention, a method for generating control commands for a robot is provided, the method comprising: Acquire the robot's target task, as well as the observed images, user commands, and robot status at the current time step; Based on the observed image and ontology state at the current time step, as well as the observed image and ontology state at the historical time steps of the target task, the first modality feature is generated. Generate second modal features based on user commands at the current time step; The first modality features, the second modality features, the pre-determined inference features, and the pre-determined action features are sequentially concatenated to obtain the input data for the current time step; the inference features and the action features are determined by the VLA model when iterative training is completed; Input the current time step's input data into the VLA model to obtain the robot's control commands for the current time step.

[0006] According to another aspect of the present invention, a robot control command generation apparatus is provided, the apparatus comprising: The information acquisition module is used to acquire the robot's target task, as well as the observed images, user commands, and robot status at the current time step. The first modality feature generation module is used to generate first modality features based on the observed image and ontology state at the current time step and the observed image and ontology state at the historical time steps of the target task. The second modality feature generation module is used to generate second modality features based on user instructions at the current time step; The input data generation module is used to sequentially concatenate the first modality features, the second modality features, the pre-determined inference features, and the pre-determined action features to obtain the input data for the current time step; the inference features and the action features are determined by the VLA model when iterative training is completed; The control command generation module is used to input the input data of the current time step into the VLA model to obtain the control command of the robot at the current time step.

[0007] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising: At least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to execute the robot control instruction generation method according to any embodiment of the present invention.

[0008] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the robot control instruction generation method according to any embodiment of the present invention.

[0009] According to another aspect of the present invention, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the robot control instruction generation method described in any embodiment of the present invention.

[0010] The technical solution of this invention involves acquiring the robot's target task and the robot's observation image, user commands, and body state at the current time step; generating a first modal feature based on the observation image and body state at the current time step, as well as the observation image and body state at previous time steps of the target task; generating a second modal feature based on the user commands at the current time step; sequentially concatenating the first modal feature, the second modal feature, pre-determined inference features, and pre-determined action features to obtain the input data for the current time step; the inference features and action features are determined by the VLA model during iterative training; and inputting the input data for the current time step into the VLA model to obtain the robot's control commands for the current time step. This technical solution solves the problem of the VLA model's difficulty in reliably mapping multimodal features to control commands. By adding pre-determined inference features between the multimodal features and action features of the VLA model's input data, it bridges the semantic reasoning gap between high-level commands and low-level control, achieving a reliable mapping from multimodal features to control commands.

[0011] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 This is a flowchart of a robot control command generation method according to Embodiment 1 of the present invention; Figure 2 This is a flowchart of a robot control command generation method according to Embodiment 2 of the present invention; Figure 3 This is a schematic diagram of the structure of a robot control command generation device according to Embodiment 3 of the present invention; Figure 4 This is a schematic diagram of the structure of an electronic device that implements the robot control command generation method of the present invention. Detailed Implementation

[0014] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0015] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be used interchangeably where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices. The acquisition, storage, use, and processing of data in the technical solutions of this application all comply with the relevant provisions of national laws and regulations.

[0016] Example 1 Figure 1 This is a flowchart illustrating a method for generating control commands for a robot, as provided in Embodiment 1 of the present invention. This embodiment is applicable to robot task execution scenarios, particularly for generating robot control commands. This method can be executed by a robot control command generation device, which can be implemented in hardware and / or software and can be configured in an electronic device. Figure 1 As shown, the method includes: S110: Acquire the robot's target task, as well as the observed image, user commands, and robot status at the current time step.

[0017] This solution can be executed by electronic devices such as computers and servers. Specifically, the electronic device can be the robot itself or a management platform that interacts with the robot. The electronic device can acquire the robot's target task, such as grasping a target object and transporting it to a target location, or preparing a target dish. The target task can be the task that the robot is currently performing. The robot typically needs one or more time steps to complete the target task. The electronic device can acquire the observation image, user commands, and robot status at the current time step.

[0018] Among them, the observed image can be an environmental image of the robot in the process of performing the target task, the user command can be the instruction information for the robot's actions issued by the user in the form of voice, gesture and operation, etc., and the body state can be information such as the position, angle, posture, opening and closing degree of the robot's various moving parts and the contact state with the target object.

[0019] It should be noted that during the robot's execution of the target task, there are not user instructions at every time step. If there are no user instructions at the current time step, the electronic device can set the user instructions for the current time step to be empty, or it can use the user instructions from the previous time step or the user instructions from the initial time step of the target task as the user instructions for the current time step, depending on the needs of the target task.

[0020] S120. Based on the observed image and ontology state at the current time step, as well as the observed image and ontology state at the historical time steps of the target task, generate the first modality feature.

[0021] Electronic devices can generate features for the current time step based on the observed image and the ontology state at the current time step, or they can generate features for historical time steps based on the observed images and the ontology state at historical time steps of the target task. Specifically, electronic devices can directly concatenate the observed image and the ontology state at the same time step to obtain the features for that time step, or they can perform feature extraction once or multiple times on the observed image and the ontology state at the same time step, and then concatenate the feature extraction results of the observed image and the ontology state to obtain the features for that time step.

[0022] Understandably, the historical time steps of the target task may include one or more. After obtaining the current time step features and the historical time step features of the target task, the electronic device can fuse the features of each time step of the target task to obtain the first modality features.

[0023] S130. Generate second modal features based on user instructions at the current time step.

[0024] Electronic devices can directly use the user command at the current time step as the second modality feature, or they can perform one or more feature extractions on the user command at the current time step and use the feature extraction results as the second modality feature.

[0025] S140. The first modality features, the second modality features, the pre-determined inference features, and the pre-determined action features are sequentially concatenated to obtain the input data for the current time step; the inference features and the action features are determined by the VLA model when iterative training is completed.

[0026] As is easily understood, the first modal features come from image data and state data, while the second modal features come from command data. Electronic devices can input the first and second modal features as multimodal features into the VLA model. The first and second modal features can provide the robot with high-level semantic information and action execution goals.

[0027] To guide the robot in generating low-level control commands, the electronic device adds motion features to the input data of the VLA model. To bridge the semantic reasoning gap between high-level semantics and low-level control, the electronic device adds inference features to the input data of the VLA model. These inference and motion features are determined during the iterative training of the VLA model. Before training begins, the inference and motion features are empty. During iterative training, the VLA model continuously learns the mapping relationship between the multimodal features of the training samples and control commands, updating and refining the inference and motion features. In the application of the VLA model, inference features can serve as a bridge between multimodal features and control commands, enabling more accurate control command prediction and improving the interpretability and reliability of the VLA model.

[0028] It should be noted that the first modality features, second modality features, inference features, and action features in the input data of the current time step must be concatenated in sequence, in order to construct an inference chain from the feature space to the action space.

[0029] In fact, the inference feature is a potential inference space that can automatically align multimodal features with robot physical actions without human text annotation. It solves the problems of existing VLA models that cannot spontaneously characterize semantic planning space without human intervention to describe action details, have difficulty naturally aligning visual observation intentions with robot action trajectories, and have difficulty performing closed-loop dynamic replanning when action execution fails.

[0030] S150. Input the input data of the current time step into the VLA model to obtain the control command of the robot at the current time step.

[0031] As is easily understood, VLA models are typically large-scale visual language models built on the Transformer architecture. Electronic devices can input the input data of the current time step into the VLA model. The VLA model can perform temporal context awareness based on the multimodal features in the input data of the current time step, align the multimodal features with action features based on inference features, and guide the generation of control commands based on action features, thereby obtaining the control commands for the robot at the current time step.

[0032] The technical solution of this invention involves acquiring the robot's target task and the robot's observation image, user commands, and body state at the current time step; generating a first modal feature based on the observation image and body state at the current time step, as well as the observation image and body state at previous time steps of the target task; generating a second modal feature based on the user commands at the current time step; sequentially concatenating the first modal feature, the second modal feature, pre-determined inference features, and pre-determined action features to obtain the input data for the current time step; the inference features and action features are determined by the VLA model during iterative training; and inputting the input data for the current time step into the VLA model to obtain the robot's control commands for the current time step. This technical solution solves the problem of the VLA model's difficulty in reliably mapping multimodal features to control commands. By adding pre-determined inference features between the multimodal features and action features of the VLA model's input data, it bridges the semantic reasoning gap between high-level commands and low-level control, achieving a reliable mapping from multimodal features to control commands.

[0033] Example 2 Figure 2 This is a flowchart of a robot control command generation method provided in Embodiment 2 of the present invention. This embodiment refines the working principle of the VLA model based on the above embodiment. Figure 2 As shown, the method includes: S201. Obtain the robot's target task, as well as the observed image, user instructions, and robot status at the current time step.

[0034] It's easy to understand that using features from a single time step or a small number of consecutive time steps to construct the first modality features can easily lead to "temporal myopia" in VLA models. That is, when performing long-cycle target tasks, if the target object is occluded or the environment undergoes dynamic changes, it can easily lead to target tracking failure or loss of target state. If the data dimension of the first modality features is expanded without limit in order to achieve persistent memory, it can easily lead to an exponential increase in computational costs, resulting in slow control command output speed and making it difficult to meet the real-time requirements of target task execution.

[0035] To overcome the "temporal myopia" problem of the VLA model, this solution constructs a dual-scale temporal memory center. Based on the exponential moving average mechanism of the long-term memory center and the feature queue of the short-term memory center, under the limited input data dimension of the VLA model, it adaptively compresses and retains the long-term scene evolution layout and the short-term fine motion trend. This enables the robot to maintain stable context awareness when facing highly dynamic, complex and unstructured real-world physical interaction scenarios, such as when the target object is occluded or the environment changes. This greatly improves the robustness and success rate of long-cycle operation tasks.

[0036] S202. Generate the first feature of the current time step based on the observed image and the ontological state at the current time step.

[0037] Specifically, the electronic device can extract features from the observed image and ontology state at the current time step based on a pre-trained encoder to obtain the first feature of the current time step. For example, it can perform block-level feature extraction on the observed image and ontology state at the current time step using pre-trained encoders such as DINOv2 (Distillation with No Labels version 2) and SigLIP (Sigmoid Loss for Language-Image Pre-training) to obtain the first feature of the current time step. Similarly, the electronic device can generate the first features of historical time steps based on the generation method of the first feature of the current time step. The first feature can be represented as... ,in, Indicates the time step index. , Indicates the number of blocks. This represents the dimension of each block-level feature.

[0038] S203. Based on a pre-set decay rate set, perform an exponentially weighted moving average on the first feature of the current time step and the first feature of the historical time step of the target task to obtain the second feature of the current time step; the second feature includes the second sub-feature matched by each decay rate in the decay rate set.

[0039] The time span of the long-term memory center can include all time steps from the start of the target task to the current moment, and can cover all historical time steps during the execution of the target task, so as to avoid task execution failure due to the loss of historical information.

[0040] The environmental layout of historical time steps typically evolves slowly. To compress the first feature of a historical time step, this scheme can set a decay rate set containing a preset number of different decay rates, and then perform exponentially weighted moving average (EWMA) processing on the first feature of the current time step and the first feature of the target task's historical time steps. EWMA is a time series analysis method that assigns exponentially decaying weights to historical data, giving more weight to recent data.

[0041] Specifically, the second feature of the current time step obtained after exponentially weighted moving average processing can include second sub-features matched with each decay rate in the decay rate set. Each second sub-feature can be represented as: ,in, Indicates the time step index. Indicates the decay rate index. Indicates the first Each attenuation rate value is available. Indicates the first The first time step The second sub-feature of the decay rate. The dimension of each second sub-feature is the same as the dimension of the first feature, i.e. .

[0042] S204. Calculate the preset order change of the first feature at the current time step, and determine the weight coefficient of each decay rate in the decay rate set based on the preset order change of the first feature at the current time step.

[0043] After obtaining the first feature of the current time step, the electronic device can calculate the first or multiple order changes of the first feature of the current time step, set the weight coefficients of each decay rate in the decay rate set based on the order changes of the first feature of the current time step, and track the time statistics of the first feature of the target task at each time step to capture the local motion speed and variability of the robot-controlled object.

[0044] Specifically, the changes of each order of the first feature at the current time step can be uniformly represented as: ;in, Indicates the time step index. Indicates the order index, , Represents order The corresponding coefficients, Indicates time step The first feature order of change Indicates time step The zeroth order change of the first characteristic, . This represents the multiplication function, used to... Perform element-wise multiplication Second-rate.

[0045] In a specific example, the electronic device can calculate the second-order changes of the first feature at the current time step, where the first-order change is represented as: The second-order change is expressed as: ;in, This represents the element-wise multiplication operator.

[0046] The electronic device can concatenate the first feature at the current time step with the changes of the first feature at each time step, and determine the weight coefficients of the second sub-features for each decay rate based on a multilayer perceptron. Specifically, the weight coefficient vector formed by the weight coefficients of the second sub-features for each decay rate can be represented as: ;in, Indicates the number of orders. Describes the multilayer perceptron function. express Activation function , This indicates the number of attenuation rates in the attenuation rate set.

[0047] S205. Determine the third feature of the current time step based on the second feature of the current time step, the weight coefficients of each decay rate in the decay rate set, and the preset order change of the first feature of the current time step.

[0048] Electronic devices can use the weighting coefficients of each decay rate in the decay rate set to weight the second sub-features that match each decay rate in the second feature of the current time step. Based on the weighted sum of the second sub-features that match each decay rate and the changes of each order of the first feature of the current time step, a third feature of the current time step, also known as the long-term memory feature, is generated.

[0049] Specifically, the third feature can be represented as: ; Indicates the time step index. Indicates the decay rate index. This represents the number of attenuation rates in the attenuation rate set. express The Middle The weighting coefficients for each decay rate, Indicates the first The first time step The second sub-feature of the attenuation rate, - They represent the first The first-order feature of each time step is of order 1- order of change - They represent - The weighting coefficients.

[0050] S206. Based on the attention mechanism, determine the target vector of the current time step from the first feature sequence; the first feature sequence is a sequence composed of the first feature of the current time step and the first features of a preset number of historical time steps, and the target vector includes a query vector, a key vector and a value vector.

[0051] For the short-term memory center, the electronic device can define a first feature sequence. The short-term memory center focuses on the first features of a predetermined number of historical time steps and the first feature of the current time step within this first feature sequence to capture fine-grained recent motion features of the robot that are more closely associated with the current time step. Specifically, the first feature sequence includes... The first feature of each time step, and the sequence of the first features of the current time step, can be represented as: ,in, This represents the first feature of the current time step. - Indicates the time step before the current time step The first characteristic of a historical time step .

[0052] Electronic devices can determine the query vector, key vector, and value vector for the current time step from a first feature sequence based on an attention mechanism. The query vector represents the content in the first feature sequence that needs attention, used for "asking" or "searching" for relevant information. The key vector serves as an "index" or "identifier" for each element in the first feature sequence, used to match it with the query vector and calculate relevance. The value vector contains the actual information of each element in the first feature sequence, and is weighted and summed according to attention weights. Specifically, the query vector for the current time step can be represented as: The key vector at the current time step can be represented as: The value vector of the current time step can be represented as: ,in, This represents the first feature of the current time step. The layer normalization function is used to stabilize the feature distribution. This represents the result of the first feature sequence at the current time step after preprocessing such as temporal enhancement and positional encoding. , and Indicates the learnable weight parameters. , , .

[0053] S207. Determine the fourth feature of the current time step based on the target vector of the current time step.

[0054] For each time step in the first feature sequence , The electronic device can calculate the temporal attention weight corresponding to this time step, represented as... ,in, This indicates temperature hyperparameters. Indicates recentity bias. This represents the query vector at the current time step. The key vector representing the time step at the current time step The corresponding values.

[0055] After obtaining the temporal attention weights corresponding to each time step, the electronic device can perform a weighted summation of the values ​​of each time step in the value vector according to the temporal attention weights corresponding to each time step to obtain the fourth feature, namely the short-term memory feature.

[0056] Specifically, the fourth feature of the current time step can be represented as: ,in, This represents the time step index in the first feature sequence. This indicates the number of time steps of the first feature included in the first feature sequence. Indicates time step The corresponding time attention weights, Represents the time step in the value vector The value of , This represents the element-wise multiplication operator. .

[0057] S208. Determine the gating parameters based on the first, third, and fourth features of the current time step.

[0058] After obtaining the third and fourth features at the current time step, the electronic device can fuse dual-scale memory features to obtain the first modality feature. The electronic device can concatenate the first, third, and fourth features of the current time step, and determine the gating parameters based on the concatenation result using the Sigmoid activation function. Specifically, the gating parameters can be expressed as: ,in, This represents the concatenation result of the first, third, and fourth features at the current time step. This represents the Sigmoid activation function.

[0059] S209. Based on the gating parameters, fuse the third and fourth features of the current time step to obtain the first modal feature of the current time step.

[0060] Electronic devices can fuse the third and fourth features of the current time step based on gating parameters to obtain the first modal feature of the current time step. Specifically, the first modal feature of the current time step can be represented as: ,in, The third feature represents the current time step. The fourth feature represents the current time step. Indicates the gating parameters, This represents the element-wise multiplication operator.

[0061] In this scheme, the process of fusing the third and fourth features of the current time step based on the gating parameters to obtain the first modal feature of the current time step includes: The third and fourth features of the current time step are fused according to the gating parameters to obtain the fifth feature of the current time step; If the robot's storage space meets the preset conditions, the fifth feature of the current time step will be flattened, the flattened result will be spliced ​​with the first feature of the current time step, and the spliced ​​result will be used as the first modal feature of the current time step. If the robot's storage space does not meet the preset conditions, then the fifth feature of the current time step is multiplied based on the first projection matrix, and the fifth feature of the current time step is added based on the second projection matrix. Based on the multiplication result, the addition result, and the first feature of the current time step, the first modal feature of the current time step is generated.

[0062] Based on the fusion of the third and fourth features of the current time step using gating parameters, the electronic device can further fuse the first feature of the current time step according to the size of the robot's storage space to obtain the first modal feature of the current time step.

[0063] In this scheme, the fifth feature of the current time step, obtained by fusing the third and fourth features based on the gating parameters, can be expressed as: ,in, The third feature represents the current time step. The fourth feature represents the current time step. Indicates the gating parameters, This represents the element-wise multiplication operator.

[0064] The preset condition could be that the robot's storage space is greater than a preset storage space threshold. In a feasible solution, if the robot's storage space meets the preset condition, the electronic device can flatten the fifth feature of the current time step, concatenate the flattened result with the first feature of the current time step, and use the concatenated result as the first modal feature of the current time step. The first modal feature of the current time step can be represented as: ,in, The fifth feature represents the current time step. Represents the flattening function. This represents the first feature of the current time step. express and The splicing result.

[0065] The generation method of the first modality features mentioned above allows the VLA model's fully self-attention mechanism to explicitly pay attention to and invoke historical details, which is particularly effective for maintaining long-term contextual information.

[0066] In another feasible solution, if the robot's storage space does not meet the preset conditions, the electronic device can perform multiplication on the fifth feature of the current time step based on the first projection matrix, and add the fifth feature of the current time step based on the second projection matrix. Based on the multiplication result, the addition result, and the first feature of the current time step, the first modal feature of the current time step is generated. Specifically, the first modal feature of the current time step can be represented as: ;in, This represents the first feature of the current time step. This represents the element-wise multiplication operator. The fifth feature represents the current time step. Denotes the first projection matrix. The second projection matrix is ​​represented by the first projection matrix and the second projection matrix, both of which are linear projection matrices and can be determined after the VLA model training is completed.

[0067] The aforementioned first modality feature generation method achieves the injection of dual-scale memory features without increasing the sequence length, avoiding the secondary computation cost of the attention mechanism, thereby improving decoding efficiency and robot control frequency.

[0068] This solution provides two complementary first-modal feature generation strategies, which can be freely selected according to the robot's storage space size to balance the context modeling capability and inference efficiency of the VLA model.

[0069] S210. Generate second modal features based on user instructions at the current time step.

[0070] S211. The first modality features, the second modality features, the pre-determined inference features, and the pre-determined action features are sequentially concatenated to obtain the input data for the current time step; the inference features and the action features are determined by the VLA model when iterative training is completed.

[0071] The input data for the current time step can be represented as: ,in, This represents the first modal feature, for example, it could be... , or , Indicates the second modal feature. Indicates the characteristics of reasoning, It indicates the characteristics of an action.

[0072] S212. Extract a first pooling representation, a second pooling representation, and a third pooling representation from the input data of the current time step. The first pooling representation is determined based on inference features, the second pooling representation is determined based on action features, and the third pooling representation is determined based on the last feature unit in the second modality features of the current time step.

[0073] To avoid the high cost of manual annotation and to enable the VLA model to transform abstract high-level features into precise low-level control commands, electronic devices introduce inference features during the VLA model training process. This constructs a potential inference space where multimodal features are aligned with the low-level control command output, thereby effectively overcoming the "many-to-one" semantic inference gap caused by the ambiguity of natural language command descriptions.

[0074] Specifically, the electronic device can perform feature extraction on the input data at the current time step at least once, and determine the first pooling representation, the second pooling representation, and the third pooling representation from the feature extraction results.

[0075] In a specific example, the input data at the current time step The feature extraction results are represented as , , Indicates the length of the feature sequence. Representing the block-level feature dimension, the first pooling representation can be... The average value of the feature sequence at the feature marker position in the inference process is expressed as: , The second pooling characterization can be The average value of the feature sequence at the action feature marker position is expressed as: , The third pooling characterization can be The last feature sequence at the second modality feature marker position is represented as , It should be noted that, The last feature sequence of the second modality feature marker position is adjacent to the first inference feature marker position, which can effectively summarize the multimodal features upstream of it.

[0076] S213. Based on the first pooling representation, the second pooling representation, and the third pooling representation, determine the control command for the robot's current time step based on the contrastive learning unit in the VLA model.

[0077] The electronic device can project the first, second, and third pooling representations into a unified shared contrast space, obtaining the first projection, second projection, and third projection, respectively. In the shared contrast space, the first projection is considered a reasoning bridge, maximizing the inclusion of joint information about multimodal features and underlying control commands. Based on the contrastive learning unit in the VLA model, the electronic device can output the control command for the robot's current time step according to the first, second, and third projections.

[0078] In this scheme, the contrastive learning unit in the VLA model is trained based on a first contrastive loss and a second contrastive loss; the first contrastive loss is used to align the inference features and action features, and the second contrastive loss is used to align the inference features and multimodal features.

[0079] During the training of the contrastive learning unit in the VLA model, minimizing the first contrastive loss enables the contrastive learning unit to accurately predict the paired second projection based on the first projection, and minimizing the second contrastive loss enables the contrastive learning unit to accurately predict its corresponding first projection based on the third projection. In a specific example, the first contrastive loss can be expressed as: The second contrast loss can be expressed as: ;in, , Both represent training sample indices. Indicates the number of training samples. Indicates training samples The first projection, Indicates training samples The second projection, Indicates training samples The third projection, Indicates training samples The second projection, Indicates training samples The third projection, and Temperature hyperparameters representing a smooth distribution This represents a similarity function, such as cosine similarity, Euclidean distance, or other similarity functions. This represents an exponential function with a base of natural numbers.

[0080] Electronic devices can combine the first contrast loss and the second contrast loss to obtain the mutual information inference loss, which can be expressed as: ; Indicates the first comparative loss. Indicates the second comparative loss. This represents the weighting coefficient of the first contrast loss. This represents the weighting coefficient of the second contrast loss.

[0081] Electronic devices can train the contrastive learning units in the VLA model based on the mutual information inference loss of the training samples, thereby obtaining contrastive learning units that can accurately predict control commands at the current time step.

[0082] Based on the above scheme, the VLA model is trained based on total loss, which includes a first contrastive loss, a second contrastive loss, and a regression loss. The regression loss is determined based on the control command labels of the training samples and the control command prediction results of the training samples output by the VLA model.

[0083] Existing VLA models typically employ scalar quantization strategies when generating robot control commands, mapping each degree of freedom in the continuous action space to an independent discrete label in the vocabulary. This forces the VLA model to generate a long string of labels for each dimension when predicting a single physical action, forming a sequential autoregressive generation chain. As the length of the action trajectory increases, the scalar-wise control command generation mechanism becomes inefficient in decoding, easily leading to extremely high inference latency, and failing to meet the high-frequency control requirements of robots in the real world.

[0084] This scheme abandons the inefficient autoregressive decoding paradigm and instead adopts a vector-level parallel prediction mechanism, thereby significantly improving the robot's real-time response capability while ensuring control accuracy. This scheme introduces a parallel motion decoding strategy, which no longer assigns an independent label to each motion degree of freedom, but instead assigns a single dedicated motion label to each complete motion. For actions containing... Each action and has An action sequence with multiple degrees of freedom reduces the length of the action label sequence to that of an autoregressive decoding method. .

[0085] In the final hidden state after obtaining all action markers back, The electronic device flattens the hidden states of all time steps into a single vector and projects them through a lightweight global projection matrix. , The entire motion trajectory is decoded in parallel during a single forward pass. The control quality of the robot at the current time step can be expressed as: , This represents the flattening function. The action decoder in the VLA model is trained directly on the continuous action space using a standard regression objective, rather than discrete classification. The control command labels of the training samples are used as the basis for this optimization. For supervision, Electronic devices minimize element-wise Loss is used to constrain the control command prediction results output by the VLA model. .

[0086] Specifically, the regression loss can be expressed as: ;in, Indicates the number of actions. This indicates the degree of freedom for each movement. Indicates the action index. Indicates the degree of freedom index. Indicates the first The first action Prediction results of control commands for each degree of freedom Indicates the first The first action Control instruction labels for each degree of freedom.

[0087] The overall training objective of the VLA model can be to minimize the total loss, which is determined based on the regression loss and the mutual information inference loss. The total loss can be expressed as: ,in, This represents the weighting coefficient, used to balance the mutual information inference loss and the regression loss.

[0088] This solution proposes a visual language action robot control method based on mutual information reasoning and temporal memory center, which effectively overcomes the limitations of existing VLA models in highly dynamic, complex and unstructured real-world physical interaction scenarios. Compared with existing technologies, this solution has the following significant technical effects: (1) It effectively overcomes the "temporal myopia" problem and achieves robust long-cycle control with low computing power cost. In response to the "temporal myopia" and target state loss problems of VLA models caused by single-frame input or simple splicing mechanism, a dual-scale temporal memory center is introduced. Through the exponential moving average mechanism of the long-term memory center and the feature queue of the short-term memory center, it can adaptively compress and retain the long-term scene evolution layout and short-term fine motion trend without unlimited expansion of the "context window". This enables the robot to maintain stable context perception when facing highly dynamic, complex and unstructured real-world physical interaction scenarios, such as when the target is occluded or the environment changes, which greatly improves the robustness and success rate of long-cycle operation tasks; (2) It bridges the gap of "many-to-one" semantic reasoning and gives the VLA model the ability to plan autonomously and correct closed loops without annotation. To address the shortcomings of lack of planning in end-to-end mapping and the "many-to-one" semantic reasoning gap and high manual annotation costs faced by explicit text supervision, a latent semantic reasoning module guided by mutual information was constructed. By inserting learnable latent reasoning features into the input data of the VLA model and using bidirectional contrastive loss of action alignment and context alignment for constraints, a latent reasoning space with relatively aligned multimodal feature input and physical action output was successfully constructed without the need for manual intervention to describe action details. This not only improved the alignment accuracy between natural language instruction intent and underlying action execution, but also endowed the VLA model with the ability to perform autonomous closed-loop error correction and dynamic replanning when action execution is blocked; (3) Breaking the bottleneck of autoregressive decoding efficiency, achieving high-frequency real-time physical control with extremely low latency. To address the low decoding efficiency and extremely high inference latency issues caused by the traditional "autoregressive decoding paradigm" combined with "scalar quantization," a vector-level parallel motion decoder is proposed. This decoder abandons the method of serially generating discrete markers for each motion degree of freedom and time step. Instead, it decodes the entire motion trajectory of the current time step in parallel using a lightweight global projection matrix during a single forward pass. This vector-level parallel prediction mechanism completely breaks through the efficiency bottleneck of serial control command generation, significantly reduces inference latency, and substantially improves the throughput of low-level control command generation. It meets the stringent requirements of high-frequency real-time robot control in real-world environments, achieving a high degree of synchronization between perception, inference, and physical motion execution.

[0089] Example 3 Figure 3 This is a schematic diagram of a robot control command generation device provided in Embodiment 3 of the present invention. Figure 3 As shown, the device includes: The information acquisition module 310 is used to acquire the robot's target task, as well as the observed image, user instructions, and body state of the robot at the current time step. The first modality feature generation module 320 is used to generate first modality features based on the observed image and ontology state at the current time step and the observed image and ontology state at the historical time steps of the target task. The second modal feature generation module 330 is used to generate second modal features based on the user command at the current time step; The input data generation module 340 is used to sequentially concatenate the first modality features, the second modality features, the pre-determined inference features, and the pre-determined action features to obtain the input data for the current time step; the inference features and the action features are determined by the VLA model when iterative training is completed; The control command generation module 350 is used to input the input data of the current time step into the VLA model to obtain the control command of the robot at the current time step.

[0090] In this scheme, the first modal feature generation module 320 includes: The first feature generation unit is used to generate the first feature of the current time step based on the observed image and the ontology state at the current time step. The second feature generation unit is used to perform an exponentially weighted moving average processing on the first feature of the current time step and the first feature of the historical time step of the target task based on a pre-set decay rate set to obtain the second feature of the current time step; the second feature includes a second sub-feature matched with each decay rate in the decay rate set. The weight coefficient determination unit is used to calculate the preset order change of the first feature at the current time step, and determine the weight coefficient of each decay rate in the decay rate set based on the preset order change of the first feature at the current time step. The third feature generation unit is used to determine the third feature of the current time step based on the second feature of the current time step, the weight coefficient of each decay rate in the decay rate set, and the preset order change of the first feature of the current time step. The target vector determination unit is used to determine the target vector of the current time step from the first feature sequence based on the attention mechanism; the first feature sequence is a sequence composed of the first feature of the current time step and the first features of a preset number of historical time steps, and the target vector includes a query vector, a key vector and a value vector. The fourth feature generation unit is used to determine the fourth feature of the current time step based on the target vector of the current time step; The gating parameter determination unit is used to determine the gating parameters based on the first feature, third feature, and fourth feature of the current time step. The first modal feature generation unit is used to fuse the third and fourth features of the current time step according to the gating parameters to obtain the first modal feature of the current time step.

[0091] Based on the above scheme, the first modal feature generation unit is specifically used for: The third and fourth features of the current time step are fused according to the gating parameters to obtain the fifth feature of the current time step; If the robot's storage space meets the preset conditions, the fifth feature of the current time step will be flattened, the flattened result will be spliced ​​with the first feature of the current time step, and the spliced ​​result will be used as the first modal feature of the current time step. If the robot's storage space does not meet the preset conditions, then the fifth feature of the current time step is multiplied based on the first projection matrix, and the fifth feature of the current time step is added based on the second projection matrix. Based on the multiplication result, the addition result, and the first feature of the current time step, the first modal feature of the current time step is generated.

[0092] In one feasible solution, the control command generation module 350 is specifically used for: Extract a first pooling representation, a second pooling representation, and a third pooling representation from the input data at the current time step. The first pooling representation is determined based on inference features, the second pooling representation is determined based on action features, and the third pooling representation is determined based on the last feature unit in the second modality features at the current time step. Based on the first pooling representation, the second pooling representation, and the third pooling representation, the control command for the robot's current time step is determined using the contrastive learning unit in the VLA model.

[0093] In this embodiment, the contrastive learning unit in the VLA model is trained based on a first contrastive loss and a second contrastive loss; the first contrastive loss is used to align the inference features and action features, and the second contrastive loss is used to align the inference features and multimodal features.

[0094] Based on the above scheme, the VLA model is trained based on total loss, which includes a first contrastive loss, a second contrastive loss, and a regression loss. The regression loss is determined based on the control command labels of the training samples and the control command prediction results of the training samples output by the VLA model.

[0095] The robot control command generation device provided in this embodiment of the invention can execute the robot control command generation method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0096] Example 4 Figure 4 A schematic diagram of an electronic device 410 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0097] like Figure 4 As shown, the electronic device 410 includes at least one processor 411 and a memory communicatively connected to the at least one processor 411. The memory may be a read-only memory (ROM) 412, a random access memory (RAM) 413, etc. The memory stores computer programs executable by the at least one processor. The processor 411 can perform various appropriate actions and processes based on the computer program stored in the ROM 412 or loaded from storage unit 418 into the RAM 413. The RAM 413 may also store various programs and data required for the operation of the electronic device 410. The processor 411, ROM 412, and RAM 413 are interconnected via a bus 414. An input / output (I / O) interface 415 is also connected to the bus 414.

[0098] Multiple components in electronic device 410 are connected to I / O interface 415, including: input unit 416, such as keyboard, mouse, etc.; output unit 417, such as various types of displays, speakers, etc.; storage unit 418, such as disk, optical disk, etc.; and communication unit 419, such as network card, modem, wireless transceiver, etc. Communication unit 419 allows electronic device 410 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0099] Processor 411 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 411 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 411 performs the various methods and processes described above, such as methods for generating control instructions for a robot.

[0100] In some embodiments, the robot control command generation method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 418. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 410 via ROM 412 and / or communication unit 419. When the computer program is loaded into RAM 413 and executed by processor 411, one or more steps of the robot control command generation method described above may be performed. Alternatively, in other embodiments, processor 411 may be configured to execute the robot control command generation method by any other suitable means (e.g., by means of firmware).

[0101] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0102] Computer programs used to implement the methods of the present invention can be written in any combination of one or more programming languages. These computer programs can be provided to the processor of a general-purpose computer, a special-purpose computer, or other control instruction generation device for a programmable robot, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The computer programs can be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0103] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0104] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0105] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0106] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0107] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0108] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for generating control commands for a robot, characterized in that, The method includes: Acquire the robot's target task, as well as the observed images, user commands, and robot status at the current time step; Based on the observed image and ontology state at the current time step, as well as the observed image and ontology state at the historical time steps of the target task, the first modality feature is generated. Generate second modal features based on user commands at the current time step; The first modality features, the second modality features, the pre-determined inference features, and the pre-determined action features are sequentially concatenated to obtain the input data for the current time step; the inference features and the action features are determined by the VLA model when iterative training is completed; Input the current time step's input data into the VLA model to obtain the robot's control commands for the current time step.

2. The method according to claim 1, characterized in that, The generation of first modality features based on the observed image and ontology state at the current time step, as well as the observed image and ontology state at historical time steps of the target task, includes: Generate the first feature of the current time step based on the observed image and the ontology state at the current time step; Based on a pre-set decay rate set, an exponentially weighted moving average is applied to the first feature of the current time step and the first feature of the target task's historical time steps to obtain the second feature of the current time step; the second feature includes the second sub-feature matched by each decay rate in the decay rate set. Calculate the preset order change of the first feature at the current time step, and determine the weighting coefficient of each decay rate in the decay rate set based on the preset order change of the first feature at the current time step. The third feature of the current time step is determined based on the second feature of the current time step, the weight coefficients of each decay rate in the decay rate set, and the preset order change of the first feature of the current time step. Based on the attention mechanism, the target vector for the current time step is determined from the first feature sequence; the first feature sequence is a sequence composed of the first feature of the current time step and the first features of a preset number of historical time steps, and the target vector includes a query vector, a key vector and a value vector. The fourth feature of the current time step is determined based on the target vector of the current time step; Determine the gating parameters based on the first, third, and fourth features of the current time step; The third and fourth features of the current time step are fused based on the gating parameters to obtain the first modal feature of the current time step.

3. The method according to claim 2, characterized in that, The process of fusing the third and fourth features of the current time step based on the gating parameters to obtain the first modal feature of the current time step includes: The third and fourth features of the current time step are fused according to the gating parameters to obtain the fifth feature of the current time step; If the robot's storage space meets the preset conditions, the fifth feature of the current time step will be flattened, the flattened result will be spliced ​​with the first feature of the current time step, and the spliced ​​result will be used as the first modal feature of the current time step. If the robot's storage space does not meet the preset conditions, then the fifth feature of the current time step is multiplied based on the first projection matrix, and the fifth feature of the current time step is added based on the second projection matrix. Based on the multiplication result, the addition result, and the first feature of the current time step, the first modal feature of the current time step is generated.

4. The method according to claim 2, characterized in that, The step of inputting the input data of the current time step into the VLA model to obtain the control command of the robot at the current time step includes: Extract a first pooling representation, a second pooling representation, and a third pooling representation from the input data at the current time step. The first pooling representation is determined based on inference features, the second pooling representation is determined based on action features, and the third pooling representation is determined based on the last feature unit in the second modality features at the current time step. Based on the first pooling representation, the second pooling representation, and the third pooling representation, the control command for the robot's current time step is determined using the contrastive learning unit in the VLA model.

5. The method according to claim 4, characterized in that, The contrastive learning unit in the VLA model is trained based on a first contrastive loss and a second contrastive loss; the first contrastive loss is used to align inference features and action features, and the second contrastive loss is used to align inference features and multimodal features.

6. The method according to claim 5, characterized in that, The VLA model is trained based on a total loss, which includes a first contrastive loss, a second contrastive loss, and a regression loss. The regression loss is determined based on the control command labels of the training samples and the control command prediction results of the training samples output by the VLA model.

7. A robot control command generation device, characterized in that, The device includes: The information acquisition module is used to acquire the robot's target task, as well as the observed images, user commands, and robot status at the current time step. The first modality feature generation module is used to generate first modality features based on the observed image and ontology state at the current time step and the observed image and ontology state at the historical time steps of the target task. The second modality feature generation module is used to generate second modality features based on user instructions at the current time step; The input data generation module is used to sequentially concatenate the first modality features, the second modality features, the pre-determined inference features, and the pre-determined action features to obtain the input data for the current time step; the inference features and the action features are determined by the VLA model when iterative training is completed; The control command generation module is used to input the input data of the current time step into the VLA model to obtain the control command of the robot at the current time step.

8. An electronic device, characterized in that, The electronic device includes: At least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the robot control instruction generation method according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that, when executed by a processor, implement the method for generating control instructions for the robot as described in any one of claims 1-6.

10. A computer program product, characterized in that, It includes a computer program that, when executed by a processor, implements the method for generating control instructions for a robot according to any one of claims 1-6.

Images