Robot control methods and robots

Abstract

This application relates to a robot control method and a robot, belonging to the field of robot control technology. The method includes: acquiring current visual observation data, current body state data, current control commands, and historical action sequences of a target robot at the current moment; encoding the current visual observation data, current body state data, and current control commands using a multimodal coding model to obtain the target robot's current hidden state information; inputting the current hidden state information and historical action sequences into a world model to obtain predicted hidden state information of the target robot at at least one future moment; inputting the predicted hidden state information, current hidden state information, historical action sequences, and current control commands into a visual language action model to obtain the target robot's current action sequence at the current moment; and sending the current action sequence to the target robot's actuators. This method can improve the physical continuity of robot actions.

Description

Technical Field

[0001] This application relates to the field of robot control technology, and in particular to a robot control method and a robot. Background Technology

[0002] In the field of robot control, models are typically used to output motion control signals to control the robot. Related technical solutions usually utilize models to analyze the robot's current observation data and generate motion control signals.

[0003] However, the aforementioned motion control methods suffer from insufficient physical continuity of motion due to the lack of state constraints. Therefore, improving the physical continuity of robot motion is a problem worthy of attention. Summary of the Invention

[0004] Based on this, this application addresses the aforementioned technical problems by providing a robot control method and a robot that can improve the physical continuity of robot movements.

[0005] In a first aspect, this application provides a robot control method applied to the control system of a target robot, comprising:

[0006] Acquire the target robot's current visual observation data, current body state data, current control commands, and historical action sequences prior to the current moment;

[0007] The current visual observation data, current body state data, and current control commands are encoded using a multimodal coding model to obtain the current hidden state information of the target robot at the current moment.

[0008] By inputting the current hidden state information and the historical action sequence into the world model, the predicted hidden state information of the target robot at at least one future moment can be obtained.

[0009] The predicted hidden state information, current hidden state information, historical action sequence and current control command are input into the visual language action model to obtain the current action sequence of the target robot at the current moment;

[0010] Send the current action sequence to the actuator of the target robot.

[0011] The aforementioned robot control method first encodes the current visual observation data, current body state data, and current control commands using a multimodal coding model to obtain the current hidden state information of the target robot at the current moment. Then, the current hidden state information and historical action sequences are input into the world model to obtain the predicted hidden state information of the target robot at at least one future moment. Finally, the predicted hidden state information, current hidden state information, historical action sequences, and current control commands are input into the visual language action model to obtain the current action sequence of the target robot at the current moment. By introducing the predicted hidden state information of the world model into the generation process of the current action sequence, the generated current action sequence takes into account both the current multimodal data and the future dynamic evolution trend, thereby improving the physical consistency of the current action sequence.

[0012] In one embodiment, the world model is implemented based on RSSM; the world model includes a deterministic state branch network and a stochastic state branch network; inputting the current hidden state information and historical action sequence into the world model to obtain the predicted hidden state information of the target robot at at least one future moment includes: inputting the current hidden state information and historical action sequence into the deterministic state branch network to predict the deterministic state information at at least one future moment; inputting the current hidden state information and historical action sequence into the stochastic state branch network to predict the stochastic state information at at least one future moment; and concatenating the deterministic state information and the stochastic state information to obtain the predicted hidden state information.

[0013] In this embodiment, by obtaining deterministic state information and random state information, and concatenating the deterministic state information and random state information, after inputting the predicted latent state information into the visual language action model, the visual language action model can simultaneously process the predictable physical laws and unpredictable environmental uncertainties in the predicted latent state information, thereby predicting a more accurate current action sequence and improving the physical consistency of the current action sequence.

[0014] In one embodiment, the current visual observation data includes at least one of a forward-viewing camera image, a wrist-viewing camera image, and a side-viewing camera image; the current body state data includes at least one of joint angle data, end-effector pose data, and gripper state.

[0015] In this embodiment, by acquiring at least one current visual observation data and at least one current ontology state data, more accurate prediction of hidden state information can be obtained in subsequent processes, laying the foundation for predicting a more accurate current action sequence, thereby improving the physical consistency of the current action sequence.

[0016] In one embodiment, the multimodal coding model includes a visual coding network, a language coding network, a state coding network, and a feature fusion network. The multimodal coding model encodes current visual observation data, current ontology state data, and current control commands to obtain the target robot's current hidden state information at the current moment. This includes: inputting the current visual observation data into the visual coding network for feature extraction to obtain visual features corresponding to the visual observation data; inputting the current control commands into the language coding network for feature extraction to obtain semantic features corresponding to the control commands; inputting the current ontology state data into the state coding network for feature extraction to obtain state features corresponding to the ontology state data; and inputting the visual features, semantic features, and state features into the feature fusion network, which performs fusion analysis on the input features and outputs the target robot's current hidden state information at the current moment.

[0017] In this embodiment, by using the coding networks in the multimodal coding model to extract features from the corresponding data, more accurate and comprehensive fused features can be obtained, which in turn can obtain more accurate current hidden state information, laying the foundation for predicting more accurate current action sequences, thereby improving the physical consistency of the current action sequence.

[0018] In one embodiment, the predicted hidden state information, current hidden state information, historical action sequence, and current control command are input into the visual language action model to obtain the current action sequence of the target robot at the current moment. This includes: mapping the predicted hidden state information, current hidden state information, historical action sequence, and current control command to the same feature space, and fusing the features mapped to the same feature space to obtain joint context features; and inputting the joint context features into the visual language action model to obtain the current action sequence of the target robot at the current moment.

[0019] In this embodiment, by obtaining joint context features based on predicted hidden state information, current hidden state information, historical action sequence, and current control command, and inputting the joint context features into the visual language action model, the visual language action model can perform joint processing on each feature, thereby obtaining a more accurate current action sequence and improving the physical consistency of the current action sequence.

[0020] In one embodiment, features mapped to the same feature space are fused to obtain joint context features, including: using at least one fusion method among splicing, linear projection, gated weighted fusion and cross-attention fusion to fuse features mapped to the same feature space to obtain joint context features.

[0021] In this embodiment, by employing at least one of the following fusion methods—splicing, linear projection, gated weighted fusion, and cross-attention fusion—features mapped to the same feature space are fused. This allows for better fusion of multiple features within the same feature space, resulting in joint context features that are easier for the visual language action model to understand. Consequently, when the visual language action model performs joint processing on various features, it obtains a more accurate current action sequence, thereby improving the physical consistency of the current action sequence.

[0022] In one embodiment, the current action sequence includes action blocks based on at least one control quantity; the control quantity includes at least one of joint position increment, joint velocity, end-effector pose increment, and gripper opening / closing amount.

[0023] In this embodiment, the current action sequence, composed of at least one of the control quantities among joint position increment, joint velocity, end-effector pose increment, and gripper opening / closing amount, can achieve continuous motion control of the target robot and improve the physical consistency of the target robot's actions.

[0024] In one embodiment, the world model and the visual language action model are trained according to the following steps: acquiring sample visual observation data, sample ontology state data, sample control commands, and sample action sequences prior to the sample time of the sample robot at the sample time; encoding the sample visual observation data, sample ontology state data, and sample control commands through a first initial model to obtain the first sample hidden state information of the sample robot at the sample time; inputting the first sample hidden state information and the sample action sequence into a second initial model to obtain the second sample hidden state information of the sample robot after the sample time; inputting the second sample hidden state information, the first sample hidden state information, the sample action sequence, and the sample control commands into a third initial model to obtain the predicted action sequence of the sample robot at the sample time; determining a first loss based on the second sample hidden state information and the true hidden state information; determining an action supervision loss based on the predicted action sequence and the true action sequence; and jointly training the first initial model, the second initial model, and the third initial model based on the first loss and the action supervision loss to obtain a multimodal coding model, a world model, and a visual language action model.

[0025] In this embodiment, the first initial model, the second initial model, and the third initial model are jointly trained using a first loss and an action supervision loss. This enables a chain-like reasoning process involving a multimodal coding model, a world model, and a visual-language-action model during robot control. First, the multimodal coding model encodes the current visual observation data, the current ontological state data, and the current control commands to obtain the target robot's current hidden state information at the current moment. Then, the current hidden state information and historical action sequences are input into the world model to obtain the target robot's predicted hidden state information at at least one future moment. Finally, the predicted hidden state information, the current hidden state information, the historical action sequences, and the current control commands are input into the visual-language-action model to obtain the target robot's current action sequence at the current moment. The world model can incorporate the predicted hidden state information into the generation process of the current action sequence, ensuring that the generated current action sequence takes into account both the current multimodal data and future dynamic evolution trends, thereby improving the physical consistency of the current action sequence.

[0026] In one embodiment, the first loss includes a hidden state loss and an observation reconstruction loss. Determining the first loss based on the second sample hidden state information and the true hidden state information includes: determining the hidden state loss based on the information difference between the second sample hidden state information and the true hidden state information; decoding the second sample hidden state information to obtain reconstructed visual observation data and reconstructed ontology state data; and decoding the true hidden state information to obtain true visual observation data and true ontology state data; and determining the observation reconstruction loss based on the reconstructed visual observation data, the reconstructed ontology state data, the true visual observation data, and the true ontology state data.

[0027] In this embodiment, the hidden state loss can be used to ensure that the true hidden state information encoded by the first initial model is consistent with the hidden state information of the second sample predicted by the second initial model. The observation reconstruction loss can be used to reconstruct visual observation data and ontology state data, enabling the multimodal coding model to accurately encode visual observation data and ontology state data, and enabling the world model to accurately predict hidden state information. This lays the foundation for obtaining an accurate current action sequence and improving the physical consistency of the current action sequence during subsequent robot control.

[0028] In one embodiment, determining the action supervision loss based on the predicted action sequence and the actual action sequence includes: determining the action supervision loss based on the action difference between the predicted action sequence and the actual action sequence.

[0029] In this embodiment, by using motion supervision loss, the predicted motion sequence obtained by the third initial model can be constrained to be close to the real motion sequence, so that the visual language motion model can accurately predict the current motion sequence. This lays the foundation for obtaining an accurate current motion sequence and improving the physical consistency of the current motion sequence in the subsequent robot control process.

[0030] Secondly, this application also provides a robot control device, configured in the control system of a target robot, comprising:

[0031] The acquisition module is used to acquire the target robot's current visual observation data, current body state data, current control commands, and historical action sequences prior to the current moment.

[0032] The encoding module is used to encode the current visual observation data, the current body state data and the current control command through a multimodal encoding model to obtain the current hidden state information of the target robot at the current moment;

[0033] The first input module is used to input the current hidden state information and the historical action sequence into the world model to obtain the predicted hidden state information of the target robot at at least one future moment.

[0034] The second input module is used to input the predicted hidden state information, the current hidden state information, the historical action sequence and the current control command into the visual language action model to obtain the current action sequence of the target robot at the current moment;

[0035] The sending module is used to send the current action sequence to the actuator of the target robot.

[0036] Thirdly, this application also provides a robot including a memory and a control system, the memory storing a computer program, and the control system including at least one processor, which executes the computer program to implement the steps of the method described above.

[0037] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described above.

[0038] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the method described in any of the above aspects.

[0039] Regarding the beneficial effects of any of the technical solutions in the second to fifth aspects mentioned above, refer to the beneficial effects of the corresponding technical solutions in the first aspect; repeated examples will not be listed here. Attached Figure Description

[0040] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0041] Figure 1 This is a schematic diagram of an optional flowchart of a robot control method in one embodiment;

[0042] Figure 2 This is a schematic diagram of an optional process for the joint training of a multimodal coding model, a world model, and a visual language action model in one embodiment.

[0043] Figure 3 This is a schematic diagram of an optional structure of the robot control system in one embodiment;

[0044] Figure 4 This is a schematic diagram of an optional structure of a robot control device in one embodiment. Detailed Implementation

[0045] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of this application.

[0046] The terms "first," "second," etc., used in this application may be used to describe various elements, but these elements are not limited by these terms. These terms are used only to distinguish the first element from the second element. The terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" used in this application refers to two or more. The term "and / or" used in this application refers to one of the embodiments, or any combination of multiple embodiments.

[0047] The robot control method provided in this application can be applied to the control system of a target robot, such as... Figure 1 As shown, it includes the following steps:

[0048] S110: Acquire the target robot's current visual observation data, current body state data, current control commands, and historical action sequences prior to the current moment.

[0049] Here, the current visual observation data can be understood as the visual image data at the current moment, and can be denoted as: In some embodiments, the current visual observation data may include at least one of a front-view camera image, a wrist-view camera image, and a side-view camera image. In some embodiments, acquiring the target robot's current visual observation data at the current moment may include at least one of the following: receiving a front-view camera image acquired by a front-view camera at the current moment; receiving a wrist-view camera image acquired by a wrist-view camera at the current moment; and receiving a side-view camera image acquired by a side-view camera at the current moment.

[0050] The current ontology state data can be understood as data reflecting the ontology state of the target robot at the current moment, and can be denoted as: For example, the current body state data may include at least one of the robot's joint angles, end effector pose data, and gripper state. Here, joint angles can be understood as the rotation angles of the joints; end effector pose data can be understood as the position and orientation data of the end effector (such as a gripper) in a specified coordinate system; and gripper state can be understood as the opening and closing state of the gripper.

[0051] Here, the current control command can be understood as the command to control the target robot at the current moment, and can be denoted as: For example, the current control command may include at least one of voice commands, button commands, remote control commands, and gesture commands.

[0052] Here, the historical action sequence can be understood as the sequence of actions at least one historical moment prior to the current moment, and can be denoted as: ,in, Representing historical moments The next action block; , The variable is a positive integer. A sequence of historical actions at any given historical moment can include an action block composed of at least one control variable based on that historical moment. An action block can be understood as a series of continuous actions required for the target robot to perform a complete task. The control variable includes at least one of joint position increment, joint velocity, end-effector pose increment, and gripper opening / closing amount. A joint position increment can be understood as the difference between the joint angle at a later time step and the joint angle at a previous time step; a joint velocity can be understood as the ratio of the joint position increment to the time interval; an end-effector pose increment can be understood as the difference between the end-effector pose at a later time step and the end-effector pose at a previous time step; and a gripper opening / closing amount can be understood as the maximum relative displacement distance between the two gripper tips during the opening / closing process.

[0053] S120 uses a multimodal coding model to encode the current visual observation data, current body state data, and current control commands to obtain the current hidden state information of the target robot at the current moment.

[0054] Multimodal coding models can be used to encode multimodal data to obtain its latent state information. The current latent state information can be understood as the internal state information of the multimodal data at the current moment, captured by the multimodal coding model.

[0055] In some embodiments, the multimodal coding model may include a visual coding network, a language coding network, a state coding network, and a feature fusion network. The visual coding network extracts visual features from the current visual observation data; the language coding network extracts semantic features from the current control commands; the state coding network extracts state features from the current ontology state data; and the feature fusion network performs fusion analysis on the input features to obtain the current latent state information.

[0056] Accordingly, encoding the current visual observation data, current ontology state data, and current control commands using a multimodal coding model to obtain the target robot's current hidden state information at the current moment can include: inputting the current visual observation data into a visual coding network for feature extraction to obtain the visual features corresponding to the visual observation data; inputting the current control commands into a language coding network for feature extraction to obtain the semantic features corresponding to the control commands; inputting the current ontology state data into a state coding network for feature extraction to obtain the state features corresponding to the ontology state data; and inputting the visual features, semantic features, and state features into a feature fusion network, which performs fusion analysis on the input features and outputs the target robot's current hidden state information at the current moment.

[0057] In some embodiments, the feature fusion network can align, concatenate, or project the input features to obtain the smallest processing unit Token sequence or latent representation Latent, i.e., the current latent state information.

[0058] In some embodiments, the current visual observation data Current ontology state data and current control commands By inputting the multimodal coding model, the current hidden state information can be obtained: ;in, This represents the coding function corresponding to the multimodal coding model.

[0059] In this embodiment, by using the coding networks in the multimodal coding model to extract features from the corresponding data, more accurate and comprehensive fused features can be obtained, which in turn can obtain more accurate current hidden state information, laying the foundation for predicting more accurate current action sequences, thereby improving the physical consistency of the current action sequence.

[0060] S130, input the current hidden state information and historical action sequence into the world model to obtain the predicted hidden state information of the target robot at at least one future moment.

[0061] In this context, the predicted latent state information can be understood as the predicted internal state information of the target robot at at least one future moment.

[0062] In some embodiments, the world model can be implemented based on a Recurrent State Space Model (RSSM). The world model may include deterministic state branch networks and stochastic state branch networks.

[0063] Among them, deterministic state branch networks can be used to predict the deterministic state information of the target robot; random state branch networks can be used to predict the random state information of the target robot. Deterministic state information can be understood as deterministic state information, and random state information can be understood as random state information.

[0064] In some embodiments, the current hidden state information and historical action sequences can be input into a deterministic state branch network to predict deterministic state information at least one future time. The current hidden state information and historical action sequences can be input into a stochastic state branch network to predict stochastic state information at least one future time. By concatenating the deterministic and stochastic state information, the predicted hidden state information of the target robot at at least one future time can be obtained.

[0065] In some embodiments, deterministic state information and random state information can be input into a splicing network in a world model. The deterministic state information and random state information are spliced ​​together by the splicing network to obtain the predicted latent state information of the target robot at at least one future moment.

[0066] In some embodiments, the current hidden state information and historical action sequences By inputting the world model, we can obtain the predicted hidden state information of the target robot in the next time step: ,in, It can represent the prediction function corresponding to the world model.

[0067] In this embodiment, by obtaining deterministic state information and random state information, and concatenating the deterministic state information and random state information, after inputting the predicted latent state information into the visual language action model, the visual language action model can simultaneously process the predictable physical laws and unpredictable environmental uncertainties in the predicted latent state information, thereby predicting a more accurate current action sequence and improving the physical consistency of the current action sequence.

[0068] S140, the predicted hidden state information, current hidden state information, historical action sequence and current control command are input into the visual language action model to obtain the current action sequence of the target robot at the current moment.

[0069] The current action sequence may include action blocks composed of at least one control variable based on the current moment. The control variable may include at least one of joint position increment, joint velocity, end-effector pose increment, and gripper opening / closing amount.

[0070] In some embodiments, the predicted hidden state information, the current hidden state information, the historical action sequence and the current control command are mapped to the same feature space, and the features mapped to the same feature space are fused to obtain joint context features.

[0071] Mapping the predicted hidden state information, the current hidden state information, the historical action sequence, and the current control command to the same feature space can be understood as unifying the predicted hidden state information, the current hidden state information, the historical action sequence, and the current control command into a feature vector of the same dimension.

[0072] In some embodiments, the predicted latent state information, the current latent state information, the historical action sequence, and the current control command can be input into the feature mapping network in the visual language action model to obtain features in the same feature space. Then, the features in the feature space are input into the fusion network in the visual language action model to obtain joint context features.

[0073] In this embodiment, by obtaining joint context features based on predicted hidden state information, current hidden state information, historical action sequence, and current control command, and inputting the joint context features into the visual language action model, the visual language action model can perform joint processing on each feature, thereby obtaining a more accurate current action sequence and improving the physical consistency of the current action sequence.

[0074] In some embodiments, at least one of the following fusion methods can be used to fuse features mapped to the same feature space to obtain joint context features: splicing, linear projection, gated weighted fusion, and cross-attention fusion.

[0075] In this embodiment, by employing at least one of the following fusion methods—splicing, linear projection, gated weighted fusion, and cross-attention fusion—features mapped to the same feature space are fused. This allows for better fusion of multiple features within the same feature space, resulting in joint context features that are easier for the visual language action model to understand. Consequently, when the visual language action model performs joint processing on various features, it obtains a more accurate current action sequence, thereby improving the physical consistency of the current action sequence.

[0076] In some embodiments, the hidden state information will be predicted. Current hidden state information Historical action sequence and current control commands By mapping features to the same feature space and fusing the features mapped to the same feature space, joint context features can be obtained: ,in, The feature fusion function can be one of the following: splicing, linear projection, gated weighted fusion, or cross-attention fusion.

[0077] In some embodiments, joint context features The input is fed into the action sequence prediction network of the Vision-Language-Action (VLA) model to obtain the current action sequence of the target robot at the current moment: ,in, This represents the action generation function corresponding to the visual language action model.

[0078] In some embodiments, the output layer of the visual language action model can output at least one sequence of actions at a future time, which can be represented as follows: This can improve control smoothness and execution efficiency to some extent.

[0079] S150, send the current action sequence to the actuator of the target robot.

[0080] In some embodiments, the current action sequence can be The data is sent to the actuator of the target robot, which can then execute the current action sequence. Matching actions to obtain visual observation data for the next moment. and ontology state data By repeating the above steps, closed-loop continuous control of the target robot can be achieved.

[0081] In the aforementioned robot control method, the current visual observation data, current body state data, and current control commands are first encoded using a multimodal coding model to obtain the current hidden state information of the target robot at the current moment. Then, the current hidden state information and historical action sequences are input into the world model to obtain the predicted hidden state information of the target robot at at least one future moment. Finally, the predicted hidden state information, current hidden state information, historical action sequences, and current control commands are input into the visual language action model to obtain the current action sequence of the target robot at the current moment. By introducing the predicted hidden state information of the world model into the generation process of the current action sequence, the generated current action sequence takes into account both the current multimodal data and the future dynamic evolution trend, thereby improving the physical consistency of the current action sequence.

[0082] Based on the technical solutions of the above embodiments, this application also provides an optional embodiment, in which a joint training method for a multimodal coding model, a world model, and a visual language action model is described in detail.

[0083] See Figure 2 The joint training method for the multimodal coding model, world model, and visual-language-action model shown includes:

[0084] S210: Acquire the sample robot's visual observation data, sample body state data, sample control commands, and sample action sequence before the sample time at the sample moment.

[0085] The sample robot can be the target robot, or it can be another robot with the same attributes (such as type, state, function, etc.) as the target robot.

[0086] In some embodiments, visual observation data, body state data, control commands, and sequence of actions of the sample robot at multiple sample moments can be acquired, which can be represented in chronological order as follows: ,in, Indicates the first Visual observation data of samples at each sample time point; Show the first Sample ontology state data at each sample time point; This indicates the sample control command (the sample control command is the same at all sample times). No. The sequence of actions of a sample before each sample time point; , It is a positive integer.

[0087] S220: The visual observation data, physical state data and control commands of the sample are encoded by the first initial model to obtain the first hidden state information of the sample robot at the sample time.

[0088] The first initial model can be understood as an untrained multimodal coding model.

[0089] The process of encoding the sample visual observation data, sample body state data and sample control instructions through the first initial model is similar to the process in S120, and will not be described in detail here.

[0090] S230, input the first sample hidden state information and the sample action sequence into the second initial model to obtain the second sample hidden state information of the sample robot after the sample time.

[0091] The second initial model can be understood as an untrained world model.

[0092] The process of inputting the first sample hidden state information and the sample action sequence into the second initial model to obtain the second sample hidden state information of the sample robot after the sample time is similar to the process in S130, and will not be described again here.

[0093] S240, input the second sample hidden state information, the first sample hidden state information, the sample action sequence and the sample control command into the third initial model to obtain the predicted action sequence of the sample robot at the sample time.

[0094] The third initial model can be understood as an untrained visual language action model.

[0095] The process of inputting the second sample hidden state information, the first sample hidden state information, the sample action sequence, and the sample control command into the third initial model to obtain the predicted action sequence of the sample robot at the sample time is similar to the process in S140, and will not be repeated here.

[0096] S250, determine the first loss based on the hidden state information of the second sample and the true hidden state information.

[0097] In some embodiments, the first loss may include hidden state loss and observation reconstruction loss.

[0098] Accordingly, determining the first loss based on the second sample hidden state information and the true hidden state information may include: determining the hidden state loss based on the information difference between the second sample hidden state information and the true hidden state information; decoding the second sample hidden state information to obtain reconstructed visual observation data and reconstructed ontology state data; and decoding the true hidden state information to obtain true visual observation data and true ontology state data; and determining the observation reconstruction loss based on the reconstructed visual observation data, the reconstructed ontology state data, the true visual observation data, and the true ontology state data.

[0099] The true hidden state information can be understood as the true hidden state information of the sample robot after the sample time by encoding the sample visual observation data, sample body state data and sample control instructions after the sample time by the first initial model.

[0100] Among them, the hidden state loss can be used to ensure that the true hidden state information encoded by the first initial model is consistent with the hidden state information of the second sample predicted by the second initial model.

[0101] In some embodiments, the hidden state loss can be expressed as: ;in, This represents the hidden state information of the second sample. This represents the true hidden state information.

[0102] In some embodiments, the visual observation reconstruction loss can be determined based on the difference between the reconstructed visual observation data and the real visual observation data; the state observation reconstruction loss can be determined based on the difference between the reconstructed ontology state data and the real ontology state data.

[0103] The visual observation reconstruction loss can be used to reconstruct visual observation data based on the latent state information of the second sample predicted by the second initial model. In some embodiments, the visual observation reconstruction loss can be at least one of mean squared error loss (MSE), mean absolute error loss (L1 loss), or perceptual loss.

[0104] The state observation reconstruction loss can be used to reconstruct ontology state data based on the latent state information of the second sample predicted by the second initial model. In some embodiments, the state observation reconstruction loss can be at least one of MSE or Huber loss.

[0105] In some embodiments, the visual observation reconstruction loss can be expressed as: ;in, Represents real visual observation data, This indicates the reconstruction of visual observation data.

[0106] In this embodiment, the hidden state loss can be used to ensure that the true hidden state information encoded by the first initial model is consistent with the hidden state information of the second sample predicted by the second initial model. The observation reconstruction loss can be used to reconstruct visual observation data and ontology state data, enabling the multimodal coding model to accurately encode visual observation data and ontology state data, and enabling the world model to accurately predict hidden state information. This lays the foundation for obtaining an accurate current action sequence and improving the physical consistency of the current action sequence during subsequent robot control.

[0107] S260, determine the action supervision loss based on the predicted action sequence and the actual action sequence.

[0108] In some embodiments, the action supervision loss can be determined based on the action difference between the predicted action sequence and the actual action sequence.

[0109] The actual action sequence can be understood as the actual action sequence of the sample robot at the sample time.

[0110] The action supervision loss can be used to constrain the predicted action sequence obtained from the third initial model to be close to the actual action sequence. In some embodiments, the action supervision loss can be at least one of mean squared error loss, L1 loss, or Huber loss.

[0111] In some embodiments, the action supervision loss can be expressed as: ,in, Indicates the predicted action sequence, This represents the actual sequence of actions.

[0112] In this embodiment, by using motion supervision loss, the predicted motion sequence obtained by the third initial model can be constrained to be close to the real motion sequence, so that the visual language motion model can accurately predict the current motion sequence. This lays the foundation for obtaining an accurate current motion sequence and improving the physical consistency of the current motion sequence in the subsequent robot control process.

[0113] S270, based on the first loss and the action supervision loss, jointly train the first initial model, the second initial model and the third initial model to obtain the multimodal coding model, the world model and the visual language action model.

[0114] In some embodiments, the model loss can be determined based on the first loss and the action supervision loss; the model loss is then used to jointly train the first initial model, the second initial model, and the third initial model.

[0115] In some implementations, the hidden state loss, observation reconstruction loss, and action supervision loss in the first loss can be weighted and summed to obtain the model loss.

[0116] In some embodiments, the posterior hidden state distribution can be determined based on the sample action sequence before the sample time, the first sample hidden state information, and the sample visual observation data at and before the sample time; and the prior hidden state distribution can be determined based on the sample action sequence before the sample time and the first sample hidden state information; and the distribution constraint loss can be determined based on the distribution difference between the posterior hidden state distribution and the prior hidden state distribution.

[0117] In some embodiments, the distribution constraint loss can be expressed as: ,in, This represents the Kullback-Leibler divergence function. This represents the visual observation data of samples at and before the sample time. This represents the actions of the sample prior to the sample time. This represents the hidden state information of the first sample. This represents the posterior hidden state distribution. This represents the sequence of actions performed on a sample prior to the sample time. This represents the distribution of the prior hidden states.

[0118] In some embodiments, posterior hidden state information can be determined based on the predicted action sequence of the sample robot and the first sample hidden state information; dynamic consistency loss can be determined based on the posterior hidden state information and the second sample hidden state information.

[0119] In some embodiments, the dynamic consistency loss can be expressed as: ,in, This represents the hidden state information of the first sample; Indicates the predicted action sequence; Represents the true hidden state information. This represents the state transition function of the world model.

[0120] In some embodiments, the first initial model, the second initial model, and the third initial model can be jointly trained based on the first loss, the action supervision loss, the distribution constraint loss, and the dynamic consistency loss to obtain a multimodal coding model, a world model, and a visual language action model.

[0121] In this embodiment, the first initial model, the second initial model, and the third initial model are jointly trained using a first loss and an action supervision loss. This enables a chain-like reasoning process involving a multimodal coding model, a world model, and a visual-language-action model during robot control. First, the multimodal coding model encodes the current visual observation data, the current ontological state data, and the current control commands to obtain the target robot's current hidden state information at the current moment. Then, the current hidden state information and historical action sequences are input into the world model to obtain the target robot's predicted hidden state information at at least one future moment. Finally, the predicted hidden state information, the current hidden state information, the historical action sequences, and the current control commands are input into the visual-language-action model to obtain the target robot's current action sequence at the current moment. The world model can incorporate the predicted hidden state information into the generation process of the current action sequence, ensuring that the generated current action sequence takes into account both the current multimodal data and future dynamic evolution trends, thereby improving the physical consistency of the current action sequence.

[0122] After training is complete, the parameters of the multimodal coding model, the world model, and the visual language action model can be saved.

[0123] This application also provides a robot control system that can be applied to the above-described robot control method.

[0124] join Figure 3 The robot control system shown may include a multimodal input module, a multimodal encoding module, a world model prediction module, a visual language action generation module, and an action execution and feedback module.

[0125] The multimodal input module can be used to receive the target robot's current visual observation data, current body state data, current control commands, and historical action sequences before the current moment.

[0126] The multimodal coding module is used to represent the current visual observation data, the current ontology state data, and the current control command into a unified latent space to obtain the current latent state information.

[0127] Furthermore, the multimodal coding module may include a visual coding unit, a state coding unit, a language coding unit, and a feature fusion unit. Specifically, the visual coding unit extracts visual features from the current visual observation data; the state coding unit extracts state features from the current ontology state data; the language coding unit extracts semantic features from the current control commands; and the feature fusion unit performs fusion processing such as alignment, splicing, or projection on the visual, semantic, and state features to form a unified token sequence or a unified latent representation.

[0128] The world model prediction module is used to obtain the predicted hidden state information of the target robot at at least one future moment based on the current hidden state information and the historical action sequence.

[0129] Furthermore, the world model prediction module may include a deterministic state branch unit, a stochastic state branch unit, and an information splicing unit. The deterministic state branch unit is used to predict at least one deterministic state information for a future time based on the current hidden state information and the historical action sequence; the stochastic state branch unit is used to predict at least one stochastic state information for a future time based on the current hidden state information and the historical action sequence; and the information splicing unit is used to obtain the predicted hidden state information based on the deterministic state information and the stochastic state information.

[0130] The action generation module is used to obtain the current action sequence of the target robot at the current moment based on the predicted hidden state information, the current hidden state information, the historical action sequence, and the current control command.

[0131] The action execution and feedback module is used to send the current action sequence to the actuator of the target robot, and to feed back the new visual observation data and new body state data after executing the current action sequence to the multimodal input module to form closed-loop control.

[0132] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.

[0133] Based on the same inventive concept, this application also provides a robot control device for implementing the robot control method described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations in one or more robot control device embodiments provided below can be found in the limitations of the robot control method described above, and will not be repeated here.

[0134] In one exemplary embodiment, such as Figure 4As shown, a robot control device is provided, configured in the control system of a target robot, including: an acquisition module 410, an encoding module 420, a first input module 430, a second input module 440, and a sending module 450, wherein:

[0135] The acquisition module 410 is used to acquire the target robot's current visual observation data, current body state data, current control commands, and historical action sequences before the current moment.

[0136] Encoding module 420 is used to encode the current visual observation data, the current body state data and the current control command through a multimodal encoding model to obtain the current hidden state information of the target robot at the current moment;

[0137] The first input module 430 is used to input the current hidden state information and the historical action sequence into the world model to obtain the predicted hidden state information of the target robot at at least one future moment.

[0138] The second input module 440 is used to input the predicted hidden state information, the current hidden state information, the historical action sequence and the current control command into the visual language action model to obtain the current action sequence of the target robot at the current moment;

[0139] The sending module 450 is used to send the current action sequence to the actuator of the target robot.

[0140] In one embodiment, the world model is implemented based on RSSM; the world model includes a deterministic state branch network and a random state branch network; the first input module 430 includes: a first prediction unit, used to input the current hidden state information and the historical action sequence into the deterministic state branch network to predict at least one deterministic state information at a future time; a second prediction unit, used to input the current hidden state information and the historical action sequence into the random state branch network to predict at least one random state information at a future time; and a splicing unit, used to splice the deterministic state information and the random state information to obtain predicted hidden state information.

[0141] In one embodiment, the current visual observation data includes at least one of a forward-viewing camera image, a wrist-viewing camera image, and a side-viewing camera image; the current body state data includes at least one of joint angle data, end-effector pose data, and gripper state.

[0142] In one embodiment, the multimodal coding model includes a visual coding network, a language coding network, a state coding network, and a feature fusion network; the coding module 420 includes: a first extraction unit, used to input current visual observation data into the visual coding network for feature extraction to obtain visual features corresponding to the visual observation data; a second extraction unit, used to input current control commands into the language coding network for feature extraction to obtain semantic features corresponding to the control commands; a third extraction unit, used to input current ontology state data into the state coding network for feature extraction to obtain state features corresponding to the ontology state data; and a fusion unit, used to input visual features, semantic features, and state features into the feature fusion network to obtain the current hidden state information of the target robot at the current moment, which is obtained by the feature fusion network performing fusion analysis on the input features.

[0143] In one embodiment, the second input module 440 includes: a mapping unit, used to map the predicted hidden state information, the current hidden state information, the historical action sequence and the current control command to the same feature space, and to perform fusion processing on the features mapped to the same feature space to obtain joint context features; and a first input unit, used to input the joint context features into the visual language action model to obtain the current action sequence of the target robot at the current moment.

[0144] In one embodiment, the mapping unit is specifically used to: perform fusion processing on features mapped to the same feature space using at least one of the following fusion methods: splicing, linear projection, gated weighted fusion, and cross-attention fusion, to obtain joint context features.

[0145] In one embodiment, the current action sequence includes action blocks based on at least one control quantity; the control quantity includes at least one of joint position increment, joint velocity, end-effector pose increment, and gripper opening / closing amount.

[0146] In one embodiment, the robot control device further includes a training module, comprising: an acquisition unit for acquiring sample visual observation data, sample body state data, sample control commands, and sample action sequences prior to the sample time of the sample robot at the sample time; an encoding unit for encoding the sample visual observation data, sample body state data, and sample control commands through a first initial model to obtain first sample latent state information of the sample robot at the sample time; a second input unit for inputting the first sample latent state information and the sample action sequence into a second initial model to obtain second sample latent state information of the sample robot after the sample time; a third input unit for inputting the second sample latent state information, the first sample latent state information, the sample action sequence, and the sample control commands into a third initial model to obtain a predicted action sequence of the sample robot at the sample time; a first determination unit for determining a first loss based on the second sample latent state information and the true latent state information; a second determination unit for determining an action supervision loss based on the predicted action sequence and the true action sequence; and a training unit for jointly training the first initial model, the second initial model, and the third initial model based on the first loss and the action supervision loss to obtain a multimodal coding model, a world model, and a visual-language-action model.

[0147] In one embodiment, the first loss includes a hidden state loss and an observation reconstruction loss; the first determining unit is specifically configured to: determine the hidden state loss based on the information difference between the second sample hidden state information and the true hidden state information; decode the second sample hidden state information to obtain reconstructed visual observation data and reconstructed ontology state data; and decode the true hidden state information to obtain true visual observation data and true ontology state data; and determine the observation reconstruction loss based on the reconstructed visual observation data, the reconstructed ontology state data, the true visual observation data, and the true ontology state data.

[0148] In one embodiment, the second determining unit is specifically configured to: determine the action supervision loss based on the action difference between the predicted action sequence and the actual action sequence.

[0149] Each module in the aforementioned robot control device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0150] In one exemplary embodiment, a robot is provided, including a memory and a control system. The memory stores a computer program, and the control system includes at least one processor that executes the computer program to implement the steps in the above-described method embodiments.

[0151] In one exemplary embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above-described method embodiments.

[0152] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0153] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program mentioned can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0154] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0155] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A robot control method characterized by, A control system applied to a target robot, the method comprising: Acquire the target robot's current visual observation data, current body state data, current control commands, and historical action sequences prior to the current moment; The current visual observation data, the current body state data, and the current control command are encoded using a multimodal coding model to obtain the current hidden state information of the target robot at the current moment. The current hidden state information and the historical action sequence are input into the world model to obtain the predicted hidden state information of the target robot at at least one future moment; The predicted hidden state information, the current hidden state information, the historical action sequence, and the current control command are input into the visual language action model to obtain the current action sequence of the target robot at the current moment. Send the current action sequence to the actuator of the target robot; The multimodal coding model, the world model, and the visual language action model are trained according to the following steps: Acquire the sample robot's visual observation data, sample body state data, sample control commands, and sample action sequence prior to the sample time at the sample moment; The visual observation data of the sample, the physical state data of the sample, and the control instructions of the sample are encoded by the first initial model to obtain the first hidden state information of the sample robot at the sample time. The first sample hidden state information and the sample action sequence are input into the second initial model to obtain the second sample hidden state information of the sample robot after the sample time. The hidden state loss is determined based on the information difference between the second sample hidden state information and the true hidden state information. Decoding the latent state information of the second sample yields reconstructed visual observation data and reconstructed ontology state data; and, Decoding the real hidden state information yields real visual observation data and real ontology state data; The observation reconstruction loss is determined based on the reconstructed visual observation data, the reconstructed ontology state data, the real visual observation data, and the real ontology state data. The second sample hidden state information, the first sample hidden state information, the sample action sequence, and the sample control command are input into the third initial model to obtain the predicted action sequence of the sample robot at the sample time. Based on the predicted action sequence and the actual action sequence, determine the action supervision loss; Based on the hidden state loss, the observation reconstruction loss, and the action supervision loss, the first initial model, the second initial model, and the third initial model are jointly trained to obtain a multimodal coding model, a world model, and a visual language action model.

2. The method of claim 1, wherein, The world model is implemented based on the Cyclic State-Space Model (RSSM); the world model includes a deterministic state branch network and a stochastic state branch network. The current hidden state information and the historical action sequence are input into the world model to obtain the predicted hidden state information of the target robot at at least one future moment, including: The current hidden state information and the historical action sequence are input into the deterministic state branch network to predict the deterministic state information of at least one future moment. The current hidden state information and the historical action sequence are input into the random state branch network to predict the random state information of at least one future moment; The predicted hidden state information is obtained by concatenating the deterministic state information and the random state information.

3. The method of claim 1, wherein, The current visual observation data includes at least one of the following: front-view camera image, wrist-view camera image, and side-view camera image; the current body state data includes at least one of the following: joint angle data, end-effector pose data, and gripper state.

4. The method of claim 1, wherein, The multimodal coding model includes a visual coding network, a language coding network, a state coding network, and a feature fusion network; The current visual observation data, the current ontological state data, and the current control command are encoded using a multimodal coding model to obtain the current hidden state information of the target robot at the current moment, including: The current visual observation data is input into the visual coding network for feature extraction to obtain the visual features corresponding to the visual observation data. The current control command is input into the language coding network for feature extraction to obtain the semantic features corresponding to the control command; The current ontology state data is input into the state coding network for feature extraction to obtain the state features corresponding to the ontology state data; The visual features, semantic features, and state features are input into the feature fusion network to obtain the current hidden state information of the target robot at the current moment. The feature fusion network performs fusion analysis on the input features and outputs the current hidden state information of the target robot at the current moment.

5. The method of claim 1, wherein, The predicted hidden state information, the current hidden state information, the historical action sequence, and the current control command are input into the visual language action model to obtain the current action sequence of the target robot at the current moment, including: The predicted hidden state information, the current hidden state information, the historical action sequence, and the current control command are mapped to the same feature space, and the features mapped to the same feature space are fused to obtain joint context features; The joint context features are input into the visual language action model to obtain the current action sequence of the target robot at the current moment.

6. The method of claim 5, wherein, Features mapped to the same feature space are fused to obtain joint context features, including: At least one of the following fusion methods—striking, linear projection, gated weighted fusion, and cross-attention fusion—is used to fuse features mapped to the same feature space to obtain joint context features.

7. The method of claim 1, wherein, The current action sequence includes an action block based on at least one control variable; The control quantities include at least one of joint position increment, joint velocity, end-effector pose increment, and gripper opening / closing quantity.

8. The method of claim 1, wherein, Based on the predicted action sequence and the actual action sequence, determine the action supervision loss, including: The action supervision loss is determined based on the action differences between the predicted action sequence and the actual action sequence.

9. A robot comprising a memory having stored a computer program and a control system comprising at least one processor, characterized in that, The processor, when executing the computer program, implements the steps of the method according to any one of claims 1 to 8.

Images