Robot strategy model training method and device, equipment, storage medium and robot control method

Abstract

The application provides a robot strategy model training method and device, equipment, a storage medium and a robot control method, and relates to the technical field of robot control. The method comprises the following steps: acquiring robot trajectory data; performing a masked trajectory learning task on a robot strategy model to be trained based on the trajectory data to obtain a trained robot strategy model, wherein the masked trajectory learning task comprises an action masking branch and a future observation latent representation masking branch; and performing flow matching action generation training on the trained robot strategy model by using observation latent representation, robot ontology state and language instructions at a current time based on model parameters of the trained robot strategy model to obtain a trained robot strategy model. The method reduces the inference delay of the robot strategy model.

Description

Technical Field

[0001] This application relates to the field of robot control technology, and in particular to a method, apparatus, device, storage medium for training a robot strategy model and a robot control method. Background Technology

[0002] In robot language-conditional manipulation tasks, large-scale vision-language-action models (VLAs) are often used as general robot policy models. For example, OpenVLA uses a VLA model with approximately 7 bytes of parameters to train on large-scale robot demonstration data.

[0003] However, existing VLA models have a huge number of parameters, high inference latency and deployment costs, making it difficult to meet the real-time requirements of robot closed-loop control. Summary of the Invention

[0004] This application provides a method, apparatus, device, storage medium, and robot control method for training a robot strategy model, which addresses the shortcomings of existing VLA models in meeting the real-time requirements of robot closed-loop control and reduces the inference latency of robot strategy models.

[0005] This application provides a method for training a robot policy model, including: Acquire robot trajectory data, which includes at least current observation data, future observation data, action blocks, robot body state, and language commands; Based on the trajectory data, the robot policy model to be trained is subjected to an occlusion trajectory learning task to obtain a trained robot policy model. The occlusion trajectory learning task includes an action occlusion branch and a future observation potential representation occlusion branch. The action occlusion branch is used to reconstruct the occluded action block based on the current observation potential representation, the future observation potential representation, the robot body state, and the language instruction. The future observation potential representation occlusion branch is used to predict the future observation potential representation based on the current observation potential representation and the action block. Based on the model parameters of the trained robot policy model, the robot policy model is trained by flow matching action generation using the observed latent representation, robot body state and language instructions at the current moment, to obtain a trained robot policy model.

[0006] In some embodiments, the current observation data and the future observation data are multi-view images at time t and time t+H respectively in the multi-view image observation sequence, where H is a preset positive integer; the action block consists of actions of H consecutive time steps in the action sequence; the robot body state is the state at time t in the robot body state sequence; and the language instruction is a task description corresponding to the entire trajectory.

[0007] In some embodiments, acquiring robot trajectory data includes: Multi-view image observations are input into a visual encoder to extract visual latent representations, and the visual latent representations are spliced ​​or fused to obtain the observation latent representation at the current moment. The multi-view image observations of future time moments are input into the exponential moving average target encoder to obtain the potential representation of the observations of future time moments. The parameters of the exponential moving average target encoder are updated with training through exponential moving average. Encode the robot's state into a state token using a linear or multilayer perceptron. The language instructions are input into the text encoder and encoded into language condition vectors; The observation potential representation and the state token together constitute the observation-side conditions, and the language condition vector serves as condition information.

[0008] In some embodiments, the step of performing an occlusion trajectory learning task on the robot policy model to be trained based on the trajectory data to obtain a trained robot policy model includes: In the motion masking branch, random noise is added to the motion block, and the random noise and the motion block are interpolated within the range of 0 to 1 in the stream time to obtain a noisy motion block; Using the current potential representation of observation, the future potential representation of observation, the robot's body state, and the language command as conditions, the robot policy model predicts the velocity field from the noisy action block to the real action block. The mean square error between the predicted value of the velocity field and the actual velocity target is used as the loss of the action occlusion branch, and the parameters of the robot policy model are updated based on the loss of the action occlusion branch.

[0009] In some embodiments, the step of performing an occlusion trajectory learning task on the robot policy model to be trained based on the trajectory data to obtain a trained robot policy model includes: In the future observation potential representation occlusion branch, the robot policy model predicts the future observation potential representation based on the current observation potential representation, the action block, the robot body state, and the language instruction. The prediction results are aligned with the L1 normalized future time observation potential representations output by the exponential moving average target encoder, and gradient backpropagation is stopped on the target side output by the exponential moving average target encoder. The L1 alignment loss is used as the loss of the future observation potential representation occlusion branch, and the parameters of the robot policy model are updated according to the loss of the future observation potential representation occlusion branch.

[0010] In some embodiments, the step of performing an occlusion trajectory learning task on the robot policy model to be trained based on the trajectory data to obtain a trained robot policy model further includes: The parameters of the robot policy model are updated using the weighted sum of the loss of the action occlusion branch and the loss of the future observation potential representation occlusion branch as the total loss, thus obtaining the trained robot policy model. The action occlusion branch and the future observation potential representation occlusion branch are sampled together in the same training batch or randomly rotated between different training batches.

[0011] In some embodiments, the robot policy model to be trained employs a hybrid transformer dual-stream backbone network, which includes an observation stream and an action stream. The observation stream is used to carry the current observation potential representation, the robot's body state, and the future observation potential representation or future potential slots used in the occlusion trajectory learning task. The motion flow is used to carry noisy motions, masked motions, or clean motions; The observation stream and the action stream exchange information through a joint attention mechanism.

[0012] In some embodiments, the step of training the trained robot policy model by using the model parameters of the trained robot policy model, the observed latent representation at the current moment, the robot's ontological state, and the language instructions to perform flow matching action generation training on the trained robot policy model to obtain a trained robot policy model includes: Based solely on the observed potential representation, robot body state, and language commands at the current moment, random noise is added to the real action block, and the random noise and the action block are interpolated within the range of 0 to 1 in the streaming time to obtain a noisy action block. The trained robot policy model is used to predict the velocity field from the noisy action block to the action block. The mean square error between the predicted value of the velocity field and the actual velocity target is used as the training loss to update the model parameters of the trained robot policy model.

[0013] This application also provides a robot control method, including: Acquire multi-view image observations, robot body state, and language commands at the current moment; The multi-view image observations are input into a visual encoder to extract the multi-view visual latent representations, and the multi-view visual latent representations are spliced ​​or fused to obtain the observation latent representation at the current moment. Using the observed potential representation at the current moment, the robot's body state, and the language instructions as conditions, the trained robot policy model, trained by the above-mentioned robot policy model training method, iteratively denoises from random noise action blocks to generate target action blocks. Send at least one action from the target action block to the robot controller for execution; Wherein, the action block is a sequence of robot actions for H consecutive time steps, where H is a preset positive integer, and the random noise action block is a random noise tensor with the same dimension as the action block.

[0014] This application also provides a training device for a robot strategy model, comprising the following modules: The data acquisition module is used to acquire robot trajectory data, which includes at least current observation data, future observation data, action blocks, robot body state, and language commands. The first training module is used to perform an occlusion trajectory learning task on the robot policy model to be trained based on the trajectory data, so as to obtain a trained robot policy model. The occlusion trajectory learning task includes an action occlusion branch and a future observation potential representation occlusion branch. The action occlusion branch is used to reconstruct the occluded action block based on the current observation potential representation, the future observation potential representation, the robot body state, and the language instruction. The future observation potential representation occlusion branch is used to predict the future observation potential representation based on the current observation potential representation and the action block. The second training module is used to perform flow matching action generation training on the trained robot policy model based on the model parameters of the trained robot policy model, using the observed potential representation, robot body state and language instructions at the current moment, to obtain a trained robot policy model.

[0015] This application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement a training method for a robot strategy model as described above.

[0016] This application also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a training method for a robot strategy model as described above.

[0017] This application also provides a computer program product, including a computer program that, when executed by a processor, implements a training method for a robot strategy model as described above.

[0018] The robot policy model training method, apparatus, device, storage medium, and robot control method provided in this application acquire robot trajectory data, perform occlusion trajectory learning tasks including action occlusion branches and future observation potential representation occlusion branches on the robot policy model to be trained based on the trajectory data, and then perform flow matching action generation training based on the trained model parameters to obtain a trained robot policy model, thereby reducing the inference latency of the robot policy model. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart illustrating the training method for the robot strategy model provided in this application.

[0021] Figure 2 This is a flowchart illustrating the robot control method provided in this application.

[0022] Figure 3 This is a schematic diagram of the structure of the training device for the robot strategy model provided in this application.

[0023] Figure 4 This is a schematic diagram of the structure of the electronic device provided in this application. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0025] In robot language-conditional manipulation tasks, large-scale vision-language-action models (VLAs) are often used as general robot policy models. For example, OpenVLA uses a VLA model with approximately 7 bytes of parameters to train on large-scale robot demonstration data.

[0026] However, existing VLA models have a huge number of parameters, high inference latency and deployment costs, making it difficult to meet the real-time requirements of robot closed-loop control.

[0027] Figure 1 This is a flowchart illustrating the training method for the robot strategy model provided in this application. For example... Figure 1 As shown, the method includes the following: Step 101: Obtain robot trajectory data, which includes at least current observation data, future observation data, action blocks, robot body state, and language commands.

[0028] The action block is a sequence of robot actions over H consecutive time steps, such as a trajectory starting from the current moment. Trajectory data can be obtained by collecting robot operation demonstration data or from existing robot operation datasets. Each trajectory includes a multi-view RGB image sequence, a robot body state sequence (e.g., joint positions, end effector pose, gripper state), a continuous action sequence (e.g., end effector pose increments, joint control variables, or gripper opening and closing actions), and the corresponding language task instructions. Training samples can be constructed using a trajectory sliding window; for each time step t, the observations at the current time step are selected. Future moment observation Action blocks = [ , , ..., ], ontology state and language instructions .

[0029] It should be noted that the multi-view images in the trajectory data are used to provide the robot with multi-angle visual perception of the environment, the ontology state is used to provide the robot's own physical state information, the action blocks are used to provide action supervision signals for continuous multi-step actions, and the language instructions are used to provide semantic conditions at the task level. The above multimodal data together constitute the data foundation for the subsequent occlusion trajectory learning task.

[0030] Step 102: Based on the trajectory data, perform an occlusion trajectory learning task on the robot policy model to be trained to obtain a trained robot policy model. The occlusion trajectory learning task includes an action occlusion branch and a future observation potential representation occlusion branch. The action occlusion branch is used to reconstruct the occluded action block based on the current observation potential representation, the future observation potential representation, the robot body state, and the language instruction. The future observation potential representation occlusion branch is used to predict the future observation potential representation based on the current observation potential representation and the action block.

[0031] Among them, the latent representation of observation is a low- or medium-dimensional feature representation obtained by encoding the robot's multi-view image observations by the visual encoder. The occlusion trajectory learning task involves occluding actions or future states within the robot's trajectory window and training a model to predict the occluded content based on existing information.

[0032] It should be noted that the action occlusion branch and the future observation potential representation occlusion branch are two complementary learning tasks. They impose constraints on the observation potential representation from the reverse and forward directions, respectively, so that the observation potential representation is simultaneously supervised by action interpretation and future prediction, thereby establishing a connection with robot actions and physical state transitions.

[0033] Step 103: Based on the model parameters of the trained robot policy model, and using the observed latent representation, robot body state and language instructions at the current moment, perform flow matching action generation training on the trained robot policy model to obtain a trained robot policy model.

[0034] Among them, flow matching is a continuous action generation training method that establishes continuous interpolation between real actions and random noise, and trains the model to predict the velocity field from noise to action. Step 103 uses the model parameters trained in step 102 as initialization, and trains only with the observed latent representation, ontology state and language instructions at the current time.

[0035] It should be noted that flow matching training constructs a continuous path from random noise to real action, enabling the model to learn to generate the velocity field of the action along this path, thus allowing iterative generation of the target action starting from any initial noise.

[0036] The above embodiments employ techniques such as acquiring robot trajectory data and performing a masking trajectory learning task that includes action masking branches and future observation potential representation masking branches, as well as stream matching action generation training based on trained model parameters, to achieve the technical effect of reducing the inference latency of robot policy models.

[0037] For example, training the robot policy model may include a Stage-1 occlusion trajectory learning module for randomly selecting a training branch within the robot trajectory window: Action occlusion branch: Given the current observation potential representation and the future observation potential representation, reconstruct the occluded action block through flow matching; Future Observation Latent Representation Masking Branch: Given the current observation latent representation and a clean action block, predict the future observation latent representation and perform L1 alignment with the future latent representation output by the Exponential Moving Average (EMA) target encoder. Here, L1 alignment refers to using the L1 loss function (i.e., mean absolute error) to measure the difference between the predicted features and the target features, used for supervised alignment of future observation latent representations in this application. Compared to L2 loss, L1 loss is more robust to outliers, which helps maintain the stability of the latent representation.

[0038] The training of the robot's policy model can also include a Stage-2 action policy training module, which trains a flow-matching action generation model using current observation latent representations, ontology states, and language commands as conditions. Deployment does not require input of future observation latent representations, and the inference process is consistent with Stage-2 training.

[0039] In some embodiments, the current observation data and the future observation data are multi-view images at time t and time t+H respectively in the multi-view image observation sequence, where H is a preset positive integer; the action block consists of actions of H consecutive time steps in the action sequence; the robot body state is the state at time t in the robot body state sequence; and the language instruction is a task description corresponding to the entire trajectory.

[0040] Where H is a preset positive integer representing the time span of the action block. Time t is the current time, and time t+H is the H-th time step after the current time.

[0041] It should be noted that the time interval between the current observation and the future observation is consistent with the time span of the action block, which is H time steps. This enables the model to establish a causal correspondence between "the change from the current state to the future state" and "continuous H-step actions".

[0042] For example, in the process of constructing the trajectory sliding window, for each time t, the t-th frame is taken from the multi-view image observation sequence as the current time observation and the t+H-th frame is taken as the future time observation. The actions of H consecutive time steps are taken from the action sequence to form an action block. At the same time, the ontology state at time t and the language instructions corresponding to the entire trajectory are obtained to form a training sample.

[0043] The above embodiments enable the model to learn the mapping relationship from state changes to actions by setting the time interval between the current observation and the future observation to be consistent with the length of the action block.

[0044] In some embodiments, acquiring robot trajectory data includes: Multi-view image observations are input into a visual encoder to extract visual latent representations, and the visual latent representations are spliced ​​or fused to obtain the observation latent representation at the current moment. The multi-view image observations of future time moments are input into the exponential moving average target encoder to obtain the potential representation of the observations of future time moments. The parameters of the exponential moving average target encoder are updated with training through exponential moving average. Encode the robot's state into a state token using a linear or multilayer perceptron. The language instructions are input into the text encoder and encoded into language condition vectors; The observation potential representation and the state token together constitute the observation-side conditions, and the language condition vector serves as condition information.

[0045] The visual encoder transforms multi-view image observations into dense patch-level visual latent representations. The parameters of the exponential moving average target encoder are not directly updated via gradient backpropagation, but rather slowly updated using an exponential moving average method following the training process to provide stable target predictions. The state token represents the feature vector of the robot's ontology state. The linguistic condition vector represents the semantic features of the linguistic commands.

[0046] It should be noted that extracting dense patch-level representations through a visual encoder can preserve local spatial information in the image; obtaining potential representations for future observations through an exponential moving average target encoder can provide stable supervision targets for the occlusion branches of these potential representations, reducing target drift during training. The potential representations for future observations and the state token together constitute the observation-side conditions, used to provide the model with information about its current environment and its own state; the linguistic condition vector, as conditional information, is used to provide the model with task-level semantic guidance.

[0047] The above embodiments reduce the risk of potential representation collapse caused by training compressed modules from scratch or self-supervised targets by using a stable visual latent space, dense patch representation, EMA target encoder, and stopping gradient alignment.

[0048] For example, an observation encoding module can be used to encode images into latent representations by inputting multi-view image observations and optional ontological states of the robot at the current moment. Preferably, the visual encoder is initialized with a visual base model that already has a stable visual representation space, such as a DINOv2-type visual encoder, and retains dense patch-level representations, rather than forcibly compressing them through a query compression module trained from scratch. A linguistic condition module can also be used to encode natural language task instructions into linguistic condition vectors. Preferably, a lightweight text encoder, such as a T5-small-type encoder, can be used. Linguistic conditions are injected into the policy backbone through cross-attention, rather than being mixed with observation tokens and action tokens into the same token stream.

[0049] In some embodiments, the step of performing an occlusion trajectory learning task on the robot policy model to be trained based on the trajectory data to obtain a trained robot policy model includes: In the motion masking branch, random noise is added to the motion block, and the random noise and the motion block are interpolated within the range of 0 to 1 in the stream time to obtain a noisy motion block; Using the current potential representation of observation, the future potential representation of observation, the robot's body state, and the language command as conditions, the robot policy model predicts the velocity field from the noisy action block to the real action block. The mean square error between the predicted value of the velocity field and the actual velocity target is used as the loss of the action occlusion branch, and the parameters of the robot policy model are updated based on the loss of the action occlusion branch.

[0050] Here, the flow time τ∈[0,1] represents the interpolation progress from noise to real motion; τ=0 represents pure noise, and τ=1 represents a real motion block. The velocity field represents the direction and speed at which the noisy motion block should move towards the real motion block under the flow time τ. The real velocity target is the difference between the real motion block and the random noise.

[0051] It should be noted that the action occlusion branch forces the model to infer the content of the occluded action based on current and future observations by occluding action blocks; that is, learning "what kind of action leads to the change from the current state to the future state." The model needs to predict the velocity field to make the noisy action block gradually approximate the real action block, thereby establishing an inverse mapping from state change to action.

[0052] The above embodiments enable the model to learn the inverse mapping from state changes to actions by masking action blocks and reconstructing actions based on state changes. This branch forces the model to infer action blocks from the changes in the current observed latent representation to the future observed latent representation, that is, to learn which action can explain the state transition, thereby constraining the observed latent representation to action changes and exhibiting action rooting.

[0053] For example, let the current time be t, and the multi-view observation be... The entity's state is The language instructions are The action block is Future observations Observation encoder Encode image observations into latent representations: ; In the action occlusion branch, the model receives the latent representation of the current observation. Potential characterization for future observation ontological state and language instructions , Indicates noise. Action block. These are considered variables to be reconstructed. To accommodate continuous actions, this application adopts a conditional flow matching approach: ;express Sampling from a standard normal distribution; ;express Sample from a uniform distribution in [0,1]; ; ; in For a noisy action block under the current time τ, For the actual velocity target. Model predicts the velocity field: ;in, The network parameters represent the robot's policy model. This represents the velocity field predicted by the model.

[0054] The training loss is: E represents the mathematical expectation. This represents the square of the L2 distance.

[0055] In some embodiments, the step of performing an occlusion trajectory learning task on the robot policy model to be trained based on the trajectory data to obtain a trained robot policy model includes: In the future observation potential representation occlusion branch, the robot policy model predicts the future observation potential representation based on the current observation potential representation, the action block, the robot body state, and the language instruction. The prediction results are aligned with the L1 normalized future time observation potential representations output by the exponential moving average target encoder, and gradient backpropagation is stopped on the target side output by the exponential moving average target encoder. The L1 alignment loss is used as the loss of the future observation potential representation occlusion branch, and the parameters of the robot policy model are updated according to the loss of the future observation potential representation occlusion branch.

[0056] Layer normalization is used to scale the predicted results and target values ​​to the same scale range, making L1 alignment more stable. Gradient stopping refers to stopping gradient backpropagation on the target side. That is, the target value output by the exponential moving average target encoder is only used as a supervision signal and does not participate in gradient backpropagation to prevent representation collapse.

[0057] It's important to note that the future observation latent representation masking branch forces the model to predict future states based on current observations and action blocks by masking the latent representations of future observations; that is, it learns "how the future state will change after performing the current action." The model needs to predict the latent representations of observations at future time steps and align them with the stable target provided by the exponential moving average target encoder, thereby establishing a positive mapping from action to future state change. Through the gradient stopping operation, the parameters of the target encoder are not affected by the current batch training, keeping the training objective stable and preventing the model from getting trapped in trivial solutions.

[0058] The above embodiments enable the model to learn a positive mapping from action to future state by masking the potential representation of future observations and predicting future states based on current observations and actions. This establishes a positive correlation between the potential representation of observations and actions and physical state transitions, thus enabling action prediction.

[0059] For example, in the future observation latent representation masking branch, the model receives the current observation latent representation. Clean action blocks ontological state and language instructions And predict potential characterizations from future observations: ;in, This represents the prediction network with parameter θ, which is the part of the robot policy model used to predict the potential representation of future observations.

[0060] To obtain stable targets, this application uses the EMA target encoder to obtain potential target representations for future observations: ;in, This represents the exponential moving average target encoder; And it uses layer-normalized L1 alignment loss, and stops gradients on the target side: ; Where LN represents layer normalization, Indicates stopping gradients. This represents the L1 norm (i.e., the mean absolute error).

[0061] The technical advantage of this branch is that the model must determine which future potential factors can be explained by current observations and actions, avoiding learning purely visual reconstruction targets that are unrelated to actions; at the same time, it does not need to predict pixel-level future images, thereby reducing computational costs.

[0062] In some embodiments, the step of performing an occlusion trajectory learning task on the robot policy model to be trained based on the trajectory data to obtain a trained robot policy model further includes: The parameters of the robot policy model are updated using the weighted sum of the loss of the action occlusion branch and the loss of the future observation potential representation occlusion branch as the total loss, thus obtaining the trained robot policy model. The action occlusion branch and the future observation potential representation occlusion branch are sampled together in the same training batch or randomly rotated between different training batches.

[0063] The loss is the action occlusion branch loss. With potential characterization of future observations, the branch loss is masked. The weighted sum, with weight coefficients respectively and This is used to adjust the contribution ratio of the two branches to the total loss.

[0064] The action occlusion branch and the future observation latent representation occlusion branch constrain the model from two directions: inverse mapping (inferring actions from state changes) and forward mapping (predicting future states from actions), respectively. By weighted summing of the losses from the two branches as the overall training objective, the model can simultaneously learn the bidirectional causal relationship between actions and state changes. The sampling strategy for the two branches can employ batch mixing or alternating sampling to ensure the model fully learns both types of tasks during training.

[0065] The above embodiments enable the model to simultaneously acquire the dual capabilities of action interpretation and future prediction through weighted joint optimization of the two branch losses, thereby learning the bidirectional causal relationship between actions and state changes.

[0066] For example, the total loss for Stage-1 is: ; in and represents the weighting coefficients. During training, random sampling can be used to select the action occlusion branch or the potential representation occlusion branch for future observations, or the two types of samples can be mixed in the same batch.

[0067] The technical effects of this stage include: 1. Simultaneously learning both "actions to explain future changes" and "actions to predict future potential states"; 2. Making the observed potential representations more useful for robot control; 3. Avoiding the waste of visual detail modeling caused by pixel reconstruction; 4. Reducing the risk of target drift or representation collapse in self-supervised learning through EMA targets and stopping gradients.

[0068] In some embodiments, the robot policy model to be trained employs a hybrid transformer dual-stream backbone network, which includes an observation stream and an action stream. The observation stream is used to carry the current observation potential representation, the robot's body state, and the future observation potential representation or future potential slots used in the occlusion trajectory learning task. The motion flow is used to carry noisy motions, masked motions, or clean motions; The observation stream and the action stream exchange information through a joint attention mechanism.

[0069] The observation stream is a feature stream in the two-stream backbone network used to process observation-related information. It carries the current observation potential representation, ontology state, and future observation potential representations or future potential slots used in the occlusion trajectory learning task. The action stream is a feature stream in the two-stream backbone network used to process action-related information. It carries noisy actions, occluded actions, or clean actions. The joint attention mechanism refers to the information exchange between the observation stream and the action stream through attention computation, enabling the two streams to pay attention to each other's features. The future potential slot is a reserved learnable latent variable position in the observation stream, used to replace the future observation potential representation in the occlusion branch of the future observation potential representation. It receives information from the action stream and the current observation and outputs a prediction of the future observation potential representation.

[0070] It's important to note that the hybrid transformer dual-stream backbone network enables the model to explicitly distinguish between "what was seen" and "what needs to be done" by placing observation and action information in separate feature streams. The observation stream carries the current observation, ontology state, and future observations or potential future slots, providing state information to the model; the action stream carries action-related information, providing action representations. The two streams interact through a joint attention mechanism, allowing the observation stream to adjust its state representation based on action information, and the action stream to adjust its action representation based on state information, thus achieving structured association modeling between state and action. Linguistic conditional vectors are injected into both streams as conditional information, providing task-level semantic guidance.

[0071] The above embodiments achieve structured modeling of state and action information through two-stream separation and joint attention interaction, thereby improving the model's ability to represent state-action relationships.

[0072] For example, the robot policy model to be trained adopts a Mixture-of-Transformers (MoT) two-stream structure, including an observation stream and an action stream. The observation stream carries the current observation latent representation, ontology state, and future observation latent representations or future potential slots used in the training branch; the action stream carries noisy actions, occluded actions, or clean actions. The two streams interact through joint attention, and linguistic conditions are injected through cross-attention between the streams.

[0073] In some embodiments, the step of training the trained robot policy model by using the model parameters of the trained robot policy model, the observed latent representation at the current moment, the robot's ontological state, and the language instructions to perform flow matching action generation training on the trained robot policy model to obtain a trained robot policy model includes: Based solely on the observed potential representation, robot body state, and language commands at the current moment, random noise is added to the real action block, and the random noise and the action block are interpolated within the range of 0 to 1 in the streaming time to obtain a noisy action block. The trained robot policy model is used to predict the velocity field from the noisy action block to the action block. The mean square error between the predicted value of the velocity field and the actual velocity target is used as the training loss to update the model parameters of the trained robot policy model.

[0074] The noisy action block is the interpolation result of the real action block and random noise over the flow time τ. When τ=0, it is pure noise; when τ=1, it is the real action block. The velocity field represents the direction and speed at which the noisy action block should move towards the real action block under the flow time τ. The real velocity target is the difference between the real action block and the random noise.

[0075] It's important to note that during the Stage-2 training phase, no latent representations of future observations are input. The training is conducted solely based on the current latent representations, the robot's body state, and the spoken commands for flow-matching motion generation training. Since the occlusion trajectory learning in Stage-1 has already enabled the model to learn the action-related future potential transition structure, the model's parameters already internalize the predictive ability for changes from the current state to future states. Therefore, Stage-2 only requires current observations to complete motion generation training, without the need for explicit input of future observations. The model learns the mapping relationship from the current state to the action by predicting the velocity field, gradually approximating the noisy action block to the real action block.

[0076] The above embodiments enable the model to directly generate actions from the current state by training the flow matching action generation based solely on the current observation, thereby reducing the input dimensionality and computational overhead during inference. This application places the main control computation within a compact latent interaction policy backbone, rather than a large language model or pixel-level video generation backbone, thus significantly reducing the number of parameters, memory usage, and inference latency.

[0077] For example, Stage-2 trains the final deployable policy. Unlike Stage-1, Stage-2 does not input latent representations of future observations during inference; it only uses latent representations of current observations. ontological state and language instructions Action generation still uses stream matching: ; ; ; ; The training objective is: ;in For the network parameters of the robot policy model, This represents the velocity field predicted by the model.

[0078] During deployment, starting with random noise action blocks, continuous action blocks are generated by progressively integrating or sampling the learned velocity field and sent to the robot controller. Since Stage-1 has already enabled the model to implicitly learn the potential future transition structure related to the action, Stage-2 can utilize this predictive structure through the hidden state of MoT even without explicitly inputting future observations.

[0079] Figure 2 This is a flowchart illustrating the robot control method provided in this application. Figure 2 As shown, the robot control method includes: Step 201: Obtain multi-view image observations, robot body state, and language commands at the current moment.

[0080] Step 202: Input the multi-view image observation into the visual encoder, extract the multi-view visual latent representations, and stitch or fuse the multi-view visual latent representations to obtain the observation latent representation at the current moment.

[0081] Step 203: Using the observed potential representation at the current moment, the robot's body state, and the language command as conditions, the trained robot policy model, trained using the robot policy model training method described above, iteratively denoises from the random noise action block to generate the target action block; wherein, the action block is a sequence of robot actions for H consecutive time steps, H is a preset positive integer, and the random noise action block is a random noise tensor with the same dimension as the action block.

[0082] Step 204: Send at least one action in the target action block to the robot controller for execution.

[0083] Iterative denoising refers to the process of gradually updating a random noise action block according to the velocity field predicted by the model, starting from the random noise action block, so that it gradually approximates the actual action distribution. An action block is a sequence of robot actions over H consecutive time steps, such as a trajectory starting from the current moment. A random noise action block is a random Gaussian noise tensor with the same dimension as the action block.

[0084] It should be noted that the robot control method is executed during the deployment phase, at which point the policy model has already been trained using the aforementioned training methods. During inference, no latent representations of future observations are required as input, nor is it necessary to predict pixel-level future images; only multi-view image observations of the current moment, the robot's own state, and verbal commands are needed as input. The model starts with random noise action blocks, iteratively denoises them to gradually generate target action blocks, and sends the actions in the target action blocks to the robot controller for execution, achieving closed-loop control.

[0085] The above embodiments generate actions directly from random noise using a pre-trained policy model, without relying on large-scale VLA models or pixel-level predictions. This simplifies the inference process and meets the real-time requirements of robot closed-loop control. Furthermore, deployment only requires observation at the current moment, eliminating the need for future information input, further reducing inference latency and computational overhead.

[0086] For example, the deployment process is simple. Stage-2 and the inference stage only require the current observation, ontology state and language commands, and do not require future images or potential future inputs, making it easy to deploy to a real robot closed-loop control system.

[0087] The training method for the robot policy model provided in this application differs from existing technologies in the following ways, and the corresponding effects are as follows: 1. Compared to large-scale VLA, this application does not place the main control computation in a large language model, but instead concentrates the control computation in a compact two-stream latent interaction backbone, reducing the number of parameters and inference costs. 2. Compared to ordinary behavior cloning, this application introduces Stage-1 occlusion trajectory learning before policy training, ensuring that the observed latent representation must establish a predictable relationship with actions and future changes. 3. Compared to pixel-level world models or video generation models, this application only predicts latent representations and not image pixels, reducing unnecessary background, texture, and lighting modeling. 4. Compared to unidirectional future latent alignment, this application simultaneously includes two complementary branches: action occlusion and future latent representation occlusion, which respectively constrain inverse action interpretation and forward future prediction. 5. Compared to query compression modules trained from scratch, this application preferably retains dense patch observed latent representations, reducing the risk of latent representation collapse and preserving the local spatial information required for manipulation.

[0088] The following describes an exemplary application of the embodiments of this application in a real-world application scenario.

[0089] 1. Data preparation.

[0090] Collect or use robot trajectory data, each trajectory including: 1.1 Multi-view RGB image sequence; 1.2 Robot body state sequence, such as joint position, end effector pose, gripper state, etc.; 1.3 Continuous motion sequences, such as end-effector pose increments, joint control parameters, or gripper opening and closing movements; 1.4 Corresponding language task instructions.

[0091] Training samples can be constructed using a trajectory sliding window. For each time t, the current observation is selected. Future Observation Action blocks = : ontological state and language instructions .

[0092] 2. Observational potential characterization extraction.

[0093] 2.1 Perform size normalization and image preprocessing on each viewpoint image; 2.2 Input multi-view images into the visual encoder; 2.3 Obtain dense patch-level visual latent representations; 2.4 The multi-view patch representations are spliced ​​or fused to obtain the observational latent representation z_t; 2.5 Observations on the Future Potential characterizations for future observations can be obtained using the same encoder or the EMA target encoder.

[0094] In a preferred embodiment, the image size can be set to 224×224, using two viewpoints, and the visual condition mode employs dense patching. This setting preserves local spatial evidence such as the target object's boundaries, contact area, and target location.

[0095] 3. Language and ontology state processing.

[0096] 3.1 Use a text encoder to encode language instructions into language condition vectors; 3.2 Encode the ontology state into a state token using a linear layer or multilayer perceptron; 3.3 In the MoT backbone, linguistic conditions are injected into the observation stream and action stream through cross-attention; 3.4 The ontological state and the potential representation of observation together constitute the observation-side conditions.

[0097] This approach avoids mixing language as a regular joint token with action tokens, allowing language to primarily serve as a task condition, while the main control-related computations are performed by the observation stream and action stream.

[0098] 4. MoT Dual-Stream Interactive Backbone.

[0099] The MoT backbone includes: 4.1 Observation: Carries the current observation potential representation, ontology state, and future observation potential representation or future potential slots used in Stage-1; 4.2 Motion Flow: Carries noisy motions, occluded motions, or clean motions; 4.3 Joint attention: enabling the exchange of information between the observation stream and the action stream; 4.4 Language Cross-Attention: Using language instructions as conditions to modulate two streams; 4.5 Action speed prediction head: Predicts the flow matching speed from the hidden state of the action flow; 4.6 Future Potential Characterization Prediction Head: Predicts future potential characterizations from the observation side or related latent states.

[0100] This dual-stream structure can clearly distinguish between observation-related information and action-related information, while allowing the two to interact through attention, thereby modeling structured state-action relationships.

[0101] 5. Stage-1 Training Steps.

[0102] Perform the following steps for each training batch: 5.1 Sampling window from trajectory data; 5.2 Encoding the current observations ; 5.3 Encoding Future Observations or EMA target ; 5.4 Randomly select training branches: If you choose the motion occlusion branch, then use , , and As a condition, noise is added to the action block, and the model is trained to predict the flow matching speed. If the potential representation masking branch for future observations is selected, then use , , and As a condition, predict potential future observational characteristics and align them with the EMA target using L1 grading. 4.5 Update the model parameters of the trained robot policy model according to L_stage1; 4.6 Update the target encoder parameters using an exponential moving average method.

[0103] 6. Stage-2 training steps.

[0104] 6.1 Use the model parameters after Stage-1 as the initialization or continuation of the training state; 6.2 For each sample, input only the current observation potential representation. ontological state and language instructions ; 6.3 For real action blocks Perform stream matching and noise addition; 6.4 The trained model predicts the velocity field from noise to real motion; 6.5 Obtain the final deployable strategy.

[0105] 7. Deployment steps.

[0106] When the robot performs the task: 7.1 Receive current multi-view images, ontology status, and language commands; 7.2 Encoding the current observations ; 7.3 Initialize action blocks from random noise; 7.4 Generate action blocks through iterative sampling using stream matching; 7.5 Send one or more of the preceding actions of the action block to the robot controller; 7.6 Re-observe and roll over in the next control cycle.

[0107] The deployment phase does not require future observation inputs or pixel-level future predictions, thus the inference process is compact.

[0108] 8. A specific embodiment.

[0109] In one embodiment: 8.1 The visual encoder uses a ViT-B / 14 level visual encoder, with an input image size of 224; 8.2 Use two camera perspectives; 8.3 The observation conditions employed dense patch visual latent representation; 8.4 The strategy backbone adopts a DiT-B scale MoT dual-stream Transformer; 8.5 The action block length is 8; 8.6 Both language conditions and ontology state conditions are enabled; 8.7 Stage-1 includes a motion occlusion flow matching branch and a future observation latent representation L1 alignment branch; 8.8 Stage-2 uses stream matching to train the final action policy; 8.9 The total number of parameters in the model is approximately 0.47B.

[0110] In simulation experiments, this embodiment achieved an overall success rate of approximately 98.0% on the original LIBERO benchmark; an overall success rate of approximately 76.13% on LIBERO-plus without using LIBERO-plus training data; and an average sequence length of approximately 4.61 on CALVIN ABC->D.

[0111] The training apparatus for the robot strategy model provided in this application is described below. The training apparatus for the robot strategy model described below can be referred to in correspondence with the training method for the robot strategy model described above.

[0112] Figure 3 This is a schematic diagram of the training structure of the robot policy model provided in this application. For example... Figure 1 As shown, the training device for the robot policy model includes: The data acquisition module 310 is used to acquire robot trajectory data, which includes at least current observation data, future observation data, action blocks, robot body state, and language commands. The first training module 320 is used to perform an occlusion trajectory learning task on the robot policy model to be trained based on the trajectory data, so as to obtain a trained robot policy model. The occlusion trajectory learning task includes an action occlusion branch and a future observation potential representation occlusion branch. The action occlusion branch is used to reconstruct the occluded action block based on the current observation potential representation, the future observation potential representation, the robot body state, and the language instruction. The future observation potential representation occlusion branch is used to predict the future observation potential representation based on the current observation potential representation and the action block. The second training module 330 is used to perform flow matching action generation training on the trained robot strategy model based on the model parameters of the trained robot strategy model, using the observed potential representation, robot body state and language instructions at the current moment, to obtain a trained robot strategy model.

[0113] In some embodiments, the current observation data and the future observation data are multi-view images at time t and time t+H respectively in the multi-view image observation sequence, where H is a preset positive integer; the action block consists of actions of H consecutive time steps in the action sequence; the robot body state is the state at time t in the robot body state sequence; and the language instruction is a task description corresponding to the entire trajectory.

[0114] In some embodiments, the data acquisition module 310 includes: The first processing unit is used to input multi-view image observations into a visual encoder, extract visual latent representations, and stitch or fuse the visual latent representations to obtain the observation latent representations at the current moment. The second processing unit is used to input multi-view image observations at future times into the exponential moving average target encoder to obtain the potential representation of the observations at future times. The parameters of the exponential moving average target encoder are updated with training through exponential moving average. The third processing unit is used to encode the robot's body state into a state token through a linear layer or multilayer perceptron. The fourth processing unit is used to input language instructions into the text encoder and encode them into language condition vectors; The fifth processing unit is used to construct the observation-side conditions together with the observation potential representation and the state token, and the language condition vector serves as condition information.

[0115] In some embodiments, the first training module 320 includes: A noise-adding interpolation unit is used to add random noise to the action block in the action masking branch, and interpolate the random noise and the action block in the range of 0 to 1 in the flow time to obtain a noise-adding action block. The velocity field prediction unit is used to predict the velocity field from the noisy action block to the real action block by means of the robot policy model, based on the current potential representation of observation, the future potential representation of observation, the robot body state, and the language command. The first update unit is used to update the parameters of the robot policy model based on the mean square error between the predicted value of the velocity field and the actual velocity target, as the loss of the action occlusion branch.

[0116] In some embodiments, the first training module 320 further includes: The representation prediction unit is used to predict the potential representation of observation at future moments in the future observation potential representation occlusion branch, based on the current potential representation of observation, the action block, the robot body state, and the language instruction, through the robot policy model. The characterization alignment unit is used to align the prediction result with the L1 normalized future time observation potential characterization output by the exponential moving average target encoder, and to stop gradient backpropagation on the target side output by the exponential moving average target encoder. The second update unit is used to update the parameters of the robot policy model based on the loss of the future observation potential representation occlusion branch, using the L1 alignment loss as the loss of the future observation potential representation occlusion branch.

[0117] In some embodiments, the step of performing an occlusion trajectory learning task on the robot policy model to be trained based on the trajectory data to obtain a trained robot policy model further includes: The third update unit is used to update the parameters of the robot policy model with the weighted sum of the loss of the action occlusion branch and the loss of the future observation potential representation occlusion branch as the total loss, so as to obtain the trained robot policy model. The action occlusion branch and the future observation potential representation occlusion branch are sampled together in the same training batch or randomly rotated between different training batches.

[0118] In some embodiments, the robot policy model to be trained employs a hybrid transformer dual-stream backbone network, which includes an observation stream and an action stream. The observation stream is used to carry the current observation potential representation, the robot's body state, and the future observation potential representation or future potential slots used in the occlusion trajectory learning task. The motion flow is used to carry noisy motions, masked motions, or clean motions; The observation stream and the action stream exchange information through a joint attention mechanism.

[0119] In some embodiments, the second training module 330 includes: The noise interpolation unit is used to add random noise to the real motion block based solely on the observed potential representation, robot body state, and language instructions at the current moment, and to interpolate the random noise and the motion block within the range of 0 to 1 in the streaming time to obtain the noisy motion block. A velocity field prediction unit is used to predict the velocity field from the noisy action block to the action block using the trained robot policy model. The fourth update unit is used to update the model parameters of the trained robot policy model using the mean square error between the predicted value of the velocity field and the actual velocity target as the training loss.

[0120] This application also provides a robot control device, including: The first acquisition module is used to acquire multi-view image observations, robot body state, and language commands at the current moment; The second acquisition module is used to input the multi-view image observation into the visual encoder, extract the multi-view visual latent representations, and stitch or fuse the multi-view visual latent representations to obtain the observation latent representation at the current moment. The action block generation module is used to generate target action blocks by iteratively denoising the trained robot policy model obtained by the aforementioned robot policy model training method, based on the observed potential representation at the current moment, the robot body state, and the language instructions. An action sending module is used to send at least one action in the target action block to the robot controller for execution. Wherein, the action block is a sequence of robot actions for H consecutive time steps, where H is a preset positive integer, and the random noise action block is a random noise tensor with the same dimension as the action block.

[0121] Figure 4 This is a schematic diagram of the physical structure of the electronic device provided in this application. For example... Figure 4As shown, the electronic device may include: a processor 410, a communications interface 420, a memory 430, and a communication bus 440, wherein the processor 410, the communications interface 420, and the memory 430 communicate with each other via the communication bus 440. The processor 410 can call logical instructions in the memory 430 to execute a training method for a robot strategy model, the method including: Acquire robot trajectory data, which includes at least current observation data, future observation data, action blocks, robot body state, and language commands; Based on the trajectory data, the robot policy model to be trained is subjected to an occlusion trajectory learning task to obtain a trained robot policy model. The occlusion trajectory learning task includes an action occlusion branch and a future observation potential representation occlusion branch. The action occlusion branch is used to reconstruct the occluded action block based on the current observation potential representation, the future observation potential representation, the robot body state, and the language instruction. The future observation potential representation occlusion branch is used to predict the future observation potential representation based on the current observation potential representation and the action block. Based on the model parameters of the trained robot policy model, the robot policy model is trained by flow matching action generation using the observed latent representation, robot body state and language instructions at the current moment, to obtain a trained robot policy model.

[0122] Furthermore, the logical instructions in the aforementioned memory 430 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0123] On the other hand, this application also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer is able to execute the training method for the robot strategy model provided by the above methods, the method including: Acquire robot trajectory data, which includes at least current observation data, future observation data, action blocks, robot body state, and language commands; Based on the trajectory data, the robot policy model to be trained is subjected to an occlusion trajectory learning task to obtain a trained robot policy model. The occlusion trajectory learning task includes an action occlusion branch and a future observation potential representation occlusion branch. The action occlusion branch is used to reconstruct the occluded action block based on the current observation potential representation, the future observation potential representation, the robot body state, and the language instruction. The future observation potential representation occlusion branch is used to predict the future observation potential representation based on the current observation potential representation and the action block. Based on the model parameters of the trained robot policy model, the robot policy model is trained by flow matching action generation using the observed latent representation, robot body state and language instructions at the current moment, to obtain a trained robot policy model.

[0124] Furthermore, this application also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements a training method for a robot policy model provided by the methods described above, the method comprising: Acquire robot trajectory data, which includes at least current observation data, future observation data, action blocks, robot body state, and language commands; Based on the trajectory data, the robot policy model to be trained is subjected to an occlusion trajectory learning task to obtain a trained robot policy model. The occlusion trajectory learning task includes an action occlusion branch and a future observation potential representation occlusion branch. The action occlusion branch is used to reconstruct the occluded action block based on the current observation potential representation, the future observation potential representation, the robot body state, and the language instruction. The future observation potential representation occlusion branch is used to predict the future observation potential representation based on the current observation potential representation and the action block. Based on the model parameters of the trained robot policy model, the robot policy model is trained by flow matching action generation using the observed latent representation, robot body state and language instructions at the current moment, to obtain a trained robot policy model.

[0125] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0126] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0127] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for training a robot policy model, characterized in that, include: Acquire robot trajectory data, which includes at least current observation data, future observation data, action blocks, robot body state, and language commands; Based on the trajectory data, the robot policy model to be trained is subjected to an occlusion trajectory learning task to obtain a trained robot policy model. The occlusion trajectory learning task includes an action occlusion branch and a future observation potential representation occlusion branch. The action occlusion branch is used to reconstruct the occluded action block based on the current observation potential representation, the future observation potential representation, the robot body state, and the language instruction. The future observation potential representation occlusion branch is used to predict the future observation potential representation based on the current observation potential representation and the action block. Based on the model parameters of the trained robot policy model, the robot policy model is trained by flow matching action generation using the observed latent representation, robot body state and language instructions at the current moment, to obtain a trained robot policy model.

2. The method according to claim 1, characterized in that, The current observation data and the future observation data are the multi-view images at time t and time t+H in the multi-view image observation sequence, respectively, where H is a preset positive integer; the action block consists of actions of H consecutive time steps in the action sequence; the robot body state is the state at time t in the robot body state sequence; the language instruction is the task description corresponding to the entire trajectory.

3. The method according to claim 1 or 2, characterized in that, The acquisition of robot trajectory data includes: Multi-view image observations are input into a visual encoder to extract visual latent representations, and the visual latent representations are spliced ​​or fused to obtain the observation latent representation at the current moment. The multi-view image observations of future time moments are input into the exponential moving average target encoder to obtain the potential representation of the observations of future time moments. The parameters of the exponential moving average target encoder are updated with training through exponential moving average. Encode the robot's state into a state token using a linear or multilayer perceptron. The language instructions are input into the text encoder and encoded into language condition vectors; The observation potential representation and the state token together constitute the observation-side conditions, and the language condition vector serves as condition information.

4. The method according to any one of claims 1 to 3, characterized in that, The process of performing an occlusion trajectory learning task on the robot policy model to be trained based on the trajectory data to obtain a trained robot policy model includes: In the motion masking branch, random noise is added to the motion block, and the random noise and the motion block are interpolated within the range of 0 to 1 in the stream time to obtain a noisy motion block; Using the current potential representation of observation, the future potential representation of observation, the robot's body state, and the language command as conditions, the robot policy model predicts the velocity field from the noisy action block to the real action block. The mean square error between the predicted value of the velocity field and the actual velocity target is used as the loss of the action occlusion branch, and the parameters of the robot policy model are updated based on the loss of the action occlusion branch.

5. The method according to claim 1 or 4, characterized in that, The process of performing an occlusion trajectory learning task on the robot policy model to be trained based on the trajectory data to obtain a trained robot policy model includes: In the future observation potential representation occlusion branch, the robot policy model predicts the future observation potential representation based on the current observation potential representation, the action block, the robot body state, and the language instruction. The prediction results are aligned with the L1 normalized future time observation potential representations output by the exponential moving average target encoder, and gradient backpropagation is stopped on the target side output by the exponential moving average target encoder. The L1 alignment loss is used as the loss of the future observation potential representation occlusion branch, and the parameters of the robot policy model are updated according to the loss of the future observation potential representation occlusion branch.

6. The method according to any one of claims 1, 4, and 5, characterized in that, The step of performing an occlusion trajectory learning task on the robot policy model to be trained based on the trajectory data to obtain the trained robot policy model further includes: The parameters of the robot policy model are updated using the weighted sum of the loss of the action occlusion branch and the loss of the future observation potential representation occlusion branch as the total loss, thus obtaining the trained robot policy model. The action occlusion branch and the future observation potential representation occlusion branch are sampled together in the same training batch or randomly rotated between different training batches.

7. The method according to any one of claims 1 to 6, characterized in that, The robot policy model to be trained adopts a hybrid converter dual-stream backbone network, which includes an observation stream and an action stream. The observation stream is used to carry the current observation potential representation, the robot's body state, and the future observation potential representation or future potential slots used in the occlusion trajectory learning task. The motion flow is used to carry noisy motions, masked motions, or clean motions; The observation stream and the action stream exchange information through a joint attention mechanism.

8. The method according to any one of claims 1 to 7, characterized in that, The model parameters based on the trained robot policy model are used to train the trained robot policy model through flow matching action generation, utilizing the observed latent representation, robot body state, and language commands at the current moment, to obtain a trained robot policy model, including: Based solely on the observed potential representation, robot body state, and language commands at the current moment, random noise is added to the real action block, and the random noise and the action block are interpolated within the range of 0 to 1 in the streaming time to obtain a noisy action block. The trained robot policy model is used to predict the velocity field from the noisy action block to the action block. The mean square error between the predicted value of the velocity field and the actual velocity target is used as the training loss to update the model parameters of the trained robot policy model.

9. A robot control method, characterized in that, include: Acquire multi-view image observations, robot body state, and language commands at the current moment; The multi-view image observations are input into a visual encoder to extract the multi-view visual latent representations, and the multi-view visual latent representations are spliced ​​or fused to obtain the observation latent representation at the current moment. Using the observed potential representation at the current moment, the robot's body state, and the language instructions as conditions, the trained robot policy model, trained by the training method of any one of claims 1 to 8, iteratively denoises from random noise action blocks to generate target action blocks. Send at least one action from the target action block to the robot controller for execution; Wherein, the action block is a sequence of robot actions for H consecutive time steps, where H is a preset positive integer, and the random noise action block is a random noise tensor with the same dimension as the action block.

10. A training device for a robot strategy model, characterized in that, include: The data acquisition module is used to acquire robot trajectory data, which includes at least current observation data, future observation data, action blocks, robot body state, and language commands. The first training module is used to perform an occlusion trajectory learning task on the robot policy model to be trained based on the trajectory data, so as to obtain a trained robot policy model. The occlusion trajectory learning task includes an action occlusion branch and a future observation potential representation occlusion branch. The action occlusion branch is used to reconstruct the occluded action block based on the current observation potential representation, the future observation potential representation, the robot body state, and the language instruction. The future observation potential representation occlusion branch is used to predict the future observation potential representation based on the current observation potential representation and the action block. The second training module is used to perform flow matching action generation training on the trained robot policy model based on the model parameters of the trained robot policy model, using the observed potential representation, robot body state and language instructions at the current moment, to obtain a trained robot policy model.

11. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the training method for the robot strategy model as described in any one of claims 1 to 8.

12. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the training method for the robot strategy model as described in any one of claims 1 to 8.

13. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the training method for the robot strategy model as described in any one of claims 1 to 8.

Images