Robot operation strategy model training method and robot operation strategy model

Abstract

The application provides a robot operation strategy model training method and a robot operation strategy model, and relates to the field of artificial intelligence and robots. The method comprises: processing a video sample by using a feature extraction network to obtain a video representation; training a video generation branch according to the video representation to predict future visual information; and training an action generation branch according to the video representation to predict an action sequence; wherein the model parameters of the video generation branch and the action generation branch are decoupled. The above scheme solves the problem of lack of stable intermediate interface and inconsistent representation space between the two modules, while avoiding mutual interference caused by task target differences, and improves the learning efficiency and generalization ability of each module.

Description

Technical Field

[0001] This application relates to the fields of artificial intelligence and robotics, specifically to a training method for a robot operation strategy model and a robot operation strategy model. Background Technology

[0002] Existing robot operation strategy models typically separate or simply integrate environment understanding and action generation. The lack of a stable intermediate interface between the video generation module and the action generation module leads to inconsistent information transmission and representation spaces. Furthermore, the two modules have different training objectives; joint optimization within the same model can easily result in mutual interference in parameter updates, affecting their respective learning outcomes. Summary of the Invention

[0003] In view of this, embodiments of this application provide a method for training a robot operation strategy model and a robot operation strategy model.

[0004] In a first aspect, one embodiment of this application provides a method for training a robot operation strategy model, the robot operation strategy model including a video generation branch and an action generation branch, the method including: processing video samples using a feature extraction network to obtain video representations; training the video generation branch based on the video representations to predict future visual information; and training the action generation branch based on the video representations to predict action sequences; wherein the model parameters of the video generation branch and the action generation branch are decoupled.

[0005] In conjunction with the first aspect, in some implementations of the first aspect, the training modes of the video generation branch and the action generation branch include a decoupled training mode and a joint training mode; wherein, during decoupled training, the model parameters of the video generation branch and the action generation branch are updated independently; during joint training, the input data of the action generation branch includes the future video representation output by the video generation branch, and the model parameters of the video generation branch and the action generation branch are updated using a gradient truncation method.

[0006] In conjunction with the first aspect, in some implementations of the first aspect, the video representation is divided into historical video representation and future video representation; based on the video representation, the video generation branch is trained, including: constructing input data, which includes task text, historical video representation, and future video representation; sampling two time variables of the improved mean stream to construct a training time difference; and learning the velocity field of the video generation branch based on the input data under the training time difference.

[0007] In conjunction with the first aspect, in some implementations of the first aspect, the future video representation includes a future video representation after noise injection; wherein, a conditional mask is used to distinguish the respective regions of the historical video representation and the future video representation after noise injection.

[0008] In conjunction with the first aspect, in some implementations of the first aspect, under the training time difference, the velocity field of the video generation branch is learned based on the input data, including: obtaining the velocity field prediction value and the auxiliary directional derivative prediction value of the output of the video generation branch based on the input data and the training time difference; calculating the first loss between the velocity field prediction value and the corresponding true velocity field value, and calculating the second loss between the auxiliary directional derivative prediction value and the corresponding true directional derivative value; and updating the model parameters of the video generation branch based on the first loss and the second loss.

[0009] In conjunction with the first aspect, in some implementations of the first aspect, under the decoupled training mode of the action generation branch, the action generation branch is trained based on the video representation, including: enhancing the video representation to obtain the target visual conditions; acquiring the real action sequence aligned with the video sample time; and optimizing the action generation branch using an action flow matching method based on the target visual conditions, the real action sequence, and the robot state; wherein the real action sequence includes the real action sequence after injecting noise.

[0010] In conjunction with the first aspect, in some implementations of the first aspect, the video representation is enhanced to obtain the target visual conditions, including at least one of the following: retaining a portion of the time steps in the video representation with a preset probability to achieve sparsity processing; injecting noise into the video representation with a preset probability; and discarding the observation frames in the video representation corresponding to the current time.

[0011] Secondly, one embodiment of this application provides a robot operation strategy model, including: a feature extraction network configured to process observed videos to obtain a unified video representation; a video generation branch configured to predict future visual information based on the video representation corresponding to the observed video; and an action generation branch configured to predict action sequences based on the video representation corresponding to the observed video; wherein the model parameters of the video generation branch and the action generation branch are independent of each other.

[0012] Thirdly, one embodiment of this application provides a computer-readable storage medium storing a computer program for performing the training method described in the first aspect.

[0013] Fourthly, one embodiment of this application provides an electronic device, the electronic device comprising: a processor; a memory for storing processor-executable instructions; the processor being configured to execute the training method described in the first aspect.

[0014] Fifthly, one embodiment of this application provides a computer program product including instructions that, when executed on an electronic device, cause the electronic device to implement the training method described in the first aspect.

[0015] The above scheme establishes a unified visual input interface for the video generation branch and the action generation branch by sharing the video representation output of the feature extraction network, thus solving the problems of lack of a stable intermediate interface and inconsistent representation space between the two modules. Simultaneously, by decoupling the model parameters of the two branches, they can independently optimize their respective objectives during training: the video generation branch focuses on predicting future visual information, and the action generation branch focuses on predicting action sequences. This avoids mutual interference caused by differences in task objectives, thereby improving their respective learning efficiency and generalization ability. Attached Figure Description

[0016] The above and other objects, features, and advantages of this application will become more apparent from the more detailed description of the embodiments of this application in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of this application and form part of the specification. They are used together with the embodiments of this application to explain this application and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps.

[0017] Figure 1 The diagram shown is a flowchart illustrating a training method for a robot operation strategy model provided in an embodiment of this application.

[0018] Figure 2 The diagram shown is a training flowchart of the video generation branch provided in an embodiment of this application.

[0019] Figure 3 The diagram shown is a training flowchart for an action generation branch provided in an embodiment of this application.

[0020] Figure 4 The diagram shown is a two-stage cascaded reasoning flowchart of a robot operation strategy model provided in an embodiment of this application.

[0021] Figure 5 The diagram shown is a structural schematic of a robot operation strategy model provided in an embodiment of this application.

[0022] Figure 6 The diagram shown is a structural schematic of an electronic device provided in an embodiment of this application. Detailed Implementation

[0023] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0024] In the technical field of robot operation strategy models, in order to achieve the general goal of generating control actions based on visual observations, related technologies typically adopt two approaches.

[0025] One approach directly regresses the action sequence based on current image observations and task instructions using an end-to-end neural network model. Its basic principle is to fuse the features encoded in the image and text and directly map them into continuous values ​​in the action space. Another approach introduces video generation or world models, attempting to predict future visual changes in the environment first, and then plan actions based on the predicted future visual information.

[0026] However, when the above-mentioned approach is applied to complex robot operation scenarios that require both high-precision future visual prediction and robust action generation, its performance is less than ideal. A fundamental contradiction lies in the fact that, in order to optimize the realism and long-term consistency of video generation, the model design and training objectives inevitably compromise the effective utilization of visual conditions and the stability of training in the action generation module, and may even lead to policy failure. Specifically, when training a single model that combines visual prediction and action generation functions, the loss from video pixel-level reconstruction and the loss from action regression compete with each other at the gradient level, making it difficult for the model to converge to a state where both are optimal. Furthermore, if the video generation module and the action generation module are trained as two independent systems, the mismatch in their representation spaces makes it difficult for the generated high-dimensional video features to be directly and efficiently utilized by the action generation module, resulting in a decrease in the robustness of the action policy.

[0027] In-depth analysis reveals that the root causes of the aforementioned contradictions are multifaceted. From the model architecture perspective, the lack of a unified and compact intermediate representation interface between the visual generation and action generation modules creates barriers to information transfer. From the training process perspective, the optimization objectives of video generation and action regression are inherently different, leading to conflicting optimization directions during joint training within a model sharing parameters. Furthermore, from a data perspective, real-world robot observations are often accompanied by viewpoint occlusion, sensor noise, and uncertainties in action annotation. If the model training process is not specifically designed to address these issues, its generalization ability and robustness in real-world scenarios will be significantly reduced.

[0028] To overcome the aforementioned contradictions, this application proposes a training method for a robot operation strategy model. By introducing a shared video encoder to provide a unified input representation for both the video generation and motion generation branches, and employing a dual-branch decoupled collaborative training method, the collaborative failure problem caused by conflicting training objectives and mismatched representations between the video generation and motion generation modules is resolved. This provides a foundation for efficient and robust integrated decision-making in complex scenarios.

[0029] Before detailing the specific implementation of the training method of this application, a typical implementation environment of this application is first described. This implementation environment involves a training and application system for a robot operation strategy model, which mainly includes a robot platform, a sensing unit, a computing unit, and a model storage unit. The robot platform is equipped with a robotic arm, gripper, or other end effector for performing grasping, placing, pushing, and pulling operations in the physical environment. The sensing unit includes at least multiple vision sensors (such as RGB cameras) arranged in different positions to collect interactive videos of the robot and the environment from different perspectives, forming a multi-view observation video stream. The sensing unit may also include sensors for acquiring state information such as robot joint angles and end-effector pose. The computing unit is typically a high-performance server or embedded computing device containing a graphics processing unit (GPU), which loads and runs the robot operation strategy model and its training algorithm of this application. Specifically, the computing unit receives raw observation data (video, state) from the sensing unit and task text instructions from the user or upper-level system. After model processing, it outputs a sequence of actions in the robot joint space or operation space and sends it to the robot platform's control system for execution. The model storage unit is used to store the trained model parameters. In this environment, video samples are derived from multi-view videos recorded when the robot performs various demonstration tasks, synchronized action sequence annotations, and corresponding task text descriptions.

[0030] It should be noted that the above implementation environment is only an example. This application is also applicable to model training in a simulation environment, where the sensing data can be generated by the simulation engine.

[0031] Below, in conjunction with Figures 1 to 4 This paper elaborates on the specific implementation of the training method in this application.

[0032] Figure 1 The diagram shown is a flowchart illustrating a training method for a robot operation strategy model according to an embodiment of this application. Figure 1 As shown, the method includes the following steps.

[0033] Step S110: Use a feature extraction network to process video samples to obtain video representations.

[0034] Optionally, a video sample refers to any data containing a sequence of temporal image frames that can be used to characterize changes in the environmental state observed by the robot. Specifically, it can refer to a raw video stream captured by a real robot's camera, an image sequence rendered from a simulation environment, or an image tensor that has undergone preprocessing (such as cropping, scaling, and normalization).

[0035] Video representation refers to the low-dimensional, dense feature representation obtained by compressing and abstracting video samples through a feature extraction network. Examples include, but are not limited to, latent vector sequences output by the encoder part of an autoencoder, feature tensors extracted by a spatiotemporal convolutional neural network, or token sequences extracted by a visual transformer model.

[0036] It should be noted that the video representation can be divided into two parts based on time points. One part is the historical video representation corresponding to the historical observation period, and the other part is the future video representation corresponding to the future time period to be predicted. Both can be obtained from the labeled data during the training phase. During the inference phase, the future video representation needs to be generated by the video generation branch prediction.

[0037] In some implementations, the feature extraction network encodes each frame of the video sample, extracting feature vectors or feature maps containing spatial structure and temporal context information. It then concatenates or aligns features from multiple viewpoints and time steps to form a unified video representation. For example, given a video tensor of shape [time step T, number of channels C, height H, width W], the feature extraction network outputs a video representation of shape [time step T, feature dimension]. This video representation serves as the shared input for subsequent video generation and action generation branches.

[0038] Taking the robotic arm grasping task as an example, the video sample contains 5 frames of images captured by the robotic arm's end-effector camera in the last 5 time steps. The feature extraction network encodes these images into 512-dimensional feature vectors and splices them in chronological order to form a 5×512 video representation for use by subsequent branches.

[0039] Step S120: Based on the video representation, train the video generation branch to predict future visual information.

[0040] Optionally, the video generation branch refers to any neural network component or module capable of predicting future visual information based on current visual observations. For example, it includes, but is not limited to, network structures based on diffusion models, flow models, or autoregressive models, which receive video representations generated by a video encoder as input and output predictions of future video frames or video representations.

[0041] In some implementations, this branch is trained as a conditional generative model. Specifically, the video representation sequence is divided into historical and future parts. Historical video representations serve as conditional inputs, while future, real-world video representations serve as the target for this branch to learn and generate.

[0042] For example, during training, historical video representations are input into the Conditional Variational Autoencoder (CVAE) to obtain a latent variable distribution. Simultaneously, realistic future video representations are also input into the encoder. The decoder then attempts to reconstruct future video representations based on historical conditions and latent variables sampled from this distribution. By optimizing the reconstruction loss (e.g., mean squared error) and the regularization loss of the latent distribution (e.g., Kullback-Leibler divergence (KL)), the model parameters of the video generation branch are updated, enabling it to learn to generate reasonable future visual information based on historical conditions.

[0043] Step S130: Based on the video representation, train the action generation branch to predict the action sequence.

[0044] Optionally, the action generation branch refers to any neural network component or module capable of predicting a sequence of executable actions based on visual observations and / or state information. Examples include, but are not limited to, network structures based on flow matching models, policy networks, or sequence prediction models, which receive video representations generated by a video encoder and / or robot states as conditions and output a sequence of actions.

[0045] In some implementations, supervised learning methods such as behavior cloning can be used to train this branch. Specifically, for a training dataset containing historical observations and corresponding real action sequences, a feature extraction network first extracts historical video representations. Then, these historical video representations, along with the current robot state (such as joint angles and end-effector pose), are input into the action generation branch. The action generation branch outputs a predicted sequence of future actions. By calculating the difference between the predicted and real action sequences (such as mean squared error loss) and minimizing this loss, the model parameters of the action generation branch are updated, enabling it to learn the mapping relationship between visual and state observations and actions.

[0046] Furthermore, in this application, the model parameters of the video generation branch and the action generation branch are decoupled. This means that at the model architecture level, the video generation branch and the action generation branch each have independent and non-shared sets of model parameters. Specifically, the parameters of the video generation branch are only used to optimize the prediction task of future visual information, and the parameters of the action generation branch are only used to optimize the prediction task of action sequences. The two branches do not affect each other during training and inference. This design allows the two branches to learn and update their parameters independently for their respective task objectives, avoiding gradient interference or feature competition caused by differences in task objectives, thereby improving their respective learning efficiency and generalization ability. At the same time, parameter decoupling also provides a structural foundation for the flexible adoption of joint training strategies, ensuring that the independence between branches is maintained even when collaborative optimization is required.

[0047] The above scheme establishes a unified visual input interface for the video generation branch and the action generation branch by sharing the video representation output of the feature extraction network, thus solving the problems of lack of a stable intermediate interface and inconsistent representation space between the two modules. Simultaneously, by decoupling the model parameters of the two branches, they can independently optimize their respective objectives during training: the video generation branch focuses on predicting future visual information, and the action generation branch focuses on predicting action sequences. This avoids mutual interference caused by differences in task objectives, thereby improving their respective learning efficiency and generalization ability.

[0048] The aforementioned embodiments enable the video generation and action generation branches to be trained independently and efficiently through parameter decoupling. However, when the model is actually deployed for end-to-end inference, the action generation branch needs to utilize the prediction results produced by the video generation branch. To further improve the task consistency and cooperation between the two branches when they work together in practice, while maintaining the advantages of independent branch training, this application also provides an embodiment. Specifically, in this embodiment, the training modes for the video generation branch and the action generation branch include a decoupled training mode and a joint training mode.

[0049] During decoupled training, the model parameters of the video generation branch and the action generation branch are updated independently. That is, the two branches use their own independent loss functions and optimizers, and their parameter updates do not directly interact with each other's gradients during training. The goal of this stage is to enable the video generation branch and the action generation branch to achieve their respective optimal performance on the visual prediction and action regression tasks.

[0050] During joint training, the input data for the action generation branch includes the future video representation output by the video generation branch. This means that the real data flow during inference is simulated during the joint training phase. Specifically, for a training sample, the video sample is first processed using a feature extraction network to obtain a historical video representation. Then, this historical video representation is input into the pre-trained video generation branch, which generates a predicted future video representation. Next, instead of using the real future video representation provided in the dataset, this future video representation output by the video generation branch, along with the historical video representation, is used as the visual conditional input to the action generation branch. Based on this combined condition and the robot's state, the action generation branch predicts an action sequence and calculates the loss by comparing it with the real action sequence.

[0051] Furthermore, during joint training, gradient truncation is used to update model parameters between the video generation branch and the action generation branch. To achieve this, during computation graph construction, when the output of the video generation branch is passed to the action generation branch as input, gradient truncation is performed at that data node. Thus, when backpropagating to calculate the loss gradient of the action generation branch, this gradient will normally update the parameters of the action generation branch itself, as well as the connected parts in the feature extraction network. However, when the gradient attempts to continue backpropagating through the output node of the video generation branch to the network layers inside the video generation branch, the gradient flow is truncated. Therefore, the model parameters of the video generation branch are not directly updated due to the loss of the action generation branch. Ultimately, the video generation branch can either keep its parameters frozen during joint training or continue to be updated through its own independent generation loss.

[0052] The above scheme, by introducing a joint training mode, enables the action generation branch to learn how to adapt to and utilize the predicted output of the video generation branch, thereby improving the consistency between action planning and visual prediction when the model performs tasks end-to-end. Simultaneously, the gradient truncation mechanism ensures that the visual quality-oriented generation capabilities already learned by the video generation branch during joint training are not shifted due to the needs of the action branch, thus protecting the core performance of each branch.

[0053] To enable the video generation branch to predict future visual information more efficiently, especially supporting single-step or few-step generation during the inference stage, this application also provides a training embodiment for the video generation branch. Specifically, input data is constructed; two time variables of the improved mean stream are sampled to construct a training time difference; under the training time difference, the velocity field of the video generation branch is learned based on the input data.

[0054] Optionally, the input data includes task text, historical video representations, and future video representations. The task text refers to the natural language instructions describing the robot's operational tasks, which are converted into text semantic embedding vectors by a text encoder. Historical video representations are obtained by encoding observed historical video frames through a feature extraction network, serving as known conditions. Future video representations are obtained by encoding future real-world video frames to be predicted through the feature extraction network, serving as the target for the supervised model generation during the training phase.

[0055] In some embodiments, the future video representation includes a future video representation with injected noise.

[0056] Correspondingly, the improved mean flow refers to a training method that achieves data distribution modeling and generation by directly supervising the learning of the velocity field of noisy data. Here, the velocity field refers to the vector field in the video generation branch that describes the direction and rate of change of the state of data points as they evolve from a noisy distribution to the target data distribution over time. For example, it includes, but is not limited to, the predictions made by the video generation branch based on input conditions and time variables regarding the denoising direction and rate of change of the noisy future video representation.

[0057] Specifically, under this method, time is defined. ,in, Corresponding clean data (future video representation). This corresponds to pure Gaussian noise. During training, instead of randomly sampling a single time step as in traditional diffusion models, it starts from... Two time variables are sampled independently within the interval, denoted as... and And satisfy The difference between these two time points This constitutes the training time difference. The sampling strategy can be designed to favor smaller Δt to emphasize the learning of local dynamics.

[0058] Furthermore, based on the sampling time A clean future video representation Injecting noise yields a noisy characterization. For example, through the formula Calculation. Among them, It is standard Gaussian noise. and It depends on time. The coefficients. Then, the noisy characterization. Historical video representation Text embedding and time variables and The video is input together with the branch that generates the predicted velocity field. The training objective is to make the velocity field predicted by the video generation branch as close as possible to the true velocity field. This true velocity field can be derived from the data based on the selected probability flow path and is typically related to the noise prediction score function.

[0059] The aforementioned scheme divides video representation into historical video representation and future video representation, and constructs input data containing task text, historical video representation, and future video representation. This enables the video generation branch to simultaneously perceive historical observations and task instructions, improving the semantic accuracy of future visual information prediction. Simultaneously, by sampling and improving the two time variables of the mean flow to construct a training time difference, and learning the velocity field of the video generation branch under this time difference, the model can directly learn the velocity field on the probability flow path instead of traditional noise prediction. This supports efficient single-step or few-step generation during the inference stage, significantly reducing the number of sampling steps required to generate future videos, thus balancing generation quality and inference efficiency.

[0060] To enable the video generation branch to clearly distinguish between the known observation area and the target area to be generated during training, thereby learning the generation process of future visual information more effectively, this application also provides an optional embodiment. Specifically, conditional masks are used to distinguish the respective regions of historical video representations and future video representations after injecting noise.

[0061] Optionally, the conditional mask is a binary tensor aligned temporally with the concatenated sequence. This includes, but is not limited to, a mask vector aligned temporally with the video representation, where the values ​​identifying historical observations differ from those identifying future parts to be generated. For example, the mask value is set to 1 for the first T_h time steps (corresponding to historical video representations); and to 0 for the next T_f time steps (corresponding to noisy future video representations). It is understood that a mask value of 1 represents that the features at that time step are given, deterministic conditions, which the video generation branch should use as contextual reference; a mask value of 0 represents that the features at that time step are noisy targets to be generated, which the video generation branch needs to predict and denoise.

[0062] In some implementations, the conditional mask is used as an additional input channel or positional encoding information, which is fed into the velocity field prediction network of the video generation branch along with the concatenated video representation sequence. This mask information is then used to differentiate between different regions in computations such as self-attention.

[0063] The above scheme introduces a conditional mask, which enables the video generation branch to clearly know the boundary between the observation region and the generation region in the input sequence. This allows the video generation branch to focus more on learning the dynamic modeling of the target region (future part) while making full use of the contextual information provided by the conditional region (historical part), thereby improving the accuracy of the video generation process and the controllability in sequence generation tasks.

[0064] To further improve the learning accuracy and training stability of the video generation branch, this application also provides an embodiment that enhances the model's ability to fit probabilistic flow paths by introducing an auxiliary directional derivative prediction task and jointly optimizing it with velocity field prediction. Specifically, based on the input data and training time difference, the velocity field prediction value and the auxiliary directional derivative prediction value output by the video generation branch are obtained; a first loss between the velocity field prediction value and the corresponding true velocity field value is calculated, and a second loss between the auxiliary directional derivative prediction value and the corresponding true directional derivative value is calculated; the model parameters of the video generation branch are updated based on the first loss and the second loss.

[0065] Optionally, the velocity field prediction network V in the video generation branch Designed to have at least two output heads. One output head generates the primary velocity field prediction vector. One output head generates an auxiliary directional derivative prediction value. This auxiliary directional derivative can be the derivative of the velocity field along the direction of the probability flow trajectory, or other scalar information related to the trajectory curvature, used to show the local geometric characteristics of data evolution.

[0066] Next, the first loss between the predicted velocity field value and the corresponding true velocity field value is calculated. Simultaneously, the second loss between the predicted auxiliary directional derivative value and the corresponding true directional derivative value is calculated. Both the true velocity field value and the true directional derivative value can be analytically derived from the current training sample data based on the mathematical definition of the probability flow ordinary differential equation upon which the improved mean-flow method is based. Optionally, the derivation process involves clean future video representations. Injected noise and the sampled time variables and The first and second losses can take the form of mean squared error or L1 loss, etc.

[0067] Then, the first loss and the second loss are weighted and summed according to certain weighting coefficients to obtain the total loss function. For example... λ is a hyperparameter. Finally, the gradient is calculated using this total loss through backpropagation, and the velocity field prediction network V is updated. All model parameters.

[0068] The above scheme simultaneously acquires the velocity field prediction value and the auxiliary directional derivative prediction value from the video generation branch output, and calculates the first loss and the second loss respectively. Based on the two losses, the model parameters are updated together, so that the model not only learns the accurate prediction of the velocity field during training, but is also constrained by the auxiliary directional derivative task, thereby improving the accuracy and training stability of velocity field learning, and thus improving the prediction quality of future video representations.

[0069] The foregoing embodiments have explained the training principle of the video generation branch step by step. Below, we will use... Figure 2 Taking this as an example, the end-to-end training process of the video generation branch is fully explained. Specifically, as... Figure 2 As shown, firstly, multi-view observation data (i.e., video samples) is acquired. This data is input into a unified video representation encoding module (i.e., a feature extraction network) to complete unified feature extraction and encoding of the multi-view video data, generating historical video representations containing historical scene information, providing visual context for subsequent generation. Simultaneously, the task text is input into a multimodal understanding module to achieve semantic parsing and feature encoding of the task instructions, generating a task semantic representation. Then, combined with temporal conditions (i.e., a time variable for constructing the training time difference), all three are fed into the video generation branch to execute the prediction process of future visual information, outputting a predicted future video representation. At the same time, a real future video representation is introduced, and loss supervision is formed by calculating the difference between the two, achieving accurate training of the video generation branch, ultimately enabling this branch to accurately predict future visual information that meets the task and time constraints.

[0070] To enhance the robustness and efficiency of motion prediction in the decoupled training mode, this application also provides an optional embodiment. Specifically, the video representation is enhanced to obtain the target visual conditions; a real motion sequence aligned with the video sample time is acquired; and the motion generation branch is optimized using motion flow matching based on the target visual conditions, the real motion sequence, and the robot state.

[0071] Optionally, augmentation refers to artificially introducing various forms of degradation or perturbation into the video representation input to the action generation branch during training to simulate the challenges of real deployment environments. The aim is to prevent the action generation branch from overfitting to clean, complete training data, thereby improving its generalization ability.

[0072] In some implementations, the real action sequence includes a real action sequence with injected noise. That is, in the action flow matching training framework, one of the inputs to the action generation branch is the action sequence after noise perturbation, rather than using clean actions directly. This forces the action generation branch to learn to recover the conditional distribution of reasonable actions from the noise distribution.

[0073] Correspondingly, action flow matching refers to a method for training an action generation model by supervising the evolution velocity field of a noisy action sequence. This includes, but is not limited to, injecting noise into real future action sequences to construct noisy actions, and training the action generation branch to predict the velocity of evolution from the noisy action to the real action, conditioned on video representations and robot states.

[0074] In some implementations, when training the action generation branch, the training first involves training on real action sequences. Injecting noise, for example at random times. Below, through linear interpolation Obtain the noisy action sequence ,in The noise is standard Gaussian noise. Then, the noisy action sequence is... The target visual conditions obtained after enhancement processing Robot status and time Together input action generation branch network The network outputs a predicted value, which can be the direction of the noise (i.e., the velocity field) or the noise itself. The training objective is to minimize the difference between the network output and the true target value defined by a selected flow path (such as a linear path).

[0075] The above scheme enhances the video representation to obtain the target visual conditions, improving the robustness of the action generation branch to interference from noise and frame drops in visual observation. Simultaneously, it optimizes the action generation branch using action flow matching and introduces real action sequences with injected noise, enabling the model to learn the mapping or velocity field from noisy actions to real actions. This supports efficient action sequence generation, reduces the number of iterations during inference, and improves response speed while maintaining action prediction accuracy.

[0076] To further improve the generalization ability of the action generation branch under complex observation conditions, this application provides a specific embodiment that perturbs the video representation through various enhancement processing methods.

[0077] Optionally, this enhancement method includes retaining a portion of the time steps in the video representation with a preset probability to achieve sparsity processing. For example, suppose the video representation is a sequence of time steps containing the most recent... A sequence of features at each time step. A retention probability can be preset. (e.g., 0.6). In each training iteration, for each time step in the sequence... Generate a random number. If the random number is less than 1, then... If the condition is met, the original feature vector for that time step is retained; otherwise, the feature vector for that time step is set to zero or replaced with a dedicated learnable mask label. Ultimately, only a subset of the features from each time step are retained as the visual conditional input action generation branch.

[0078] This operation simulates real-world scenarios such as sensor frame loss, data transmission packet loss, or processing delays, forcing the model to learn to make decisions based on discontinuous observation segments.

[0079] Optionally, the enhancement method also includes injecting noise into the video representation with a preset probability. For example, for each feature dimension of the video representation sequence (whether all or a retained portion), independent Gaussian noise is added with a certain probability, i.e.: ,in , It is a scalar quantity that controls the noise intensity. Injected noise can be applied randomly to the entire sequence or to certain randomly selected time steps.

[0080] This simulates the inherent measurement noise of the sensor, pixel jitter caused by changes in ambient lighting, or the uncertainty of the feature extraction process itself.

[0081] Optionally, this enhancement method also includes discarding the observation frame corresponding to the current moment in the video representation. In many control scenarios, the observation at the current moment (i.e., the latest frame) is usually considered to have the most information content. However, during training, this latest frame (corresponding to the current moment) in the video sequence can be randomly or intentionally discarded with a certain probability. This means that the motion generation branch cannot obtain the latest visual feedback when generating the current motion, and must instead rely on earlier historical frames. To infer the state of the environment and plan actions.

[0082] This enhancement helps improve the model's robustness in situations with self-occlusion, such as the robot's own robotic arm blocking the camera, or slight delays in sensor feedback, and encourages the model to learn dependencies over longer time periods.

[0083] The above scheme simulates situations such as perception loss, sensor noise, or communication delay that may occur in real deployment environments by performing sparsification processing, injecting noise, or discarding the current observation frame on the video representation. This enables the action generation branch to adapt to diverse observation quality during training, thereby improving the robustness and generalization ability of the model in scenarios such as local observation overfitting, incomplete observation, or interference, and avoiding excessive dependence of the action branch on specific observation modes.

[0084] The foregoing embodiments have explained the training principle of action generation branches step by step. Below, we will use... Figure 3Taking this example, the training process of the action generation branch is fully explained. Specifically, firstly, the real future video is input into the encoding module to complete feature extraction and encoding, generating a representation of the real future video. This representation is then fed into the visual condition filtering and enhancement module, which enhances the video representation and outputs target visual conditions that meet the requirements of action generation, providing precise visual context constraints for the action generation branch. Simultaneously, the training process introduces two types of core inputs: firstly, the robot's state, i.e., the robot's current pose, joints, and other ontological state information, used to provide ontological state constraints for action generation; secondly, noisy action input, serving as the initial input for action generation and adapting to the training logic of action flow matching. Then, all three types of inputs are fed into the action generation branch, which performs inference and prediction of action sequences based on multi-source constraints, outputting a predicted action sequence. Furthermore, a real action sequence is introduced as a supervision label, and the difference loss between the predicted action sequence and the real action sequence is calculated through action flow matching. Based on this loss, the model parameters of the action generation branch are updated through backpropagation.

[0085] The foregoing embodiments illustrate the principles of decoupling and joint training of the video generation branch and the action generation branch. Based on this, using... Figure 4 Taking this example, the reasoning process of the robot operation strategy model is fully explained. Specifically, firstly, two types of core information are input: multi-view observation data and task text. The multi-view observation data and task text are input in parallel to the multimodal understanding module to complete semantic parsing and multimodal feature fusion of the task text, generating task semantic features. On the other hand, the multi-view observation data is separately sent to the encoding module to complete feature extraction and encoding of the multi-view observation video, generating historical video representations. These two types of features, along with temporal conditions, are input to the video generation branch. This branch, based on historical visual information, task semantic constraints, and temporal conditions, completes the predictive reasoning of future visual information, outputting future video representations. These future video representations are used directly to generate future video outputs, enabling the prediction of the future visual state of the scene. They are also sent to the visual condition filtering and enhancement module for filtering and enhancement processing, generating target visual conditions that adapt to the action generation requirements. Subsequently, the target visual conditions and the robot state are input to the action generation branch. This branch, based on the future visual context and the robot's own state, completes the reasoning and prediction of the action sequence, ultimately outputting the future action sequence, providing a decision-making basis for the robot to perform operations.

[0086] Throughout the process, the video generation branch independently completes future visual prediction, while the action generation branch completes action prediction based on the future visual representation output by the video generation branch. The parameters of the two branches are completely decoupled, which not only ensures the independence of visual prediction and action prediction, but also achieves the coordination of visual perception and action decision-making through the transmission of visual conditions, ultimately enabling the robot to predict future scenes and generate operation strategies under task constraints.

[0087] Figure 5 The diagram shown is a structural schematic of a robot operation strategy model provided in an embodiment of this application. Figure 5 As shown, the model includes a feature extraction network 510, a video generation branch 520, and an action generation branch 530.

[0088] Specifically, the feature extraction network 510 is configured to process the observed video to obtain a unified video representation.

[0089] Optionally, the network is a pre-trained neural network deployed on the robot's computing unit (such as an industrial control computer or embedded AI chip). During the inference phase, it receives video streams from multiple cameras on the robot in real time, encodes these raw image sequences, and outputs a low-dimensional, continuous sequence of video representations. This representation serves as a unified understanding of the current visual environment within the model.

[0090] The video generation branch 520 is configured to predict future visual information based on the video representation corresponding to the observed video.

[0091] Optionally, the video generation branch 520 is a separate neural network module whose input is connected to the output of the feature extraction network 510.

[0092] In applications requiring prediction of future scenes, this branch receives current and historical video representations extracted by the feature extraction network 510 (as conditions) and runs its internal generation algorithm to output a prediction of the visual scene within a future timeframe. This prediction is typically also in the form of video representations, which can be used directly or converted into image frames by a decoder.

[0093] The action generation branch 530 is configured to predict action sequences based on the video representations corresponding to the observed videos. The model parameters of the video generation branch and the action generation branch are independent of each other.

[0094] Optionally, the motion generation branch 530 is another independent neural network module, whose input is also connected to the output of the feature extraction network 510. When the robot needs to make a decision, this branch receives the video representation provided by the feature extraction network 510 and combines it with the real-time state (such as joint positions) read from the robot's sensors. Through its internal calculations, it outputs a series of control commands, i.e., the predicted sequence of motions, to drive the robot to perform operations.

[0095] In some implementations, the model parameters of the video generation branch 520 and the motion generation branch 530 are independent of each other. That is, in the productized form of the model, the video generation branch and the motion generation branch are two completely independent software modules or hardware computing units. They have their own independent weight parameter files, are separate in memory or storage, and use different parameter sets when performing forward computation.

[0096] Furthermore, in addition to the video representation from the feature extraction network 510, the action generation branch 530 can be further connected to the output of the video generation branch 520 to receive the predicted future video representation as additional input. For example, in a joint inference scenario, the action generation branch 530 simultaneously acquires the currently observed video representation and the predicted future visual information from the video generation branch 520, thereby generating a more accurate action sequence based on the prediction of visual evolution. This connection method allows the two branches to maintain parameter independence while achieving information synergy during forward propagation, balancing module decoupling with predictive decision-making.

[0097] In the above scheme, the robot operation strategy model provides two core capabilities during inference by integrating a shared feature extraction network and a dual-branch structure with independent parameters. First, it has the ability to efficiently predict future visual changes based on current observations; second, it has the ability to generate robust action sequences based on visual observations. These two capabilities maintain semantic alignment through shared visual representations and maximize their respective effectiveness due to parameter independence, thereby improving the robot's autonomy, adaptability, and task success rate when performing complex tasks.

[0098] Below, for reference Figure 6 This describes an electronic device according to embodiments of the present application. Figure 6 The diagram shown is a structural schematic of an electronic device provided in an exemplary embodiment of this application.

[0099] like Figure 6 As shown, the electronic device 60 includes one or more processors 601 and memory 602.

[0100] The processor 601 may be a central processing unit (CPU) or other form of processing unit with data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 60 to perform desired functions.

[0101] The memory 602 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 601 may execute the program instructions to implement the training methods of the various embodiments of this application described above and / or other desired functions.

[0102] In one example, the electronic device 60 may also include an input device 603 and an output device 604, which are interconnected via a bus system and / or other forms of connection mechanism (not shown).

[0103] The input device 603 may include, for example, a keyboard, a mouse, etc.

[0104] The output device 604 can output various information to the outside. The output device 604 may include, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc.

[0105] Of course, for the sake of simplicity, Figure 6 Only some of the components of the electronic device 60 relevant to this application are shown in this illustration; components such as buses, input / output interfaces, etc., are omitted. In addition, the electronic device 60 may include any other suitable components depending on the specific application.

[0106] In addition to the methods and apparatus described above, embodiments of this application may also be computer program products, which include computer program instructions that, when executed by a processor, cause the processor to perform the steps of the training methods according to various embodiments of this application described above.

[0107] The computer program product can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of this application. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0108] Furthermore, embodiments of this application may also be computer-readable storage media storing computer program instructions thereon, which, when executed by a processor, cause the processor to perform the steps in the training methods according to various embodiments of this application described above.

[0109] The computer-readable storage medium may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0110] The basic principles of this application have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this application are merely examples and not limitations, and should not be considered as essential features of each embodiment of this application. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the application to the necessity of employing the aforementioned specific details for implementation.

[0111] The block diagrams of devices, apparatuses, devices, and systems involved in this application are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0112] It should also be noted that in the apparatus, equipment, and methods of this application, the components or steps can be disassembled and / or recombined. These disassemblies and / or recombinations should be considered as equivalent solutions of this application.

[0113] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of this application. Therefore, this application is not intended to be limited to the aspects shown herein, but rather to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0114] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this application to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. A method for training a robot operation strategy model, characterized in that, The robot operation strategy model includes a feature extraction network, a video generation branch, and an action generation branch; the method includes: The feature extraction network is used to process video samples to obtain video representations; Based on the video representation, the video generation branch is trained to predict future visual information; Based on the video representation, the action generation branch is trained to predict action sequences; The model parameters of the video generation branch and the action generation branch are decoupled.

2. The training method according to claim 1, characterized in that, The training modes for the video generation branch and the motion generation branch include a decoupled training mode and a joint training mode; During the decoupled training, the model parameters of the video generation branch and the action generation branch are updated independently; during the joint training, the input data of the action generation branch includes the future video representation output by the video generation branch, and the model parameters of the video generation branch and the action generation branch are updated using a gradient truncation method.

3. The training method according to claim 1, characterized in that, The video representation is divided into historical video representation and future video representation; The step of training the video generation branch based on the video representation includes: Construct input data, which includes task text, the historical video representation, and the future video representation; By sampling two time variables of the improved mean stream, a training time difference is constructed. During the training time difference, the velocity field of the video generation branch is learned based on the input data.

4. The training method according to claim 3, characterized in that, The future video representation includes a future video representation with injected noise; wherein, the regions of the historical video representation and the future video representation with injected noise are distinguished by a conditional mask.

5. The training method according to claim 3, characterized in that, The step of learning the velocity field of the video generation branch based on the input data under the training time difference includes: Based on the input data and the training time difference, obtain the velocity field prediction value and auxiliary directional derivative prediction value of the video generation branch output; Calculate the first loss between the predicted velocity field value and the corresponding true velocity field value, and calculate the second loss between the predicted auxiliary directional derivative value and the corresponding true directional derivative value; The model parameters of the video generation branch are updated based on the first loss and the second loss.

6. The training method according to claim 1, characterized in that, In the decoupled training mode of the action generation branch, training the action generation branch based on the video representation includes: The video representation is enhanced to obtain the target visual conditions; Obtain the real action sequence that is time-aligned with the video sample; Based on the target visual conditions, the actual action sequence, and the robot state, the action generation branch is optimized using an action flow matching method; The real action sequence includes the real action sequence after noise injection.

7. The training method according to claim 6, characterized in that, The enhancement processing of the video representation to obtain the target visual conditions includes at least one of the following: A portion of the time steps in the video representation are retained according to a preset probability to achieve sparsity processing; Noise is injected into the video representation according to a preset probability; Discard the observation frame in the video representation corresponding to the current moment.

8. A robot operation strategy model, characterized in that, include: A feature extraction network is configured to process observed videos to obtain a unified video representation. The video generation branch is configured to predict future visual information based on the video representation corresponding to the observed video. The action generation branch is configured to predict action sequences based on the video representation corresponding to the observed video; The model parameters of the video generation branch and the motion generation branch are independent of each other.

9. A computer-readable storage medium, characterized in that, The storage medium stores a computer program for executing the training method according to any one of claims 1 to 7.

10. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is used to execute the training method according to any one of claims 1 to 7.

11. A computer program product, characterized in that, The computer program product includes instructions that, when executed on an electronic device, cause the electronic device to perform the training method according to any one of claims 1 to 7.

Images