Robot motion control methods, motion control models and model training methods
By performing Gaussian distribution modeling and iterative denoising prediction on the robot's current environment and historical motion data, the problems of timing confusion and environmental adaptation in robot motion control are solved, achieving high-precision motion control and robust autonomous operation.
Patent Information
- Application Number
- CN202611283915.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-08-21
- Publication Date
- 2026-11-17
AI Technical Summary
Existing robot motion control methods lack a systematic mechanism for fusing and utilizing historical action features and context in complex working conditions. They cannot efficiently distinguish between temporally confused states, making it difficult to ensure the consistency of temporal execution and environmental adaptability for long-term operation tasks. Furthermore, traditional imitation learning training modes do not consider real-world issues such as sensor noise, resulting in insufficient adaptability of motion strategy models in real-world scenarios.
By acquiring the target robot's current environmental observation data and historical action data, performing feature encoding, and then modeling with a Gaussian distribution to generate initial noise, the robot predicts future action results through iterative denoising. By combining a causal noise generator and a causal action generator, the robot achieves accurate prediction and control of its actions.
It improves the accuracy of robot motion decision-making and environmental generalization ability, enhances the robustness of autonomous operation in complex working conditions, and ensures the consistency of action timing and the diversity of environmental adaptation in long-term operation.
Smart Images

Figure CN123113335A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of artificial intelligence technology, and in particular to a robot motion control method, a motion control model, and a model training method. Background Technology
[0002] As the application scenarios of long-range autonomous operation tasks for intelligent robots continue to expand, visual motion decision-making strategies based on imitation learning are being widely used. By training strategy models with expert demonstration data, robots can quickly learn the action logic of various practical operations.
[0003] Current mainstream motion strategy generation schemes are mainly divided into two categories: pure observation condition diffusion strategies and observation history caching mechanisms. Pure observation condition diffusion strategies rely on current environmental observation information to generate actions, but they discard historical action feature context information in accordance with the Markov property assumption. They cannot infer the actual task progress based on historical execution trajectories. They are prone to misjudgment when faced with temporal confusion states where visual appearances are similar but execution actions are different, which can easily lead to action loop disorder and make it difficult to ensure the temporal execution consistency of long-term operation tasks. On the other hand, existing schemes that adopt observation history caching mechanisms focus on storing and utilizing historical observation data. They have weak anti-interference ability against perceptual noise. They not only have difficulty accurately fitting multimodal action distributions and building stable long-term dependencies, but also have defects such as reliance on manual design for historical information screening and a single form of context fusion. They cannot take into account both the compliance of physical constraints in action generation and the diversity of environmental adaptation.
[0004] Meanwhile, traditional imitation learning training models heavily rely on clean, unbiased expert demonstration data, failing to consider real-world issues such as sensor noise, actuator slippage, numerical errors in inverse kinematics solutions, and unknown environmental disturbances during actual robot operations. Various physical interaction errors accumulate continuously during the autoregressive sequence generation process, gradually contaminating historical action feature information. This ultimately leads to a distributional shift between the clean, standard historical data used for training and the biased historical data used in the actual inference stage. Therefore, the current lack of a systematic mechanism for fusing and utilizing historical action feature context makes it impossible to efficiently distinguish temporally confused states, balance the physical feasibility of actions with the diversity of dynamic generation, or adapt training methods to complex cumulative errors. Consequently, the trained motion strategy model lacks adaptability in real-world complex working conditions and undistributed scenarios, limiting the overall robustness of operational control and its practical effectiveness. Summary of the Invention
[0005] Therefore, it is necessary to provide a robot motion control method, motion control model, and model training method to address the above-mentioned technical problems and solve at least one of the problems existing in the prior art.
[0006] Firstly, a robot vision motion control method is provided, including: Acquire the target robot's current environmental observation data and historical action data, and encode them separately to obtain the current environmental observation features and historical action features; Gaussian distribution modeling is performed on the current environmental observation features and historical action features to obtain a dynamic Gaussian distribution, and the dynamic Gaussian distribution is sampled to obtain initial noise, which is then used as the current action estimate. Based on the current environmental observation features, historical action features, and current action estimates, the future action results of the target robot are obtained through iterative noise reduction prediction. Based on the predicted future action results, the target robot is controlled to perform corresponding action operations.
[0007] In one possible implementation, the step of modeling the current environmental observation features and historical action features using a Gaussian distribution to obtain a dynamic Gaussian distribution, and then sampling the dynamic Gaussian distribution to obtain initial noise, includes: Based on the current environmental observation characteristics and historical action characteristics, the mean and variance parameters of the Gaussian distribution are predicted. Based on the mean and variance parameters of the Gaussian distribution, construct the dynamic Gaussian distribution corresponding to the current moment; The initial noise is generated by probabilistically sampling the dynamic Gaussian distribution corresponding to the current time using the Gaussian reparameterization technique.
[0008] In one possible implementation, the step of predicting the future action result of the target robot through iterative denoising based on the current environmental observation features, historical action features, and current action estimates includes: The block autoregression method is used to perform back diffusion loop iteration denoising based on the corresponding environmental observation features, historical action features and action estimates in the current action block according to the preset number of iterations, and to predict the future action result corresponding to the current action block. The historical action features are updated based on the target robot's action execution result for the future action result corresponding to the current action block, and the prediction of the next action block is performed based on the updated historical action features.
[0009] In one possible implementation, the step of performing backdiffusion loop iterative denoising based on the corresponding environmental observation features, historical action features, and action estimates according to a preset number of iterations to predict the future action result corresponding to the current action block includes: In each iteration, based on the action estimate, noise level, current environmental observation features, and historical action features corresponding to the current iteration, the pure action estimate result corresponding to the current iteration is predicted; Based on preset noise scheduling parameters, noise scheduling coefficients and random Gaussian noise, the action estimate value corresponding to the next iteration round is calculated, and the single iteration update is completed. After completing the noise reduction process for all iterations, the final result of the future action corresponding to the current action block is obtained.
[0010] In one possible implementation, the step of predicting the clean action estimation result corresponding to the current iteration round based on the action estimation value, noise level, current environmental observation features, and historical action features includes: By applying causal temporal constraints to the historical action features and action estimates corresponding to the current iteration round, temporal features are extracted. The temporal features are fused with the current environmental observation features through cross-attention calculation to obtain the global fused features; Based on the global fusion features and the noise level corresponding to the current round, the pure action estimation result corresponding to the current iteration round is predicted.
[0011] In one possible implementation, the step of applying causal temporal constraints to the historical action features and action estimates corresponding to the current iteration round to extract temporal features includes: Divide the historical actions corresponding to the current action block into cached historical actions and uncached historical actions; Read the key and value characteristics corresponding to the cached historical actions; Linear transformations are performed on the action estimates corresponding to the uncached historical actions and the current action block, respectively. The query features, key features, and value features corresponding to the uncached historical actions, as well as the query features, key features, and value features corresponding to the action estimates of the current action block, are extracted. Based on causal temporal attention mask, key-value concatenation and causal attention calculation are performed on the key features and value features corresponding to cached historical actions, the query features, key features and value features corresponding to uncached historical actions, and the query features, key features and value features of the action estimate corresponding to the current action block, to generate the temporal features corresponding to the current action block; The key and value features corresponding to newly added uncached historical actions will be cached for the next action block to perform the autoregressive prediction step.
[0012] In one possible implementation, the method further includes: Based on the global time offset corresponding to the current action block, determine the absolute time step of the historical action features in the current autoregressive sliding window on the global time axis; Map the absolute time step to a fixed-length position encoding table index; The input features are encoded based on the location encoding table index to obtain a historical action feature representation after injecting cyclic timing information, and the global time offset corresponding to the current action block is updated so as to execute the autoregressive step of the next action block based on the updated global time offset.
[0013] Secondly, a motion control model is provided to implement the robot motion control method described above. The model includes a causal noise generator and a causal motion generator. The causal noise generator includes a noise encoder and a noise decoder. The noise encoder is used to model the current environmental observation features and historical action features using a Gaussian distribution to obtain a dynamic Gaussian distribution, and to sample the dynamic Gaussian distribution to obtain initial noise. The causal motion generator includes several stacked attention modules, each of which includes a causal temporal attention block and a visual-motion cross-attention block, used to iteratively denoise the initial noise and predict the future motion sequence corresponding to the target robot.
[0014] Thirdly, a training method for a motion control model is provided for training the motion control model described in the second aspect above. The training method includes a pre-training stage for a causal noise generator and a joint training stage for the causal noise generator and the causal action generator. The joint training stage includes a teacher-forced training sub-stage, a self-forced training sub-stage, and a reinforcement-forced training sub-stage.
[0015] In one possible implementation, the teacher-forced training sub-stage is used to randomly sample the noise level using the structured initial noise output by the causal noise generator as the noise source, add noise to the real action through a forward diffusion process to the noise level, obtain an intermediate noise action estimate, obtain a predicted action based on the intermediate noise action estimate, and perform iterative training based on the loss between the predicted action and the real action. The self-forced training sub-stage is used to predict future actions block by block in an autoregressive manner, and to use the future action corresponding to the current action block as the historical action feature of the next action block and perform iteration. The iteration is performed for multiple training rounds, and the action sequence fragments with intrinsic bias generated in each iteration process are stored in the intrinsic data pool. The enhanced forced training sub-stage is used to extract historical action features from the internal data pool and the expert demonstration data pool according to the uniform distribution sampling rule, and to perform iterative training based on the extracted historical action features and real actions.
[0016] The aforementioned robot motion control method, motion control model, and model training method include: acquiring current environmental observation data and historical action data of the target robot, and performing feature encoding on them respectively to obtain current environmental observation features and historical action features; modeling the current environmental observation features and historical action features using a Gaussian distribution to obtain a dynamic Gaussian distribution, and sampling the dynamic Gaussian distribution to obtain initial noise, using the initial noise as the current action estimate; predicting the future action result of the target robot through iterative denoising based on the current environmental observation features, historical action features, and current action estimate; and controlling the target robot to perform corresponding action operations based on the future action result. In this embodiment, by fully utilizing the contextual information of historical action features combined with environmental observation features to complete action prediction, it is possible to accurately identify temporal confusion states, accurately infer task progress, and ensure the consistency of long-term operation action timing. By leveraging the dynamic Gaussian distribution sampling and iterative denoising mechanism, it takes into account both the physical feasibility of actions and the diversity of environmental adaptation generation, effectively improving the robot's motion decision accuracy, environmental generalization ability, and overall robustness of autonomous operation under complex conditions. Attached Figure Description
[0017] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0018] Figure 1 This is a schematic diagram of the network architecture of a causal diffusion strategy in one embodiment of this application; Figure 2 This is a flowchart illustrating a robot motion control method according to an embodiment of this application; Figure 3 This is a schematic diagram of a reasoning framework for a motion control model in one embodiment of this application; Figure 4 This is a schematic diagram of a training framework for a motion control model in one embodiment of this application; Figure 5 This is a schematic diagram of a robot motion control device according to one embodiment of this application; Figure 6 This is a schematic diagram of a computer device according to one embodiment of this application. Detailed Implementation
[0019] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0020] The robot motion control method provided in this embodiment can be applied to, for example, Figure 1 The causal diffusion strategy network architecture shown includes a causal noise generator (CNG) and a causal action generator (CAG) set within the green dashed box. This architecture also integrates an environment observation coding module and a temporal attention mask design module. It standardizes the complete data flow transmission links for feature input, conditional constraint propagation, and action prediction output. All functional modules work together, relying on historical action feature temporal constraints and hierarchical denoising inference logic, to accurately generate the robot's temporal actions and make autonomous motion decisions.
[0021] Specifically, the network architecture includes two types of core input data: one is current environmental observation data, Figure 1 The first part uses point cloud as visual input. This data is processed by a 2D or 3D encoder labeled E to extract features and map them into low-dimensional observation features. These observation features are input into the noise decoder of the causal noise generator CNG and the denoising network of the causal action generator CAG, respectively, to provide spatial environmental context information for action prediction. The second part is the historical action feature sequence that the robot has completed. This historical action sequence is uniformly encoded and input into the causal noise generator CNG. At the same time, it is directly used as a temporal constraint condition to access the causal action generator CAG, realizing bidirectional collaborative linkage between environmental spatial information and action temporal information.
[0022] The causal noise generator (CNG) extracts low-frequency dynamic features of the task through a noise encoder, completes dynamic Gaussian distribution parameter modeling, and then outputs structured initial noise with task adaptability and physical rationality through sampling and decoding. This initial noise, combined with observation features and historical action features, is input into the CAG denoising network. The denoising network relies on the CTA structure to integrate the temporal constraints of historical action features and incorporates the current observation space constraints through the VCA mechanism, ultimately predicting and outputting future actions. In the overall architecture, the CNG is responsible for constructing an initial benchmark for action generation that takes into account both physical feasibility and task relevance. Based on this, the CAG performs refined iterative denoising with temporal smoothing and precise observation alignment, steadily and efficiently generating robot temporal actions.
[0023] Example 1 In one embodiment, such as Figure 2As shown, a robot motion control method is provided, including the following steps: In step S110, the current environmental observation data and historical action data of the target robot are acquired, and feature encoding is performed on them respectively to obtain the current environmental observation features and historical action features. Optionally, real-time environmental observation data of the target robot's operating scenario is collected, while historical motion data stored during robot operation is retrieved. Then, 2D / 3D encoders are used to perform feature encoding processing on the two types of data. The encoders perform dimensional regularization of the raw data, removal of redundant information, and extraction of deep semantic information, transforming the unstructured and disorganized raw data into a standardized representation that the model can recognize and participate in computation. This results in standardized current environmental observation features and historical motion features. The two types of features, after encoding processing, can accurately carry environmental spatial information and temporal motion information, unifying the data input specifications.
[0024] The 2D encoder is suitable for robot operation scenarios primarily based on image observation. Its input data includes multi-channel scene image observation information and robot body state information, specifically including joint motion states, end effector poses, and other motion parameters. The encoder employs a dual-branch parallel coding structure, extracting features independently from the image data and the body state data. Specifically: for the image branch, preprocessing operations such as scaling, region cropping, and pixel normalization are performed on the original input image to eliminate scale differences, noise interference, and numerical biases. Then, a convolutional backbone network automatically extracts deep visual semantic features such as scene texture, target location, and environmental structure. For the state branch, complete kinematic temporal information and state parameter information of the robot body are preserved, outputting high-fidelity low-dimensional state features to avoid loss of motion details. After completing the dual-branch feature extraction, the multi-channel image semantic features and the robot's low-dimensional state features are concatenated and fused along the feature dimension to generate a globally observed feature vector with unified dimensions and complementary information. The fused observation feature vector can simultaneously carry scene environment spatial information and robot body motion state information, and can serve as an effective condition input for the causal diffusion strategy model and causal noise generation module, providing comprehensive and accurate environmental and ontological prior constraints for subsequent dynamic Gaussian distribution modeling and iterative denoising action prediction.
[0025] The 3D encoder is suitable for robot operation scenarios where 3D point clouds are the primary form of observation. Its input data includes scene point cloud observation information and robot body state information, where the body state information includes key kinematic parameters such as robot joint states and end effector poses. This encoder also employs a dual-branch independent extraction and back-end fusion structure. Specifically, the point cloud branch uses a PointNet-like encoding structure. First, a point-by-point multilayer perceptron is used to perform nonlinear mapping on the original point cloud data, extracting local geometric features and spatial structure features point by point. Then, global pooling is used to aggregate global information, generating scene-level geometric features that characterize the overall scene topology and spatial layout. The state branch uses a multilayer perceptron network to perform feature mapping on the robot body state data, adaptively extracting body motion features strongly correlated with the operation, resulting in a dimensionally regular and semantically focused state representation. After completing the dual-branch independent feature encoding, the 3D point cloud geometric features and robot body state features are concatenated and fused according to feature dimensions, ultimately outputting a uniformly sized observation condition vector. This approach not only fully preserves the inherent three-dimensional spatial geometric structure information of the point cloud, but also effectively introduces the robot's own motion constraints, achieving deep coupling between the three-dimensional structural information of the environment and the robot's own state information, thus providing accurate prior observation conditions for subsequent motion modeling and denoising prediction.
[0026] In step S120, Gaussian distribution modeling is performed on the current environmental observation features and historical action features to obtain a dynamic Gaussian distribution, and the dynamic Gaussian distribution is sampled to obtain initial noise, which is then used as the current action estimate. Optionally, an autoregressive inference mechanism can be used to generate the robot's future actions block by block, specifically predicting continuous actions according to the temporal progression of each action block. Unlike the single-step independent action generation mode, this autoregressive mechanism uses the time axis as the progression benchmark, dividing the robot's continuous operation sequence of future actions into multiple continuous action blocks, and generating each action block iteratively. After each action block prediction and actual robot execution, the system updates the action history cache in real time, incorporating the newly generated action block into the temporal history constraints, and simultaneously discarding outdated historical action information, always maintaining a complete and valid temporal context. The generation of subsequent action blocks is based on the updated historical action cache and real-time environmental observations for modeling, sampling, and denoising inference. The execution results of previous action blocks constrain the logic of subsequent action generation, ensuring temporal coherence, motion smoothness, and task execution consistency among multiple action blocks, adapting to the robot's long-term, continuous, and multi-stage complex operation tasks.
[0027] like Figure 3As shown, at the start of autoregressive inference, the system pre-maintains an ordered cache of action history along the time axis, with the time dimension continuously advancing along the "autoregressive direction" arrow in the diagram. Historical actions are categorized into three types based on their functional state: invalid action history with long execution times and no effective constraint on the current decision; uncached action history recently generated by the model and not yet cached; and cached action history with completed attention weight calculations and solidified storage. The system fully reads the complete current action history, including both cached and uncached actions, ensuring the integrity and continuity of temporal information. In each round of autoregressive inference, the causal noise generator (CNG) is invoked to read the current complete action history and real-time environmental observation features. The internal noise encoder combines temporal motion information and spatial environment information to dynamically fit and output the mean and variance parameters of a Gaussian distribution, constructing a dynamic Gaussian distribution adapted to the current task progress and scene conditions in real time, replacing the traditional static noise distribution with fixed parameters. Then, random sampling is performed on this dynamic Gaussian distribution to obtain structured initial noise with task relevance and physical feasibility, which is sent to the CAG as the initial action estimate for this round of inference. Figure 3 The yellow square indicates the method for predicting the next action of the target robot.
[0028] In step S130, based on the current environmental observation features, historical action features, and current action estimates, the future action results of the target robot are obtained through iterative denoising prediction. Optionally, the sampled initial noise is used as the initial motion estimate and input into the downstream causal motion generator (CAG) for subsequent refined denoising optimization. The CAG uses current environmental observation features as spatial constraints and historical motion features as temporal constraints, deeply fusing these two types of features with the initial motion estimate containing structured noise. Relying on its internal temporal attention and spatial alignment mechanisms, it continuously corrects noise biases in the initial motion estimate, gradually eliminating invalid motion information that does not conform to environmental spatial constraints, robot physical motion laws, and task temporal logic. Through multiple iterations of denoising optimization, it continuously approximates the realistic and feasible target motion distribution, gradually correcting the biases and flaws in the initial motion estimate. Ultimately, it obtains a refined future motion result that is temporally smooth, observationally aligned, and meets physical feasibility and task rationality requirements, providing a reliable basis for the robot's subsequent precise execution of tasks.
[0029] It should be noted that iterative denoising prediction can be completed progressively in the form of action blocks based on an autoregressive mechanism. The subsequent denoising process is continuously constrained by the real-time updated historical action cache, ensuring the continuity and stability of action output in long-term continuous operation tasks.
[0030] In step S140, the target robot is controlled to perform the corresponding action operation based on the future action result.
[0031] Optionally, once the future action result is predicted, the system can parse it into low-level execution instructions that the robot can recognize, accurately mapping them to control dimensions such as robot joint movement, end effector displacement, and posture adjustment, driving the robot to complete the corresponding physical interaction operations in the current work scenario. Simultaneously, under the autoregressive continuous inference mechanism, after the robot executes the currently output action instructions in real time, the system updates the effective actions executed in that round to the action history cache in real time, iteratively updating the temporal context information. This provides the latest temporal constraints for the next moment's observation encoding, Gaussian distribution modeling, and action denoising prediction, forming a closed-loop control process of "prediction—execution—cache update—iterative prediction," continuously ensuring the temporal coherence, motion stability, and operational accuracy of the robot's long-range operation tasks.
[0032] This application provides a robot motion control method, comprising: acquiring current environmental observation data and historical action data of a target robot, and performing feature encoding on them respectively to obtain current environmental observation features and historical action features; modeling the current environmental observation features and historical action features using a Gaussian distribution to obtain a dynamic Gaussian distribution, and sampling the dynamic Gaussian distribution to obtain initial noise, using the initial noise as a current action estimate; predicting the future action result of the target robot through iterative denoising based on the current environmental observation features, historical action features, and current action estimate; and controlling the target robot to perform corresponding action operations based on the future action result. In this application embodiment, by fully utilizing the contextual information of historical action features combined with environmental observation features to complete action prediction, it can accurately identify temporal confusion states, accurately infer task progress, and ensure the consistency of long-term operation action timing. By leveraging the dynamic Gaussian distribution sampling and iterative denoising mechanism, it balances the physical feasibility of actions with the diversity of environmental adaptation generation, effectively improving the robot's motion decision-making accuracy, environmental generalization ability, and overall robustness of autonomous operation under complex conditions.
[0033] In one embodiment of this application, the step of modeling the current environmental observation features and historical action features using a Gaussian distribution to obtain a dynamic Gaussian distribution, and then sampling the dynamic Gaussian distribution to obtain initial noise, includes: Based on the current environmental observation characteristics and historical action characteristics, the mean and variance parameters of the Gaussian distribution are predicted. Based on the mean and variance parameters of the Gaussian distribution, construct the dynamic Gaussian distribution corresponding to the current moment; The initial noise is generated by probabilistically sampling the dynamic Gaussian distribution corresponding to the current time using the Gaussian reparameterization technique.
[0034] Optionally, the encoded current environment observation features and historical action features are concatenated along the time dimension and input into the Transformer encoder. The encoder's self-attention mechanism adaptively models the temporal coupling between historical actions and current environment observations, fully exploring the matching relationship between temporal motion change patterns and scene spatial constraints, and outputting a joint temporal latent representation that integrates global information. Subsequently, the latent representations of action branches are subjected to temporal length adaptation processing to deduce the latent vectors corresponding to future preset time steps. Based on the latent vectors of each time step, the mean and log-variance parameters of the corresponding latent variables are regressed and predicted, thereby constructing a dynamic Gaussian distribution.
[0035] Unlike traditional static Gaussian distributions with fixed parameters, this dynamic Gaussian distribution dynamically and adaptively adjusts its parameters according to environmental changes, task progress, and historical motion trends, exhibiting strong scene adaptability. After obtaining the real-time updated mean and variance parameters, a dynamic Gaussian distribution specific to the current inference moment is constructed based on these two sets of parameters, accurately characterizing the reasonable range of robot motion fluctuations and the distribution law of motion uncertainty in the current scene. On this basis, a Gaussian reparameterization technique is used to perform probabilistic sampling on this dynamic Gaussian distribution. Noise random sampling is completed through differentiable sampling, which not only ensures the gradient transferability of the sampling process and meets the requirements of model backpropagation training, but also generates structured initial noise that fits the physical motion constraints and adapts to the current task logic, providing a high-quality, scene-adaptive initial motion estimation benchmark for subsequent iterative denoising and prediction.
[0036] In one embodiment of this application, the step of predicting the future action result of the target robot through iterative denoising based on the current environmental observation features, historical action features, and current action estimates includes: The block autoregression method is used to perform back diffusion loop iteration denoising based on the corresponding environmental observation features, historical action features and action estimates in the current action block according to the preset number of iterations, and to predict the future action result corresponding to the current action block. The historical action features are updated based on the target robot's action execution result for the future action result corresponding to the current action block, and the prediction of the next action block is performed based on the updated historical action features.
[0037] Optionally, a reasoning logic is adopted to generate actions block by action block. For the current action block to be predicted, combined with the matched environmental observation features, temporal historical action features, and initial action estimates, multiple rounds of backdiffusion cyclic denoising operations are performed according to a preset number of iterations. By continuously eliminating structured noise in the initial action estimate, correcting action deviations, and fusing spatial observation constraints and temporal motion constraints, the action accuracy is gradually optimized, and finally, a refined future action result corresponding to the current action block is predicted. After completing the prediction and output of the current action block, the system updates the historical action features in the temporal cache in real time based on the actual execution feedback and action results of the target robot for the action block, and iteratively refreshes the system's temporal context information to ensure that the historical temporal data always matches the latest task execution state. Subsequently, based on the updated historical action features, the backdiffusion denoising prediction of the next action block is carried out. In this way, the process is carried out block by block and iteratively to achieve the generation of continuous, long-term robot action sequences, effectively ensuring the temporal continuity, motion smoothness, and task execution consistency between multiple action blocks, and adapting to the control requirements of complex continuous robot operation scenarios.
[0038] In one embodiment of this application, the step of performing backdiffusion loop iterative denoising based on corresponding environmental observation features, historical action features, and action estimates according to a preset number of iteration steps to predict the future action result corresponding to the current action block includes: In each iteration, based on the action estimate, noise level, current environmental observation features, and historical action features corresponding to the current iteration, the pure action estimate result corresponding to the current iteration is predicted; Based on preset noise scheduling parameters, noise scheduling coefficients and random Gaussian noise, the action estimate value corresponding to the next iteration round is calculated, and the single iteration update is completed. After completing the noise reduction process for all iterations, the final result of the future action corresponding to the current action block is obtained.
[0039] Optionally, for the current action block, based on the corresponding environmental observation features, historical action features, and initial action estimates, multiple rounds of back-diffusion iterative denoising are performed according to a preset number of iterations to accurately predict the future action result corresponding to the current action block. Specifically, the initial noise output by the CNG can be received by the CAG and used as the initial action estimate for this iteration of denoising. In each iteration, the system executes a process containing... The reverse diffusion loop of the step. For any denoising step (from Gradually down to The single-step denoising network receives the current noise action estimate. Noise level Current observation characteristics and action history As a joint input, it fully integrates scene spatial constraints, robot physical motion constraints, and task temporal constraints to accurately predict and output the pure motion estimation result corresponding to the current iteration round, gradually removing invalid noise and bias information mixed in the motion estimation value. After completing the pure motion prediction of a single round, it combines preset noise scheduling parameters and noise scheduling coefficients with random Gaussian noise to adaptively calculate and update the motion estimation value required for the next iteration round, realizing parameter update and motion optimization for a single iteration. By repeatedly executing the above multi-round back diffusion denoising and motion update process, it gradually corrects motion deviations, optimizes motion accuracy, and continuously approaches the optimal motion distribution that conforms to the real working scene and motion law, after a complete process. After iterative noise reduction processing, the final output is a high-precision, time-smooth, and physically feasible result of the future action corresponding to the current action block. This provides accurate motion prediction support for the robot's continuous block-by-block motion output and long-term task execution.
[0040] The estimated value of the next action with lower noise can be calculated using the following formula: ; in, For noise dispatch coefficient, It is random Gaussian noise. This is a single-step denoising network.
[0041] In one embodiment of this application, the step of predicting the clean action estimation result corresponding to the current iteration round based on the action estimation value, noise level, current environmental observation features, and historical action features includes: By applying causal temporal constraints to the historical action features and action estimates corresponding to the current iteration round, temporal features are extracted. The temporal features are fused with the current environmental observation features through cross-attention calculation to obtain the global fused features; Based on the global fusion features and the noise level corresponding to the current round, the pure action estimation result corresponding to the current iteration round is predicted.
[0042] Optionally, causal temporal constraints are applied to the historical action features and the estimated action values to be optimized in the current iteration. This constraint mechanism masks invalid future temporal information and ensures temporal causality, accurately uncovering the motion correlations and trends between actions and extracting highly robust temporal features. Subsequently, the extracted temporal features are fused with current environmental observation features using visual-action cross-attention calculation. This establishes a precise mapping between temporal motion information and 3D scene environment information, fully integrating environmental priors such as scene spatial structure, obstacle distribution, and target location, as well as temporal motion priors, to obtain globally fused features that balance spatial and temporal constraints. Finally, based on the noise level corresponding to the current iteration and the globally fused features, the clean action sequence corresponding to this iteration is accurately deduced and predicted.
[0043] Specifically, the CAG internally stacks multiple layers of Causal Temporal Attention (CTA) modules and Vision-Action Cross-Attention (VACA) modules. During iterative denoising, the CTA module needs to continuously access the action history to provide strict temporal constraints. These temporal constraints are the sequence of actions actually performed by the robot in the past. Confirmed. In the block-based autoregressive inference stage, the system employs a rolling update mechanism to maintain the temporal context: after each action block prediction and execution is completed, the earliest historical action block within the temporal window is automatically discarded, while the latest completed action block is added to the end of the historical sequence. This achieves dynamic sliding updates of the temporal constraint window, ensuring that the temporal information always focuses on recent effective motion trends, aligning with the robot's real-time operational status. To accurately embed dynamic temporal constraints into the iterative denoising process, the CTA module performs differentiated QKV mapping on the input features: mapping the current estimated iterative action to be optimized to a query feature, and mapping the dynamically updated effective historical action sequence to key and value features. Simultaneously, combined with a causal temporal attention masking mechanism, the current action estimate can effectively associate with complete and valid historical temporal information, fully mining and integrating the low-frequency motion patterns and operational rules inherent in historical actions. This constrains the current denoising optimization process, ensuring that the predicted future actions and historical operations are smoothly connected temporally and have highly consistent motion trends, effectively suppressing action jumps and temporal misalignments. Figure 3 As shown, the historical action sequence composed of orange and dark brown squares is the effective historical action segment input to the CTA module, used to provide dynamic timing constraints.
[0044] In one embodiment of this application, the step of applying causal temporal constraints to the historical action features and action estimates corresponding to the current iteration round to extract temporal features includes: Divide the historical actions corresponding to the current action block into cached historical actions and uncached historical actions; Read the key and value characteristics corresponding to the cached historical actions; Linear transformations are performed on the action estimates corresponding to the uncached historical actions and the current action block, respectively. The query features, key features, and value features corresponding to the uncached historical actions, as well as the query features, key features, and value features corresponding to the action estimates of the current action block, are extracted. Based on causal temporal attention mask, key-value concatenation and causal attention calculation are performed on the key features and value features corresponding to cached historical actions, the query features, key features and value features corresponding to uncached historical actions, and the query features, key features and value features of the action estimate corresponding to the current action block, to generate the temporal features corresponding to the current action block; The key and value features corresponding to newly added uncached historical actions will be cached for the next action block to perform the autoregressive prediction step.
[0045] Optionally, since the feature representations corresponding to historical actions will not change once determined during the causal generation process, a key-value cache sharing mechanism is added inside the CTA module to solve the problems of repeated calculations and high inference latency in long-term inference. This mechanism is used to store and reuse the intermediate key and value representations of historical action features, eliminating the need to repeatedly encode features of fixed historical sequences in each iteration. This significantly reduces the computational overhead under long-term time constraints and ensures the real-time performance of online inference.
[0046] Specifically, in the first One autoregressive step The denoising process (assuming the current time step is denoising) In ), the length is Historical action context Historical actions are divided into two parts: cached historical actions and uncached historical actions. For cached historical actions, their corresponding indexes... , length is Since it belongs to a historical action that has already been denoised and determined, its key and value have already been calculated and stored in the cache in the previous autoregressive step, and are denoted as follows: and (superscript) (This represents a deterministic state free of noise), and the system can directly read these two tensors from the cache. For uncached historical actions, their corresponding indices... , length is Since this part involves a new swipe to enter the history window in the current step, the system needs to... The linear transformation layer extracts the query, key, and value in real time, which can be obtained using the following formula: ; At the same time, for lengths of The current estimated value of the action to be denoised (i.e., the initial noise) Corresponding timing position Similarly, through The linear layer calculates its denoising time step The following features are represented as follows: ; Among them, the newly calculated keys and values of uncached historical actions can be used. and The data will be appended to the cache for direct reuse in subsequent autoregressive steps.
[0047] Then, the features obtained above can be spliced together along the time dimension. ), construct the current Step in noise reduction The complete query, key, and value matrix is shown below: ; ; ; Subsequently, causal temporal attention operations are performed using the complete features obtained from the cascade, thereby obtaining the temporal features, as shown below: ; in, To maintain consistency with the causal attention mask matrix during the training phase, the dimension is... This is the feature dimension of the key vector. This mask... The temporal visibility between features is explicitly constrained, specifically by restricting uncached historical actions to attention interactions only with the current block, while allowing the current denoising target to... Access all historical actions, including both cached and uncached portions.
[0048] This application employs a collaborative mechanism of feature cascade fusion and causal masking constraints, enabling the model to encode and compute only the newly added historical action segments and the current action target to be denoised in real time at each autoregressive inference step, without repeatedly performing full feature extraction on the entire temporal sequence. During the continuous generation of long-range action trajectories, the autoregressive sliding window only slightly updates local content at the end of the time sequence, while the attention key-value features corresponding to most historical actions within the window remain unchanged and effective. Therefore, the model can directly reuse the historical key-value representations stored in the cache, incrementally calculating only the interaction attention between the features of the newly added action and the current denoising target, effectively avoiding the redundant overhead of recalculating the entire sequence round by round in traditional temporal models. This incremental update and cache reuse strategy significantly reduces the amount of floating-point operations in long-term temporal inference, effectively compresses the single-step inference latency of iterative denoising, and greatly improves the real-time performance and efficiency of continuous robot action generation.
[0049] In one embodiment of this application, the method further includes: Based on the global time offset corresponding to the current action block, determine the absolute time step of the historical action features in the current autoregressive sliding window on the global time axis; Map the absolute time step to a fixed-length position encoding table index; The input features are encoded based on the location encoding table index to obtain a historical action feature representation after injecting cyclic timing information, and the global time offset corresponding to the current action block is updated so as to execute the autoregressive step of the next action block based on the updated global time offset.
[0050] Optionally, in actual robot operation scenarios, the task execution time and the length of the complete motion trajectory are uncertain, making it impossible to predetermine the total length of the action sequence to be generated before the task starts. This makes the traditional fixed-dimensional, uniquely mapped absolute position encoding method unsuitable for long-range, variable-temporal action generation tasks. To address this issue, this system abandons the conventional design of assigning an independent and unique absolute position code to each temporal position, and adopts a cyclic position encoding application method, such as... Figure 3 The circular diagram on the left illustrates the Cyclic Temporal Embeddings structure used in this application. This mechanism eliminates the need for a preset maximum sequence length, completing feature annotation for different temporal positions through periodic cyclic reuse of encoding parameters. It can adapt to action temporal sequences of arbitrary length, perfectly meeting the needs of generating robot trajectories with unknown durations and continuous variations. At the same time, it significantly reduces the parameter redundancy and sequence length limitations caused by fixed-position encoding, ensuring the flexibility and versatility of temporal feature encoding.
[0051] Specifically, the system can maintain a fixed-size position code table with a length of P, and this set of position codes is denoted as: ; During the autoregressive inference process, a global time offset is maintained in real time. (Represents the global physical time step corresponding to the starting point of the current processing window). For the th... 1 characteristic token (of which The system first calculates its absolute time step on the global time axis. : ; Then, it can be mapped to a fixed-length position encoding table index using a modulo operation. : ; After determining the index, the positional encoding of the corresponding location is injected into the feature token: ; in, The original input features, The feature representation after injecting cyclic timing information.
[0052] When the system completes the denoising and execution of the current action block and slides into the next round of autoregressive prediction, it will determine the number of action steps actually executed in that round. Update the global time offset: ; In the next inference window, the system can continue to dynamically calculate according to the above rules based on the updated global time offset. and And the position encoding table is reused cyclically.
[0053] Understandably, this application deeply and collaboratively optimizes the cache-sharing mechanism of cyclic temporal positional encoding and causal temporal attention. For historical action key-value features (KV Cache) that have been encoded, calculated, and stored, their feature representations have already embedded the correct relative position and temporal association information learned by cyclic temporal encoding. Therefore, they can be directly read and reused in subsequent autoregressive iterations without repeatedly applying positional encoding and feature calculation. The system only applies cyclic positional encoding and performs feature mapping in real time for the action token that is most recently accessed in each sliding window, greatly reducing the computational redundancy of long sequence inference. This mechanism, relying on finite-scale, cyclically reusable encoding parameters, breaks through the inherent limitations of traditional encoding methods on sequence length, enabling stable temporal modeling of long-term robot tasks of arbitrary duration and sequence length. Compared to the traditional Transformer architecture, which is prone to extrapolation mismatch, temporal feature disorder, and prediction performance degradation due to the length of the inference sequence exceeding the perception range of the training sequence, this mechanism can continuously and stably maintain the temporal differences between local adjacent actions and the temporal correlation of global long sequences, effectively ensuring temporal consistency and action stability in the process of generating ultra-long trajectories, and significantly improving the generalization performance and robustness of the model's long-range extrapolation.
[0054] Furthermore, after the CAG module completes multiple rounds of iterative denoising optimization, it can output the future complete action sequence corresponding to the current action block, as shown in the example below. Figure 3 The prediction result is shown in the blue square at the bottom center. The system prioritizes executing the first valid action in the predicted action sequence and provides real-time feedback and records the actions actually completed in this round into the action history cache. Figure 3 The action block is marked as "executed future action". Meanwhile, to maintain a fixed-size timing window and remove invalid timing information, the system marks the oldest historical action block within the window that has no constraint value on the current task as invalid and discards it. Figure 3 The trash can icon in the image illustrates this. Therefore, the overall action history sequence will strictly follow the autoregressive progression direction to complete a comprehensive sliding update, achieving dynamic iterative refresh of the temporal window. The updated new action history cache will serve as the temporal prior input for the autoregressive inference of the next action block, continuously providing the latest temporal constraints for subsequent environment encoding, Gaussian distribution modeling, and iterative denoising prediction. The system cyclically executes the above-mentioned "prediction-execution-update-repredict" autoregressive closed-loop process, progressively generating and executing action sequences block by block until the overall robot task is completed, effectively ensuring the continuity, coherence, and iterative accuracy of action generation under long-term continuous tasks.
[0055] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.
[0056] Example 2 In one embodiment, such as Figure 1 As shown, a motion control model is provided to implement the robot motion control method as described in Embodiment 1 above. The model includes: a causal noise generator (CNG) and a causal motion generator (CAG). The causal noise generator CNG includes a noise encoder ( Figure 1 (labeled as ε) and noise decoder ( Figure 1 (marked as D in the text) The noise encoder is used to model the current environmental observation features and historical action features using a Gaussian distribution to obtain a dynamic Gaussian distribution, and to sample the dynamic Gaussian distribution to obtain initial noise; The causal action generator comprises several stacked attention modules ( Figure 1 Each attention module, marked with "×P", includes a causal temporal attention block (CTA) and a visual-motor cross attention block (VACA), used to iteratively denoise the initial noise and predict the future motion sequence corresponding to the target robot.
[0057] Specifically, the built-in noise encoder deeply extracts low-frequency dynamic features strongly correlated with the current task, fully exploring the implicit motion patterns such as task progress, speed, and acceleration changes during robot operation. This temporal dynamic information is then modeled as a Gaussian distribution that dynamically evolves with the task state, accurately outputting the mean and variance parameters corresponding to this dynamic Gaussian distribution. Based on the solved dynamic Gaussian distribution, the system generates structured initial noise with scene adaptability and physical feasibility through reparameterized sampling. This initial noise serves as the initial motion estimate input to the downstream CAG module. Figure 1 The middle pointer represents the motion estimation of the CAG module; simultaneously, the CNG module outputs the corresponding future motion prediction results through a noise decoder, serving as the motion reconstruction supervision target during the model pre-training stage, thus constraining and optimizing the model parameters. The CNG module plays a crucial role in constructing a reasonable initial motion search domain: by incorporating low-frequency motion patterns, operational trends, and dynamic characteristics accumulated in the motion history into a Gaussian distribution mean, the sampled initial noise naturally converges to a physically feasible motion range for the robot, significantly improving the rationality and effectiveness of the initial motion estimation. Furthermore, by dynamically adaptively adjusting the variance parameter to precisely control the degree of random perturbation in the sampling process, it preserves the diversity and exploration space of motion generation while ensuring that motion generation does not deviate from the task logic and motion laws. This avoids the problem of overly strong constraints leading to a single generation pattern and insufficient generalization ability, providing a high-quality, high-fault-tolerant initial benchmark for subsequent iterative denoising optimization of the CAG module.
[0058] The CTA module is located in front of each attention module and is responsible for processing the action history. Temporal context injection for current action estimation .like Figure 1 The right side of the image enlarges the Causal Temporal Attention Mask. This mask matrix uses different colored blocks to distinguish the attention range. Its overall design follows four core technical principles, constructing a high-precision temporal modeling system from four dimensions: local feature constraints, context access mechanisms, generation logic specifications, and training-inference alignment. Details are as follows: Chunk-wise Isolation: Isolates the action history The action is divided into multiple non-overlapping chunks, and each historical action is restricted to establishing attention interactions only with other actions within its own chunk, completely prohibiting cross-chunk attention association calculations. This design can effectively lock the temporal feature associations within a single chunk, ensuring the integrity and consistency of the local feature distribution of each action chunk, avoiding feature interference caused by distant and unrelated temporal information, and improving the smoothness of local action trajectories.
[0059] Causal Context Access: Current Action Estimation As an independent block, its attention mask allows access to all historical actions in the entire historical action H_t, as well as The self-estimated actions within the block. This allows the current action generation process to fully rely on all historical operation states and movement trends, achieving full historical condition constraints on future action sequences, and improving the rationality of action prediction and scenario adaptability.
[0060] Forward-looking Generation Constraint: The model can predict and generate extended action estimation sequences of a certain length in a single iteration. After completing iterative denoising optimization, the system selects only the first segment of the action sequence as the final valid output and executes it, discarding the remaining subsequent predicted segments. This strategy of looking ahead and selecting the first segment for execution can not only achieve forward-looking motion planning based on long sequence prediction, but also adapt to real-time environmental changes through segment-by-segment execution and iterative updates, effectively ensuring the temporal continuity and dynamic adaptability of long-range motion trajectories.
[0061] Inter-chunk isolation for training-inference alignment: In autoregressive inference, new actions are based on... Prediction, therefore only requires permission. focus on This is correct. Forcing attention dependencies between historical blocks will lead to a training-inference mismatch. Specifically, during training, the key-value representation of the first historical action is calculated only based on its own block, while during inference, this representation is calculated and cached in the preceding autoregressive step, involving preceding blocks (if cross-block attention is allowed). Once cross-block attention exists, it will introduce additional preceding block features, causing inconsistencies in training and inference distributions. By forcing inter-block isolation, combined with a cache sharing mechanism, the key-value cache calculation logic of the training and inference phases is ensured to be completely consistent, fundamentally eliminating the distribution shift caused by the caching mechanism and achieving a high degree of alignment between the training and inference processes.
[0062] In summary, the causal temporal attention mask of this application, through the synergistic cooperation of the above four constraints, not only ensures the temporal smoothness, local consistency and look-ahead planning capability of robot action sequence generation, but also solves the training and inference distribution offset defects that are common in traditional temporal models from the mechanism level. Combined with the cache sharing mechanism, it further improves the stability, robustness and inference efficiency of long-term continuous action generation.
[0063] The VACA module, located after the CTA module, is responsible for fusing spatial information. It uses the current observation features provided by the 2D / 3D encoder as a query and performs cross-attention calculation with the temporal features output by the CTA. This achieves cross-modal alignment between visual observations and action temporality, enabling the denoising process to dynamically adjust actions based on the spatial layout of the environment. The CTA fuses temporal constraints from action history, while the VACA injects spatial constraints from the current observations; the final output generates future actions. Throughout this process, CNG establishes a physically feasible and task-aware starting point for generation, while CAG performs fine denoising with temporal smoothing and observation alignment.
[0064] Example 3 In one embodiment, such as Figure 4 As shown, a model training method is provided for training the motion control model described in Embodiment 2. The training method includes a causal noise generator pre-training stage and a joint training stage of the causal noise generator and the causal action generator. The joint training stage includes a teacher-forced training sub-stage, a self-forced training sub-stage, and a reinforcement-forced training sub-stage.
[0065] For the pre-training phase of the causal noise generator Figure 4In the leftmost region, the system can sample batch data from the expert demonstration database, specifically including visual input (point cloud or image), historical actions, and target actions. HCDP includes two types of observation encoding implementations: a 2D encoder and a 3D encoder. Both follow the processing flow of "observation input—feature extraction—cross-modal fusion—conditional output" to encode the extracted observation data into observation features. Then, the action history and observation features are jointly encoded into a conditional Gaussian distribution that evolves over time, as shown in the following process: Let the historical action sequence be Observation encoder The output observation condition characteristics are The process involves obtaining an action token sequence by linearly mapping and encoding historical actions to their positions, and an observation token sequence by linearly projecting observation features. These two types of tokens are then concatenated in the time dimension and input into a Transformer encoder. A self-attention mechanism is used to model the temporal coupling between "historical actions" and "current observations," resulting in a joint temporal latent representation. Subsequently, the action branch latent representation undergoes length adaptation (by...). Mapped to ), to obtain the future The latent vectors corresponding to each time step are used to regress the mean and log-variance of the latent variables for each future step, forming a conditional Gaussian distribution, as shown below: ; in, , .because and All of these are calculated in real time by a temporal attention network under the current window conditions, so the distribution evolves dynamically with changes in observation and action history.
[0066] Then, reparameterization techniques can be used. Obtain latent variable noise characteristics Then... Input noise decoder, decoder with Given a target sequence as input, cross-attention decoding is performed using the projection of observed features as a memory sequence, and the output is a predicted sequence of future actions, specifically represented as follows: .
[0067] In this process, the decoding end can use linear layers and bounded activation to map the latent space back to the action space dimension. .
[0068] The encoded observation features and historical action features are input into a noise encoder, which fuses and encodes them into Gaussian distribution parameters. Predictive noise features are then sampled from these parameters and decoded by a noise decoder to obtain the predicted action. The optimization objective in this stage comprises two parts: the mean squared error loss between the predicted and target actions, and the Gaussian KL divergence loss between the learned distribution and the standard normal distribution. The former ensures that the reconstructed action closely approximates the expert demonstration, while the latter prevents the distribution from deviating excessively from the standard Gaussian distribution, maintaining generation diversity. The total loss can then be expressed as: ; The mean squared error loss can be expressed by the following formula: ; The Gaussian KL divergence loss can be expressed by the following formula: ; in, Batch size; This refers to the length of the historical observation / action window. The number of steps to be predicted. For the action dimension; For observation feature dimensions; For latent variable dimensions; For historical actions; To predict actions; For the target action; The parameters are those of a conditional Gaussian distribution. Standard Gaussian noise; This is element-wise multiplication; The weighting coefficients for the KL term (used to balance reconstruction accuracy and distribution regularization strength).
[0069] Figure 4 In the training phase, the observation encoder E, noise encoder, and noise decoder are all marked with a flame icon. After this phase, CNG has the ability to generate physically feasible and task-relevant initial noise, and the observation encoder E has also learned an effective visual representation suitable for subsequent tasks.
[0070] The corresponding sub-stage of mandatory teacher training Figure 4 The second region from the left in the middle is where the causal action generator (CAG) is trained, with the historical actions in the conditional input being a real historical sequence provided by the expert demonstration. Figure 4 The observation encoder, noise encoder, and CAG are all marked with a flame icon, indicating that they are in the training and update state.
[0071] Specifically, in this stage, the system uses the structured initial noise output by the causal noise generator as the noise source, randomly samples the noise level, randomly adds noise to the actual action through a forward diffusion process to the noise level, and obtains an intermediate noise action estimate; based on the intermediate noise action estimate, the predicted action is predicted through a denoising network, and iterative training is performed based on the loss between the predicted action and the actual action; First, the CAG receives the structured initial noise generated by the CNG. Observation encoder Extracted current observation features and real historical action sequences Unlike standard diffusion strategies, CAG achieves strict alignment between the initial noise for training and inference. That is, the starting point for denoising during training is not standard Gaussian noise, but rather directly uses the physically prior noise output by the CNG. .
[0072] CAG can be implemented using a single-step denoising network. Directly predict clean future actions For any noise level of random sampling The network performs the following mapping: ; in For the first Noise motion estimation of steps ( ).network Depend on It consists of stacked attention modules, each module containing two sub-layers: Causal Temporal Attention (CTA) and Visual-Motion Cross-Attention (VACA), as described below: Causal temporal attention (CTA) is used to process historical actions. Compared with current motion estimation As input, through causal temporal attention mask constraints, self-attention within historical action blocks and causal access to all historical actions by the current estimate are achieved. Specifically, let... Given a query, key, and value matrix as input features, the output of CTA can be represented as: ; in, Scaling factor This is a causal temporal attention mask matrix. This mask ensures that: historical actions can only focus on other historical actions within the same data block; action estimation... You can follow all historical actions. It also includes actions within its own blocks; strict attention isolation is implemented between historical blocks to cooperate with the cache sharing mechanism during inference, eliminating the distribution offset between training and inference.
[0073] Visual-Motion Cross-Attention (VACA) is used to use the temporal features of the CTA output as a query. ), observation features As keys and values ( Spatial context is fused through cross-attention, as detailed below: ; By injecting visuospatial information (such as object pose and scene geometry) into temporal features, it is ensured that the final generated motion trajectory conforms to the current physical environment state.
[0074] During the training phase, a noise level k can be randomly sampled, and real actions demonstrated by experts can be used. Noise is added through the forward diffusion process. Subsequently, starting from this intermediate noisy state, the network directly predicts the clean action in a single step. CAG employs mean squared error (MSE) loss for supervised learning, with the optimization objective being to minimize the difference between the predicted action and the actual action, as detailed below: ; The model uses formulas in subsequent inference stages. Perform iterative denoising sampling.
[0075] The core objective of this stage (the first stage of mandatory training in the course) is to establish a stable and baseline policy prior π0 for the system, enabling the model to converge to the expert data distribution under real-world historical conditions. ; Because the observation encoder E participates in end-to-end joint training at this stage, the visual feature representation can adaptively extract spatial patterns (such as edge attributes of manipulated objects and relative positions of contact points) that are most strongly correlated with the current action generation task. Simultaneously, it incorporates real historical actions... This mandatory condition ensures that the model first masters the standard task execution mode, laying the foundation for model self-correction training when introducing self-generated historical data in the subsequent Curriculum-Forcing learning phase.
[0076] The self-forced training sub-stage corresponds to Figure 4 In the third region from the left in the middle, at this stage, the training data no longer relies entirely on the static trajectories demonstrated by experts, but introduces the action sequences generated by the model itself as historical conditions, realizing a gradual transition from "teacher-mandated" to "self-mandated".
[0077] Specifically, this stage is used to predict future actions block by block in an autoregressive manner, and to use the future action corresponding to the current action block as the historical action feature of the next action block and perform iteration. The iteration is performed for multiple training rounds, and the action sequence fragments with intrinsic bias generated in each iteration process are stored in the intrinsic data pool.
[0078] First, in each training epoch... Initially, the system is based on the current strategy model. Perform autoregressive trajectory expansion. For the initial observations of a given sample... The strategy first generates the first action block prediction under the condition of no history (or filled history), which can be specifically expressed as: ; Subsequently, the system simulates the chain feedback mechanism in real reasoning processes, and generates self-generated action blocks. The historical features (corresponding to the pink arrow path in the diagram) are re-inputted into the network to generate subsequent predictions. For the first... Each time step, its input historical actions It is composed of a series of action blocks generated by all previous autoregressive steps in this round, as shown below: ; At this point, the output mapping relationship of the strategy can be expressed as: ; The process iterates until the maximum step size is reached. This forms a complete self-generated motion trajectory.
[0079] It should be noted that, Figure 4 The green dashed arrows indicate the flow of endogenous samples to the endogenous data pool. Action sequence fragments generated in each round, containing endogenous bias, are directly stored in the endogenous data pool (sample pool). Specifically, the stored data is in the form of tuples of "observation-self-generated history-ground action". This data pool is dynamically accumulated and updated across multiple training rounds, as shown below: ; As training progresses, the early rounds ( History of smaller strategy generation This includes significant deviations from fluid movements and physical feasibility; while later rounds ( The magnitude of historical biases generated (which are relatively large) decreases, tending towards subtle structured drifts. This accumulation of data across rounds makes the internal data pool... It automatically covers the complete bias spectrum, ranging from "significant deviations" to "minor temporal drifts." Simultaneously, this stage utilizes mean squared error loss supervision to enable the policy to learn and correct its own generated biases.
[0080] The enhanced forced training sub-stage, corresponding to the rightmost region in the diagram, is used to extract historical action features from the internal data pool and the expert demonstration data pool according to a uniform distribution sampling rule. Iterative training is then performed based on the extracted historical action features, current observation features, and real actions. This stage first utilizes the internal data pool: this pool stores self-generated action histories spanning multiple training rounds. These histories are generated by strategies at different convergence stages and possess unique error characteristics. Figure 4 The robot icons of various colors in the internal data pool represent the diverse action history generated by different rounds of strategies. These histories contain rich error patterns, including large unstructured noise in the early rounds, temporal structured bias in the middle rounds, and subtle parameter drift in the later rounds.
[0081] Specifically, let the batch size be... For each sample Its historical conditions The selection is based on a uniform distribution strategy from the union of these two types of data sources, as shown below: ; in, It is an internal data pool containing historical data generated in multiple rounds and with biases; Provides experts with data that offers an absolutely pure history of truth values.
[0082] This dual-source sampling mechanism constructs a training paradigm similar to experience replay, which introduces differentiated temporal sample constraints during model training. This allows the final policy to simultaneously consider the characteristics of real physical interaction errors and the standard expert execution paradigm, achieving a two-way improvement in robustness and accuracy. Specifically, the model training process can be simultaneously exposed to two types of core temporal sample distributions: the first is the composite error of self-generated history, which includes physical interaction cumulative errors such as perceptual bias caused by sensor noise, motion distortion caused by actuator aging and slippage, and inverse kinematic numerical errors. The second is the ideal execution mode of expert history: maintaining the model's accurate regression ability to the standard task execution trajectory and preventing the policy from overfitting noisy patterns to cater to biased history.
[0083] Under dual-source conditional input, the causal action generator CAG performs a denoised forward propagation constrained by a biased history. For any sampled conditional history... The training process first uses forward diffusion to obtain expert truth values for actions. Add noise to a randomly sampled noise level. The intermediate noise action estimate is obtained. Subsequently, a denoising network was used. With this Assuming time-series constraints, perform single-step prediction as follows: ; This process forces CAG to have varying degrees of contamination in its historical inputs as a "condition" (when they come from...). ) or perfectly ideal (when it comes from In scenarios involving high noise, the system can accurately recover physically feasible and task-correct future actions from a noisy state. The supervisory signal uniformly adopts expert truth actions. End-to-end optimization is performed using the mean squared error (MSE) loss function, which is shown below: ; This loss function ensures that regardless of the conditional history... Predict the output based on the given state. All are rigidly constrained to converge towards the expert truth. This enables the policy to learn a robust mapping that "recovers the standard execution trajectory from historical biases".
[0084] The core purpose of this training phase is to allow the action generation strategy to fully engage with and adapt to various complex error patterns in real-world robot physical interaction scenarios, thereby improving the strategy's resilience and generalization performance under real-world conditions. Unlike the self-supervised constraint mechanisms of traditional video generation tasks, robot temporal action generation involves an internal bias accumulation effect, resulting in a unique triple-complexity error system. First, there is distortion in physical interaction, leading to model-predicted actions... After a chain conversion of inverse kinematics, controller, and actuator, the actual action is executed. ,in The problems include: First, nondeterministic transmission errors, sensor noise, and unmodeled environmental disturbances. Second, multimodal policy collapse occurs. During long-range autoregressive iterative development, the action policy is prone to irregular and unexpected transitions between multiple feasible modes in a Gaussian distribution, disrupting the temporal correlation logic of continuous action sequences, causing historical trajectory temporal breaks and motion feature disorder, ultimately leading to policy generation failure. Third, the training data manifold has inherent fragility. Limited by the number of high-quality expert demonstration samples in real-world scenarios, the distribution coverage of the policy data learned by the model is limited. In unknown scenarios outside the distribution, deviations are prone to rapid accumulation and continuous amplification of errors, resulting in a sharp decline in the accuracy and stability of action generation. The superposition of these multiple intrinsic biases makes the error characteristics of robot action generation tasks far more complex than those of traditional vision generation tasks. Therefore, it is urgent to rely on a dual-source sampling mechanism to achieve robust error learning and modal stability constraints.
[0085] By covering a rich set of instances of the aforementioned multi-source perturbations in the internal data pool, a robust strategy is ultimately output. We have learned a powerful ability to generalize and correct perturbation spectra in external open environments, as detailed below: ; in This represents any historical conditions that contain various composite errors in real physical interactions. Figure 4 The training cycle axis in the lower middle section runs through the self-forced and reinforcement-forced stages, indicating that this is a course-based learning process that deepens gradually over time: from relying on expert history to establish stable priors, to introducing self-generated history to perceive intrinsic biases, and finally using dual-source conditional history to achieve robustness in open physical environments.
[0086] Example 4 In one embodiment, a robot motion control device is provided, which corresponds one-to-one with the robot motion control method in Embodiment 1 above. For example... Figure 5 As shown, the robot motion control device includes a feature encoding unit 10, a current motion estimation value acquisition unit 20, a future motion prediction unit 30, and a motion execution unit 40. Detailed descriptions of each functional module are as follows: The feature encoding unit 10 is used to acquire the current environmental observation data and historical action data of the target robot, and to perform feature encoding on them respectively to obtain the current environmental observation features and historical action features. The current action estimation value acquisition unit 20 is used to perform Gaussian distribution modeling on the current environmental observation features and historical action features to obtain a dynamic Gaussian distribution, and to sample the dynamic Gaussian distribution to obtain initial noise, and use the initial noise as the current action estimation value; The future action prediction unit 30 is used to predict the future action result of the target robot by iterative denoising based on the current environmental observation features, historical action features and current action estimates. The action execution unit 40 is used to control the target robot to perform corresponding action operations based on the future action result.
[0087] In one embodiment of this application, the current action estimation value acquisition unit 20 is further configured to: Based on the current environmental observation characteristics and historical action characteristics, the mean and variance parameters of the Gaussian distribution are predicted. Based on the mean and variance parameters of the Gaussian distribution, construct the dynamic Gaussian distribution corresponding to the current moment; The initial noise is generated by probabilistically sampling the dynamic Gaussian distribution corresponding to the current time using the Gaussian reparameterization technique.
[0088] In one embodiment of this application, the future action prediction unit 30 is further configured to: The block autoregression method is used to perform back diffusion loop iteration denoising based on the corresponding environmental observation features, historical action features and action estimates in the current action block according to the preset number of iterations, and to predict the future action result corresponding to the current action block. The historical action features are updated based on the target robot's action execution result for the future action result corresponding to the current action block, and the prediction of the next action block is performed based on the updated historical action features.
[0089] In one embodiment of this application, the future action prediction unit 30 is further configured to: In each iteration, based on the action estimate, noise level, current environmental observation features, and historical action features corresponding to the current iteration, the pure action estimate result corresponding to the current iteration is predicted; Based on preset noise scheduling parameters, noise scheduling coefficients and random Gaussian noise, the action estimate value corresponding to the next iteration round is calculated, and the single iteration update is completed. After completing the noise reduction process for all iterations, the final result of the future action corresponding to the current action block is obtained.
[0090] In one embodiment of this application, the future action prediction unit 30 is further configured to: apply causal temporal constraints to the historical action features and action estimates corresponding to the current iteration round, and extract temporal features; The temporal features are fused with the current environmental observation features through cross-attention calculation to obtain the global fused features; Based on the global fusion features and the noise level corresponding to the current round, the pure action estimation result corresponding to the current iteration round is predicted.
[0091] In one embodiment of this application, the future action prediction unit 30 is further configured to: divide the historical actions corresponding to the current action block into cached historical actions and uncached historical actions; Read the key and value characteristics corresponding to the cached historical actions; Linear transformations are performed on the action estimates corresponding to the uncached historical actions and the current action block, respectively. The query features, key features, and value features corresponding to the uncached historical actions, as well as the query features, key features, and value features corresponding to the action estimates of the current action block, are extracted. Based on causal temporal attention mask, key-value concatenation and causal attention calculation are performed on the key features and value features corresponding to cached historical actions, the query features, key features and value features corresponding to uncached historical actions, and the query features, key features and value features of the action estimate corresponding to the current action block, to generate the temporal features corresponding to the current action block; The key and value features corresponding to newly added uncached historical actions will be cached for the next action block to perform the autoregressive prediction step.
[0092] In one embodiment of this application, the device further includes a cyclic position encoding unit, used for: Based on the global time offset corresponding to the current action block, determine the absolute time step of the historical action features in the current autoregressive sliding window on the global time axis; Map the absolute time step to a fixed-length position encoding table index; The input features are encoded based on the location encoding table index to obtain a historical action feature representation after injecting cyclic timing information, and the global time offset corresponding to the current action block is updated so as to execute the autoregressive step of the next action block based on the updated global time offset.
[0093] In this embodiment, historical action feature context information combined with environmental observation features is fully utilized to complete action prediction. This can accurately identify temporal confusion states, accurately infer task progress, and ensure the consistency of long-term operation action timing. By using dynamic Gaussian distribution sampling and iterative denoising mechanism, the physical feasibility of actions and the diversity of environmental adaptation generation are taken into account, effectively improving the robot's motion decision accuracy, environmental generalization ability, and overall robustness of autonomous operation under complex working conditions.
[0094] Specific limitations regarding the robot motion control device can be found in the limitations of the robot motion control method described above, and will not be repeated here. Each module in the aforementioned robot motion control device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.
[0095] In one embodiment, a computer device is provided, which may be a terminal device, and its internal structure diagram may be as follows: Figure 6 As shown, the computer device includes a processor, memory, and network interface connected via a system bus. The processor provides computational and control capabilities. The memory includes a readable storage medium storing computer-readable instructions. The network interface communicates with external terminals via a network connection. When executed by the processor, the computer-readable instructions implement a robot motion control method. The readable storage medium provided in this embodiment includes both non-volatile and volatile readable storage media.
[0096] In this application embodiment, a computer device is provided, including a memory, a processor, and computer-readable instructions stored in the memory and executable on the processor. When the processor executes the computer-readable instructions, it implements the steps of the robot motion control method described above.
[0097] In this embodiment of the application, a readable storage medium is provided, which stores computer-readable instructions. When the computer-readable instructions are executed by a processor, they implement the steps of the robot motion control method described above.
[0098] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by instructing related hardware with computer-readable instructions. These computer-readable instructions can be stored in a non-volatile readable storage medium or a volatile readable storage medium. When executed, these computer-readable instructions can include the processes of the embodiments of the methods described above. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.
[0099] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.
[0100] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.
Claims
1. A robot vision motion control method, characterized in that, The method includes: Acquire the target robot's current environmental observation data and historical action data, and perform feature encoding on them respectively to obtain the current environmental observation features and historical action features; Gaussian distribution modeling is performed on the current environmental observation features and historical action features to obtain a dynamic Gaussian distribution, and the dynamic Gaussian distribution is sampled to obtain initial noise, which is then used as the current action estimate. Based on the current environmental observation features, historical action features, and current action estimates, the future action results of the target robot are obtained through iterative noise reduction prediction. Based on the predicted future action results, the target robot is controlled to perform corresponding action operations.
2. The robot motion control method as described in claim 1, characterized in that, The process of modeling the current environmental observation features and historical action features using a Gaussian distribution to obtain a dynamic Gaussian distribution, and then sampling the dynamic Gaussian distribution to obtain initial noise, includes: Based on the current environmental observation characteristics and historical action characteristics, the mean and variance parameters of the Gaussian distribution are predicted. Based on the mean and variance parameters of the Gaussian distribution, construct the dynamic Gaussian distribution corresponding to the current moment; The initial noise is generated by probabilistically sampling the dynamic Gaussian distribution corresponding to the current time using the Gaussian reparameterization technique.
3. The robot motion control method as described in claim 1, characterized in that, The process of predicting the future action of the target robot through iterative denoising based on the current environmental observation features, historical action features, and current action estimates includes: The block autoregression method is used to perform back diffusion loop iteration denoising based on the corresponding environmental observation features, historical action features and action estimates in the current action block according to the preset number of iterations, and to predict the future action result corresponding to the current action block. The historical action features are updated based on the target robot's action execution result for the future action result corresponding to the current action block, and the prediction of the next action block is performed based on the updated historical action features.
4. The robot motion control method as described in claim 3, characterized in that, The process of performing back-diffusion loop iterative denoising based on corresponding environmental observation features, historical action features, and action estimates according to a preset number of iterations, and predicting the future action result corresponding to the current action block, includes: In each iteration, based on the action estimate, noise level, current environmental observation features, and historical action features corresponding to the current iteration, the pure action estimate result corresponding to the current iteration is predicted; Based on preset noise scheduling parameters, noise scheduling coefficients and random Gaussian noise, the action estimate value corresponding to the next iteration round is calculated, and the single iteration update is completed. After completing the noise reduction process for all iterations, the final result of the future action corresponding to the current action block is obtained.
5. The robot motion control method as described in claim 4, characterized in that, The process of predicting the clean action estimation result corresponding to the current iteration round based on the action estimation value, noise level, current environmental observation characteristics, and historical action characteristics includes: By applying causal temporal constraints to the historical action features and action estimates corresponding to the current iteration round, temporal features are extracted. The temporal features are fused with the current environmental observation features through cross-attention calculation to obtain the global fused features; Based on the global fusion features and the noise level corresponding to the current round, the pure action estimation result corresponding to the current iteration round is predicted.
6. The robot motion control method as described in claim 5, characterized in that, The step of applying causal temporal constraints to the historical action features and action estimates corresponding to the current iteration round to extract temporal features includes: Divide the historical actions corresponding to the current action block into cached historical actions and uncached historical actions; Read the key and value characteristics corresponding to the cached historical actions; Linear transformations are performed on the action estimates corresponding to the uncached historical actions and the current action block, respectively. The query features, key features, and value features corresponding to the uncached historical actions, as well as the query features, key features, and value features corresponding to the action estimates of the current action block, are extracted. Based on causal temporal attention mask, key-value concatenation and causal attention calculation are performed on the key features and value features corresponding to cached historical actions, the query features, key features and value features corresponding to uncached historical actions, and the query features, key features and value features of the action estimate corresponding to the current action block, to generate the temporal features corresponding to the current action block; The key and value features corresponding to newly added uncached historical actions will be cached for the next action block to perform the autoregressive prediction step.
7. The robot motion control method as described in claim 1, characterized in that, The method further includes: Based on the global time offset corresponding to the current action block, determine the absolute time step of the historical action features in the current autoregressive sliding window on the global time axis; Map the absolute time step to a fixed-length position encoding table index; The input features are encoded based on the location encoding table index to obtain a historical action feature representation after injecting cyclic timing information, and the global time offset corresponding to the current action block is updated so as to execute the autoregressive step of the next action block based on the updated global time offset.
8. A motion control model, characterized in that, For implementing the robot motion control method as described in any one of claims 1-7, the model includes: a causal noise generator and a causal motion generator; The causal noise generator includes a noise encoder and a noise decoder. The noise encoder is used to model the current environmental observation features and historical action features using a Gaussian distribution to obtain a dynamic Gaussian distribution, and to sample the dynamic Gaussian distribution to obtain initial noise. The causal motion generator includes several stacked attention modules, each of which includes a causal temporal attention block and a visual-motion cross-attention block, used to iteratively denoise the initial noise and predict the future motion sequence corresponding to the target robot.
9. A training method for a motion control model, characterized in that, The method for training the motion control model of claim 8 includes a causal noise generator pre-training stage and a joint training stage of the causal noise generator and the causal motion generator; wherein the joint training stage includes a teacher-forced training sub-stage, a self-forced training sub-stage, and a reinforcement-forced training sub-stage.
10. The training method for the causal diffusion strategy model as described in claim 9, characterized in that, The teacher-forced training sub-stage is used to take the structured initial noise output by the causal noise generator as the noise source, randomly sample the noise level, add noise to the real action through a forward diffusion process to the noise level, obtain the intermediate noise action estimate, obtain the predicted action based on the intermediate noise action estimate, and perform iterative training based on the loss between the predicted action and the real action. The self-forced training sub-stage is used to predict future actions block by block in an autoregressive manner, and to use the future action corresponding to the current action block as the historical action feature of the next action block and perform iteration. The iteration is performed for multiple training rounds, and the action sequence fragments with intrinsic bias generated in each iteration process are stored in the intrinsic data pool. The enhanced forced training sub-stage is used to extract historical action features from the internal data pool and the expert demonstration data pool according to the uniform distribution sampling rule, and to perform iterative training based on the extracted historical action features and real actions.
