A method for training a visual-language-action model of a humanoid robot combined with spatial information enhancement

Abstract

The present application relates to the field of humanoid robot training, in particular to a kind of visual-linguistic-motion model training method of humanoid robot enhanced by combining space information.The method mainly includes time step action label preprocessing, before training, the feature distribution of action sequence on each independent time step in the trajectory training set of humanoid robot is counted;Align the space semantic representation information extracted by DUST3R through VLA intermediate layer output;End-to-end training combined with alignment loss;After the completion of model training, the soft action repair algorithm based on timing correlation propagation and dynamic relaxation is executed by removing the 3D base model path in the online real-time inference stage of humanoid robot deployment.The present application aims to improve the control accuracy of fine operation action without changing the original 2D network architecture of humanoid robot, and is suitable for visual-linguistic-motion model training of humanoid robot.

Description

Technical Field

[0001] This invention relates to the field of humanoid robot training, and more specifically to a training method for humanoid robot vision-language-action models that incorporates spatial information enhancement. Background Technology

[0002] Traditional VLA models have the following core technical shortcomings when humanoid robots perform high-precision interactive tasks:

[0003] Lack of accurate 3D spatial awareness and geometric depth perception: The visual backbone network of the VLA model is only pre-trained on 2D image datasets. The visual marker features it extracts completely lack an understanding of the 3D geometry and relative distances of the physical world, which can easily lead to the robotic arm dangling in mid-air or experiencing physical collisions during grasping. For example, in actual robot operations, it is common for the robot to move above the target object but fail to grasp it due to a lack of depth information.

[0004] Limitations of Action Normalization Strategies: Existing VLA models typically employ a global statistics-based normalization approach when processing continuous action sequences. This means that the mean and standard deviation of actions at all time steps are calculated uniformly across the entire training set, and the same set of normalization parameters is used to standardize the actions of all timesteps within an action chunk. In other words, timesteps at the same position in different action chunks (such as step 0, step 8, or step 16) do not have their action distribution modeled separately but share global statistics with all other timesteps. However, this global normalization approach implicitly assumes that the actions at all timesteps follow a similar data distribution. But in real-world robot manipulation tasks, action sequences exhibit significant stage-based and temporal heterogeneity, and the action distributions at different timesteps often differ significantly. Under global normalization, the statistics are dominated by timesteps with larger action amplitudes. For example, large actions in the early stages significantly increase the overall standard deviation, causing the smaller, finer control actions in the later stages to be further compressed after normalization, reducing their dynamic range. This leads to a decrease in the model's prediction accuracy for actions.

[0005] Discontinuity between action blocks and joint mechanical jitter: To enable robots to achieve real-time closed-loop adjustment capabilities, existing general-purpose VLA models typically employ rolling window control or action block strategies during deployment. This means that each forward inference by the model predicts a sequence of actions, the control system only executes the first part of the time step, and then triggers the next forward inference to incorporate the latest visual feedback. However, in this traditional rolling control mechanism, the forward inference processes of two adjacent control cycles are completely isolated on the time axis.

[0006] On the one hand, traditional diffusion models or flow matching networks use completely random, independent, and identically distributed Gaussian white noise for state initialization at the beginning of each control cycle, resulting in a lack of underlying temporal correlation between the generated trajectories of different cycles. On the other hand, traditional motion flow generation lacks effective boundary history constraints and cross-temporal error propagation mechanisms. This leads to a statistically significant "boundary seam discontinuity" between the control endpoint of the previous cycle and the control start point of the next cycle at the instant of sliding window switching. Summary of the Invention

[0007] The purpose of this invention is to overcome the shortcomings of the prior art and provide a training method for humanoid robot vision-language-action models that combines spatial information enhancement, thereby improving the control accuracy of fine manipulation actions without changing the original 2D network architecture of the humanoid robot.

[0008] The present invention achieves the above objectives by adopting the following technical solution: The present invention provides a method for training a humanoid robot's vision-language-action model that combines spatial information enhancement, comprising the following steps:

[0009] S1. Time-step action label preprocessing;

[0010] Before training begins, the feature distribution of the action sequences in the humanoid robot trajectory training set is statistically analyzed at each independent time step. For each specific time step t∈[0,H-1] within the prediction window H, its independent mean vector is calculated. and standard deviation vector Where D is the action dimension; in each forward propagation of training, for the real action label sequence Normalization is performed on a time-step basis, with each step isolated from the others.

[0011] ;

[0012] S2. Align the spatial semantic representation information extracted by DUST3R with the output of the VLA intermediate layer to enhance spatial perception capabilities;

[0013] Acquire multi-view RGB images of the humanoid robot's current environment and the corresponding natural language operation text commands;

[0014] Input image I into the VLA backbone network to extract visual label representation features of specified hidden intermediate layers. ;

[0015] Input the same image I into a pre-trained 3D dense geometry perception network Output the three-dimensional latent geometric coordinate representation corresponding to the pixel position of each visual marker. ;

[0016] exist The spatially supervised target features are constructed by adding an accumulated position embedding vector E. ;

[0017] Through batch normalization layer Γ and a nonlinear perceptron containing two layers of linear mapping, Perform dimensional projection to obtain the mapped feature vector. ;

[0018] Constructing a 3D information alignment loss function using the cosine similarity metric function :

[0019] ;

[0020] Where N is the number of visual markers. This is the cosine similarity metric function;

[0021] S3. End-to-end training incorporating alignment loss;

[0022] The time-step normalized action sequence obtained in step S1 Diffusion noise is applied to obtain a noisy noise-added action;

[0023] The noise level, the denoised time step, and the current embodied state features of the humanoid robot are all input into the diffusion Transformer network;

[0024] Predict the current motion velocity field and calculate the mean square error loss with the original motion noise. ;

[0025] Combination and Construct the overall objective loss function End-to-end joint gradient optimization training is performed on the network parameters;

[0026] S4. Soft action repair control during the reasoning execution period;

[0027] After the humanoid robot is trained, during the deployment and online real-time inference phase, the 3D basic model path is removed, and a soft motion repair algorithm based on temporal correlation propagation and dynamic relaxation is executed.

[0028] Furthermore, in step S2, the pre-trained 3D dense geometry perception network The DUST3R architecture network is adopted, which integrates multi-view context by utilizing its internal cross-attention mechanism, and directly outputs a geometric representation containing implicit depth and local 3D topological structure information.

[0029] Furthermore, in step S4, the soft action repair algorithm based on temporal correlation propagation and dynamic relaxation specifically includes the following steps:

[0030] Timing scrolling window definition: Set the total length of the timing window for actions predicted in a single forward inference by the VLA model to be... The first step, within the current control cycle, is before the system actually issues and executes the commands. The step action is to take the last remaining part of the current prediction sequence. The step is saved as a historical overlapping context as the initial action vector for the next control cycle. In the forward denoising of the next control cycle, the known constraint region O region is defined, and the remaining unknown prediction part is defined as the free denoising region U region.

[0031] Noise initialization: During the forward denoising initialization phase of the diffusion model in the current control cycle, the full-time domain multivariate correlation noise z of this cycle is generated, and the fixed noise components corresponding to the O region are extracted and recorded. Simultaneously, retrieve the constraint action target value cached from the previous cycle. ;

[0032] Dynamic soft-fix evolution guided by flow matching path: In the iterative process of reverse denoising evolution of flow matching from time step t=1.0 to t=0.0, each evolution step Perform action repair and error propagation with dynamic relaxation features through conditional branching;

[0033] Action sequence denormalization and servo delivery: After evolving to t=0, extract the previous... The step sequence is recovered by inverse normalization using the time-step mean and standard deviation saved in step S1, and then sent to the robot joint motors for execution.

[0034] Furthermore, in the dynamic soft repair evolution guided by the flow matching path, when the denoising time step... Execution of strong constraints and correlation propagation phases:

[0035] Calculate the theoretical expected trajectory value of region O at the current evolution step based on the linear boundary condition property of flow matching. ;

[0036] Extract the original hidden state values ​​of region O predicted by the current network. Calculate the hard correction difference:

[0037] ;

[0038] Utilizing the statistical properties of the conditional Gaussian distribution, the correction amount is smoothly propagated to region U using a pre-calculated time-series covariance matrix, and the adaptive correction compensation amount for region U is calculated:

[0039] ;

[0040] in This represents the time-series autocorrelation matrix within region O. This represents the temporal cross-correlation matrix between region U and region O;

[0041] The correction compensation is superimposed on the free region to achieve coordinated convergence of motion trajectories across the entire time domain, and the O region is forcibly aligned to the flow matching standard trajectory. .

[0042] Furthermore, the strong constraint and correlation propagation phase also includes a dynamic safety valve mechanism: calculation Maximum absolute value If the value is less than or equal to the set safety threshold, the correction compensation will be added to the free region. If the threshold is exceeded, the U region remains unchanged.

[0043] Furthermore, in the dynamic soft repair evolution guided by the flow matching path, when the denoising time step... The dynamic relaxation and fine-tuning phase is executed: the rigid orbital constraints and correlation propagation mechanisms of the O region are completely removed, all degrees of freedom of the model are released, and the model is allowed to combine the latest visual environment features and ontological state to perform joint smoothing and fine-tuning of action points in the entire time domain, including the O and U regions.

[0044] Furthermore, in the aforementioned time-series scrolling window, =30 steps =26 steps, O region index is 0~3, U region index is 4~29.

[0045] Furthermore, in step S3, the diffusion Transformer network is trained using a flow matching paradigm to predict the current motion velocity field and calculate the mean square error loss with the original motion noise.

[0046] Furthermore, in step S4, while removing the 3D base model path during the inference execution period, the mean vector of each time step in step S1 is retained. and standard deviation vector Used for inverse normalization of action sequences.

[0047] The beneficial effects of this invention are as follows:

[0048] This invention overcomes the limitations of traditional embodied intelligent VLA models, which suffer from independent reasoning in each control cycle and lack temporal continuity constraints during time window scrolling. This invention innovatively proposes a soft action repair mechanism. In the forward denoising of flow matching, this mechanism uses the final action of the previous control cycle as a priori and rigidly embeds it into the starting segment of the current cycle. (Region); more importantly, by utilizing the mathematical properties of covariance in the conditional Gaussian distribution ( The abrupt disturbances caused by hard corrections are smoothly propagated to the unknown future action domain according to their temporal correlation ratio. The region fundamentally eliminates the seam gaps at the chunk splicing points; finally, with the dynamic relaxation of the 30% time step at the end of the denoising process, the model retains the continuity of historical time sequence and has the dynamic compliance to be finely adjusted online based on real-time visual feedback, thus solving the mechanical jitter and trajectory inconsistency phenomena in high-frequency robot control.

[0049] This invention breaks through the limitations of traditional global normalization, customizing independent normalization for each time step within an action chunk. Mean and standard deviation matrices. Spatial isolation normalization is implemented in both training and inference, which greatly improves the signal-to-noise ratio in the fine-grained interaction phase.

[0050] During the training period, this invention uses a spatial information enhancement module to bring the specified intermediate layer features of 2D VLA closer to the geometric representation of the pre-trained 3D dense geometry perception network (DUST3R), enabling the model to have 3D geometry and depth perception capabilities. Attached Figure Description

[0051] Figure 1 This is a flowchart of a method for training a humanoid robot's vision-language-action model that combines spatial information enhancement, provided by an embodiment of the present invention.

[0052] Figure 2 This is a flowchart of the training phase provided in an embodiment of the present invention;

[0053] Figure 3 This is a flowchart of the reasoning stage provided in an embodiment of the present invention. Detailed Implementation

[0054] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings.

[0055] The purpose of this invention is to provide a training method for humanoid robot vision-language-action models that combines spatial information enhancement, which can enhance the model's three-dimensional spatial understanding and execution efficiency without relying on explicit three-dimensional input.

[0056] This invention endows the model with 3D geometric perception by introducing a spatial information enhancement module during the training phase, and simultaneously introduces a time-step normalization mechanism to accurately capture the unique feature distribution of the task at different temporal stages; furthermore, during the inference and execution phase, it introduces soft motion repair technology to solve the problem of discontinuous motion. This invention aims to improve the control accuracy of fine manipulation actions without altering the native 2D network architecture of the humanoid robot.

[0057] The overall system architecture of this invention mainly consists of the VLA backbone model Gr00T-N1.7, the pre-trained 3D base model DUST3R feature extraction module, and the action temporal normalization processing module.

[0058] refer to Figures 1 to 3 The specific steps of the humanoid robot vision-language-action model training method combining spatial information enhancement provided by this invention are as follows:

[0059] S1. Time-step action label preprocessing;

[0060] Before training begins, the feature distribution of the action sequences in the humanoid robot trajectory training set is statistically analyzed at each independent time step. For each specific time step t∈[0,H-1] within the prediction window H, its independent mean vector is calculated. and standard deviation vector , where D is the action dimension;

[0061] In each forward propagation during training, for the real action label sequence Without using global mixture statistics, normalization is performed on a time-step basis to isolate each other:

[0062] ;

[0063] This mechanism ensures that each time-series label can be scaled independently and accurately, avoiding feature contamination.

[0064] S2. Align the spatial semantic representation information extracted by DUST3R through the output of the VLA intermediate layer;

[0065] During the training phase, multi-view RGB images of the humanoid robot's current environment and corresponding natural language operation text commands are acquired.

[0066] VLA Feature Extraction: Extracting features from an image Transform into A pixel-by-pixel sequence of visual labels is input into the multimodal backbone network. During the forward propagation, the visual label representation features corresponding to the hidden intermediate layers (e.g., layer 10) specified by the VLM are extracted. .

[0067] 3D spatial geometric feature extraction: extracting the same image Input into a pre-trained 3D dense geometry perception network (In the DUST3R architecture network). This network utilizes its internal cross-attention mechanism to integrate multi-view context, directly outputting a 3D latent geometric coordinate representation that precisely corresponds to the position of each visual marker pixel. (Includes implicit depth and local 3D topology information).

[0068] Location embedding fusion and feature dimension projection preprocessing: To ensure that the external 3D supervision signal maintains the spatial order of the physical world in subsequent self-attention interactions, the output spatial latent geometric representation is preprocessed. An additional position embedding vector is added on top. Construct spatial monitoring target features .

[0069] Meanwhile, the spatial information enhancement module uses a batch normalization layer ( (and a nonlinear perceptron containing two layers of linear mapping for the intermediate visual features of VLA) Perform dimensional projection to map it to a feature space consistent with the 3D supervised features, and obtain the mapped feature vector. .

[0070] Calculate 3D information alignment loss: using the cosine similarity metric function Construct a 3D information alignment loss function :

[0071] ;

[0072] Where N is the number of visual markers. This is the cosine similarity metric function;

[0073] S3. End-to-end training incorporating alignment loss;

[0074] The time-step normalized action sequence obtained in step S1 Diffusion noise is applied to obtain a noisy noise-added action;

[0075] The noise level, the denoised time step, and the current embodied state features of the humanoid robot are all input into the diffusion Transformer network;

[0076] Predict the current motion velocity field and calculate the mean square error loss with the original motion noise. ;

[0077] Combination and Construct the overall objective loss function End-to-end joint gradient optimization training is performed on the network parameters;

[0078] S4. Soft action repair control during the reasoning execution period;

[0079] After model training is complete, the 3D base model path is removed during the humanoid robot deployment and online real-time inference phase. To eliminate the discontinuity in motion sequence transitions and joint mechanical jitter caused by independent inference in each control cycle, this invention introduces a soft motion repair algorithm based on temporal correlation propagation and dynamic relaxation. This algorithm pre-defines multivariate initial noise with temporal correlation characteristics (its temporal cross-correlation matrix...). This is carried out under the premise of […]. Its core control process is as follows:

[0080] Timing scrolling window definition: Set the total length of the timing window for actions predicted in a single forward inference by the VLA model to be... Step. Within the current control cycle, the system only actually issues and executes the preceding steps. The last four actions in the current prediction sequence are used as historical overlap context and saved as the initial action vector for the next control cycle. It is defined as the known constraint region (denoted as ) in the forward denoising of the next control cycle. Region, index is The remaining unknown prediction portion is defined as the free denoising region (denoted as...). Region, index is ).

[0081] Noise initialization: In the forward denoising initialization phase of the diffusion model in the current control cycle, full-time domain multivariate correlation noise is first generated. (in (For the action dimension). Extract and record. Fixed noise components corresponding to the region Simultaneously, retrieve the constraint action target value cached from the previous cycle. .

[0082] Dynamic soft repair evolution guided by flow matching path: In flow matching from time step Towards In the iterative process of reverse denoising evolution, each evolution step All actions, including motion repair and error propagation, are performed using conditional branches and feature dynamic relaxation.

[0083] Strong constraints and correlation propagation phase (when denoising time step) (Time): This is the macroscopic guiding period for the denoising trajectory. First, based on the linear boundary condition property of flow matching, the following is calculated: The theoretical expected trajectory value of the region at the current evolution step :

[0084] ;

[0085] Subsequently, extract the current network predictions. Region original hidden state value Calculate the hard correction difference caused by network prediction bias. :

[0086] ;

[0087] To prevent rigid constraints from causing Region and The geometric fault at the joint of the region is analyzed, and the statistical properties of the conditional Gaussian distribution are used to smoothly propagate the correction amount to the pre-calculated temporal covariance matrix. Region, calculation Adaptive correction compensation amount of the region :

[0088] ;

[0089] In the formula, express The time-series autocorrelation matrix within the region represents Correlation between various dimensions of the region express Region and Temporal cross-correlation matrix between regions.

[0090] To ensure the robustness of the control system, a dynamic safety valve mechanism is implemented: calculation Maximum absolute value If this value is less than or equal to the set safety threshold... The correction compensation will then be superimposed onto the free region to achieve coordinated convergence of motion trajectories across the entire time domain. If the threshold is exceeded, then maintain... The area was left as it was. Finally, it was forcibly... Region alignment to flow matching standard track: .

[0091] Dynamic relaxation and fine-tuning stage (when denoising time step) (Time): At this point, the system enters the final fine-tuning phase of the denoising trajectory. The system is completely deactivated. The rigid orbital constraints and correlation propagation mechanisms of the region allow for the unification of all degrees of freedom in the model. This enables the model to incorporate the latest visual environment features and ontological states, including... Region and Joint smoothing and fine-tuning are performed on all time-domain action points, including the region.

[0092] Action sequence denormalization and servo delivery: Evolution to Then, extract the action sequence of the first 26 steps, and use the time-step average saved in step S1. and standard deviation Perform denormalization recovery and directly send the data to the robot's joint motors for execution.

[0093] To verify the effectiveness of the VLA model training method combined with spatial information enhancement proposed in this invention, a multi-task comparative experiment was conducted on the real wheeled humanoid robot Alphabot. The specific experimental configuration and hyperparameters are as follows:

[0094] Data Acquisition: Expert trajectory data was acquired using a VR remote control system. For five typical application operation tasks—"button," "open door," "close door," "rotate button," and "insert key"—200 high-quality expert motion trajectories were collected for each task.

[0095] Data storage format: All collected trajectory data (including each frame of images of the wrists and head of both arms and the corresponding absolute angles of each joint of the humanoid robot) are uniformly converted and stored in the LeRobot standard data format.

[0096] VLA Baseline Model: The baseline model uses the original Gr00t-N1.7 configuration. To reduce computational overhead and focus on physical world features, the official approach truncates the Multimodal Large Model (VLM) to the first 12 layers and freezes the underlying weights of the VLM backbone network and visual encoder during training.

[0097] Training hyperparameters: Batch size was set to 64, base learning rate to 1e-4, and model checkpoint save steps to 10k steps. All other unmentioned optimizer and diffusion head parameters remained at the default open-source configurations.

[0098] Network details of the present invention: In the spatial information enhancement module of the present invention, the weights of VLM and visual encoder are unfrozen, the hidden layer visual token representation of the 10th layer of the VLM backbone network is extracted, and feature projection and cosine similarity are implicitly aligned with the pre-trained 3D dense geometric perception network (in this embodiment, DUSt3R features containing 3D geometric semantics are used as an example). Similarly, performance testing is performed at 10k step checkpoints.

[0099] For the baseline model and the solution of this invention, 500 independent tests were conducted on 5 humanoid robot control tasks of different difficulty and precision. The task success rates are compared in Table 1 below:

[0100] Table 1 Comparison of Test Success Rates

[0101]

[0102] The quantitative experimental results shown in Table 1 above fully confirm, from a technical perspective, that the present invention possesses the following significant technical effects:

[0103] First, the present invention comprehensively improves the success rate of humanoid robot operation tasks. Under the same number of training steps and hardware constraints, the present invention significantly increases the average success rate of the humanoid robot in five operation tasks from 86.28% to 90.52%. This strongly demonstrates that introducing a spatial information enhancement mechanism into the hidden layer can enable the model to spontaneously evolve stronger spatial generalization ability within a very short training cycle.

[0104] Secondly, the higher the task precision and the more stringent the spatial perception requirements, the more significant the spatial alignment effect of this invention. Among all tasks, the "key insertion" task showed the highest improvement, reaching 8.60%. "Key insertion" is a typical high-precision shaft-hole assembly contact task with extremely low fault tolerance and heavily relies on the absolute three-dimensional distance and relative orientation between the robotic arm's end effector and the keyhole. Traditional 2D VLA, lacking depth information, is prone to physical collisions or misalignment around the keyhole; however, this invention, through 3D supervision signal input, endows the model with precise physical spatial awareness, thus achieving a breakthrough in this challenging task.

[0105] In summary, the experimental data objectively demonstrates the advanced nature of the method described in this invention. It not only successfully overcomes the deficiency of traditional general-purpose 2D visual large models in embodied intelligent physical operations due to "lack of depth perception", but also translates into a significant success rate gain in real humanoid robot interaction tasks.

[0106] The above description is merely a preferred embodiment of the present invention. It should be understood that the present invention is not limited to the forms disclosed herein and should not be construed as excluding other embodiments. It can be used in various other combinations, modifications, and environments, and can be altered within the scope of the concept described herein through the above teachings or related technologies or knowledge. Modifications and variations made by those skilled in the art that do not depart from the spirit and scope of the present invention should be within the protection scope of the appended claims.

Claims

1. A method for training a humanoid robot's vision-language-action model that incorporates spatial information enhancement, characterized in that, Includes the following steps: S1. Time-step action label preprocessing; Before training begins, the feature distribution of the action sequences in the humanoid robot trajectory training set is statistically analyzed at each independent time step. For each specific time step t∈[0,H-1] within the prediction window H, its independent mean vector is calculated. and standard deviation vector Where D is the action dimension; in each forward propagation of training, for the real action label sequence Normalization is performed on a time-step basis, with each step isolated from the others. ； S2. Align the spatial semantic representation information extracted by DUST3R through the output of the VLA intermediate layer; Acquire multi-view RGB images of the humanoid robot's current environment and the corresponding natural language operation text commands; Input image I into the VLA backbone network to extract visual label representation features of specified hidden intermediate layers. ; Input the same image I into a pre-trained 3D dense geometry perception network Output the three-dimensional latent geometric coordinate representation corresponding to the pixel position of each visual marker. ; exist The spatially supervised target features are constructed by adding an accumulated position embedding vector E. ; Through batch normalization layer Γ and a nonlinear perceptron containing two layers of linear mapping, Perform dimensional projection to obtain the mapped feature vector. ; Constructing a 3D information alignment loss function using the cosine similarity metric function : ； Where N is the number of visual markers. This is the cosine similarity metric function; S3. End-to-end training incorporating alignment loss; The time-step normalized action sequence obtained in step S1 Diffusion noise is applied to obtain a noisy noise-added action; The noise level, the denoised time step, and the current embodied state features of the humanoid robot are all input into the diffusion Transformer network; Predict the current motion velocity field and calculate the mean square error loss with the original motion noise. ; Combination and Construct the overall objective loss function End-to-end joint gradient optimization training is performed on the network parameters; S4. Soft action repair control during the reasoning execution period; After the humanoid robot is trained, during the deployment and online real-time inference phase, the 3D basic model path is removed, and a soft motion repair algorithm based on temporal correlation propagation and dynamic relaxation is executed.

2. The training method for a humanoid robot's vision-language-action model combining spatial information enhancement according to claim 1, characterized in that, In step S2, the pre-trained 3D dense geometry perception network The DUST3R architecture network is adopted, which integrates multi-view context by utilizing its internal cross-attention mechanism, and directly outputs a geometric representation containing implicit depth and local 3D topological structure information.

3. The method for training a humanoid robot's vision-language-action model combining spatial information enhancement according to claim 1, characterized in that, In step S4, the soft action repair algorithm based on temporal correlation propagation and dynamic relaxation specifically includes the following steps: Timing scrolling window definition: Set the total length of the timing window for actions predicted in a single forward inference by the VLA model to be... The first step, within the current control cycle, is before the system actually issues and executes the commands. The step action is to take the last remaining part of the current prediction sequence. The step is saved as a historical overlapping context as the initial action vector for the next control cycle. In the forward denoising of the next control cycle, the known constraint region O region is defined, and the remaining unknown prediction part is defined as the free denoising region U region. Noise initialization: During the forward denoising initialization phase of the diffusion model in the current control cycle, the full-time domain multivariate correlation noise z of this cycle is generated, and the fixed noise components corresponding to the O region are extracted and recorded. Simultaneously, retrieve the constraint action target value cached from the previous cycle. ; Dynamic soft-fix evolution guided by flow matching path: In the iterative process of reverse denoising evolution of flow matching from time step t=1.0 to t=0.0, each evolution step Perform action repair and error propagation with dynamic relaxation features through conditional branching; Action sequence denormalization and servo delivery: After evolving to t=0, extract the previous... The step sequence is recovered by inverse normalization using the time-step mean and standard deviation saved in step S1, and then sent to the robot joint motors for execution.

4. The method for training a humanoid robot's vision-language-action model combining spatial information enhancement according to claim 3, characterized in that, In the dynamic soft repair evolution guided by the flow matching path, when the denoising time step Execution of strong constraints and correlation propagation phases: Calculate the theoretical expected trajectory value of region O at the current evolution step based on the linear boundary condition property of flow matching. ; Extract the original hidden state values ​​of region O predicted by the current network. Calculate the hard correction difference: ； Utilizing the statistical properties of the conditional Gaussian distribution, the correction amount is smoothly propagated to region U using a pre-calculated time-series covariance matrix, and the adaptive correction compensation amount for region U is calculated: ； in This represents the time-series autocorrelation matrix within region O. This represents the temporal cross-correlation matrix between region U and region O; The correction compensation is superimposed on the free region to achieve coordinated convergence of motion trajectories across the entire time domain, and the O region is forcibly aligned to the flow matching standard trajectory. .

5. The method for training a humanoid robot's vision-language-action model combining spatial information enhancement according to claim 4, characterized in that, The strong constraint and correlation propagation phase also includes a dynamic safety valve mechanism: calculation Maximum absolute value If the maximum absolute value is less than or equal to the set safety threshold, the correction compensation will be added to the free region. If the threshold is exceeded, the U region remains unchanged.

6. The method for training a humanoid robot's vision-language-action model combining spatial information enhancement according to claim 3, characterized in that, In the dynamic soft repair evolution guided by the flow matching path, when the denoising time step The dynamic relaxation and fine-tuning phase is executed: the rigid orbital constraints and correlation propagation mechanisms of the O region are completely removed, all degrees of freedom of the model are released, and the model is allowed to combine the latest visual environment features and ontological state to perform joint smoothing and fine-tuning of action points in the entire time domain, including the O and U regions.

7. The method for training a humanoid robot's vision-language-action model combining spatial information enhancement according to claim 3, characterized in that, In the time-series scrolling window =30 steps =26 steps, O region index is 0~3, U region index is 4~29.

8. The method for training a humanoid robot's vision-language-action model combining spatial information enhancement according to claim 1, characterized in that, In step S3, the diffusion Transformer network is trained using a flow matching paradigm to predict the current motion velocity field and calculate the mean square error loss with the original motion noise.

9. The method for training a humanoid robot's vision-language-action model combining spatial information enhancement according to claim 1, characterized in that, In step S4, while removing the 3D base model path during the inference execution period, the mean vector of each time step in step S1 is retained. and standard deviation vector Used for inverse normalization of action sequences.

Images