A method, system, medium and product for training and reasoning of embodied intelligence model

Abstract

The application discloses a kind of training and reasoning method, system, medium and product of embodied intelligent model, belong to robot decision field.The method of the application includes task adaptation to pre-trained vision-language-action (VLA) model;Freeze the model parameters of vision-language-action model;Iterative training is carried out to action residual expert model using basic action block and multi-modal context feature obtained by task adaptation.The application aims to retain the complete task execution capability of the existing basic VLA model, directly learn the remaining action error using offline demonstration data, and improve the correction of fine-grained action error based on maintaining coarse-grained action accuracy, focusing on improving the accuracy of precision-sensitive interactive phase, solving the problem of uneven learning of coarse and fine granularity in the complete task demonstration trajectory of the VLA model after task adaptation, and has no dependence on specific VLA model.

Description

Technical Field

[0001] This invention belongs to the fields of robot learning, embodied intelligence, vision-language-action models and robot decision-making, and specifically relates to a training and reasoning method, system, medium and product for embodied intelligence models. Background Technology

[0002] Vision-language-action (VLA-MA) models can map camera images, natural language task commands, and robot state into executable actions, and have become an important means of implementing general robot operation strategies. To adapt general models to specific downstream tasks, task demonstration trajectories are typically used to adapt or fine-tune pre-trained VLA-MA models for specific tasks. Actual robot operations usually involve actions of different granularities, such as coarse-grained and fine-grained actions. Coarse-grained actions include arrival, transport, movement, and repositioning, and their trajectories are generally smooth and have a large tolerance range. Fine-grained actions include contact, precise alignment, grasping closure, insertion, confined placement, and post-contact stabilization, and their action amplitudes may be small, but they have high requirements for the accuracy of end-effector position, posture, and execution timing. Even a small error in a fine-grained action can lead to grasping failure, collision, insertion deviation, or placement failure. In the task adaptation process of the complete task demonstration trajectory, coarse-grained actions usually occupy more time steps and have relatively smooth and easy-to-fit action patterns, thus easily contributing significantly to the optimization of the unified training objective. In contrast, fine-grained key actions such as contact, alignment, grasping, and insertion have shorter durations and relatively sparse effective supervision, but require higher action accuracy. Therefore, the basic actions output by the task-adapted vision-language-action model have smaller errors in coarse actions but larger errors in fine actions.

[0003] To address the issue of errors in basic robot movements, existing technologies have introduced solutions that further incorporate compensation or residuals to correct these errors. For example, Chinese invention patent application CN122185241A discloses a control method for a legged robot. This method first generates multiple candidate action sequences using a generative motion prior model. The optimal action sequence is then selected through dynamic model prediction and multi-objective evaluation considering stability, energy consumption, collision risk, and slip risk. Finally, a residual reinforcement learning model is used to output the action residuals to compensate for control errors caused by factors such as unknown ground friction, load variations, actuator performance degradation, and perception bias. Another Chinese invention patent application CN121962646A discloses a generative large-scale robot manipulation method and system for indoor dynamic scenes. This method acquires dynamic target motion information through optical flow and predicts the robot's future state. Macro-action blocks are generated from a low-frequency vision-language-action model. A high-frequency reinforcement learning residual refiner continuously corrects the macro-actions based on real-time observations to compensate for state drift caused by moving targets, inference delays, and open-loop execution of action blocks. Although the above-mentioned schemes differ in their application scenarios and basic action generation methods, they all employ reinforcement learning to obtain action residuals. Their optimization process relies on robot-environment interaction and corresponding rewards or value signals, essentially belonging to the technical route of using reinforcement learning for online adaptation or closed-loop compensation of existing actions. Such methods typically require additional robot interaction rounds and reinforcement learning training processes, and may also lead to high task reset costs and real robot exploration costs in situations involving rich operational tasks. Furthermore, Chinese invention patent application CN121960563A discloses a method for determining humanoid robot action parameters. However, firstly, this scheme does not distinguish between coarse-grained and fine-grained action stages in an action sequence, nor does it address or analyze the imbalance in prediction accuracy at different action granularities; secondly, the action loss in this scheme mainly supervises the final action, i.e., the mean square error between the coarse action and the action resulting from the sum of multiple residual terms and the actual action; finally, this scheme does not set up a granularity-aware training mechanism. Overall, this scheme belongs to a general residual refinement method oriented towards overall action parameter prediction.

[0004] In summary, while existing control schemes can improve robot motion accuracy by adding corrections, they mainly compensate for dynamic disturbances, dynamic environmental changes, or general coarse motion errors. They cannot solve the problem of unbalanced coarse-grained learning in the complete task demonstration trajectory of the vision-language-action model after task adaptation. Summary of the Invention

[0005] The technical problem to be solved by this invention is to provide a training and inference method, system, medium, and product for embodied intelligent models, addressing the aforementioned problems in the prior art. This invention aims to retain the basic vision-language-action strategy and its existing complete task execution capabilities, while directly using offline demonstration data to learn the remaining action errors. It strengthens the correction of fine-grained action errors while maintaining coarse-grained action accuracy, focusing on improving the accuracy of actions in the precision-sensitive interaction stage, and solving the problem of unbalanced coarse-grained and fine-grained learning in the complete trajectory of the vision-language-action model after task adaptation.

[0006] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows: A method for training and reasoning embodied intelligence models includes the following steps: S101, Read or collect the complete task demonstration trajectory. The complete task demonstration trajectory is an offline training sample consisting of the robot's observations at various trajectory points and expert action blocks during the complete task process. The expert action block consists of one or more predefined actions. S102, Task adaptation is performed on the pre-trained visual-language-action model to enable it to generate basic actions. The visual-language-action model includes a visual-language model and a motion expert model. The visual-language model comprises multiple Transformer layers. After the robot's observations are input into the visual-language model, the hierarchical key-value pairs (KV) generated by each Transformer layer are concatenated and output as multimodal contextual features. The final output features of the visual-language model are then predicted by the motion expert model to generate corresponding basic action blocks. Each basic action block includes a preset action temporal length. The system comprises multiple control steps within a given range, each control step being a D-dimensional action. The basic action block is divided into coarse-grained action blocks and fine-grained action blocks. The actions of each control step within a coarse-grained action block are coarse-grained actions, and the actions of each control step within a fine-grained action block are fine-grained actions. The action residual expert model includes: an action projection module for projecting the input basic action block; a multi-level Transformer module for extracting features from the projected basic action block and multimodal context features; and a zero-initialized residual output head for regressing the action residual based on the output features of the Transformer module. S103, Freeze the model parameters of the vision-language-action model; S104. The motion residual expert model is iteratively trained based on the basic action blocks and multimodal context features output by the vision-language-action model, thereby obtaining an embodied intelligence model composed of the vision-language-action model and the motion residual expert model for realizing the robot's action reasoning.

[0007] Optionally, when iteratively training the action residual expert model based on the basic action blocks output by the vision-language-action model under parameter freezing conditions and their corresponding multimodal context features, the loss function used includes the predicted action residuals and the deviation between the residual supervision targets.

[0008] Optionally, the calculation function expression for the residual loss weight of the expert action block is: ; in, Let i be the weight of the i-th expert action block. Greater than 1, For the first Does each expert action block belong to the category of fine motor skills indicators? If so, then... =1, otherwise It is 0.

[0009] Optionally, the observation of each trajectory point includes the robot's visual observation at each trajectory point, the robot's state, and natural language task instructions. The robot's state includes some or all of the robot's joint positions, tactile sensation, temperature, humidity, force sensation, and hearing.

[0010] Optionally, the expert action block can be a coarse-grained action or a fine-grained action. The coarse-grained action includes some or all of the actions of arrival, transport, movement, and repositioning, while the fine-grained action includes some or all of the actions of contact, precise alignment, gripping and closing, insertion, confined placement, and stabilization after contact.

[0011] Optionally, step S104 may be followed by: S201, The robot's current observations are used to perform task adaptation using a vision-language-action model to obtain basic action blocks and their corresponding multimodal context features; S202, Input the basic action block and its corresponding multimodal context features into the trained action residual expert model to predict the action residual; S203, add the basic action blocks and action residuals element by element to obtain the robot's final action block at the current moment, which is used to implement the robot's actions.

[0012] Furthermore, the present invention also provides a training and inference system for an embodied intelligence model, including a microprocessor and a memory interconnected thereto, wherein the microprocessor is programmed or configured to execute the training and inference method of the embodied intelligence model.

[0013] Furthermore, the present invention also provides a computer-readable storage medium storing a computer program or instructions that are programmed or configured to execute the training and inference method of the embodied intelligence model by a processor.

[0014] Furthermore, the present invention also provides a computer program product, including a computer program or instructions that are programmed or configured to execute the training and reasoning method of the embodied intelligence model via a processor.

[0015] Compared with the prior art, the present invention can mainly achieve the following beneficial effects: 1. This invention addresses the issue of insufficient coarse-grained motion learning but inadequate fine-grained motion prediction accuracy in training complete demonstration trajectories using a task-adapted visual-language-action (VLA) model. First, the basic VLA model is task-adapted using task demonstration data. After adaptation, the basic policy is frozen, allowing it to output basic action blocks and multimodal context features based on visual observations, robot body state, and language commands. Subsequently, a lightweight motion residual expert (ARE) model is trained, using the basic action blocks as action priors and reusing the multimodal context features generated by the basic policy to predict remaining motion errors. The predicted motion residuals are added to the basic action blocks to obtain the final robot action. This approach retains the existing complete task execution capability of the basic VLA policy while directly learning its remaining motion errors using offline demonstration data. It strengthens the correction of fine-grained motion errors while maintaining coarse-grained motion accuracy, focusing on improving the accuracy of actions in precision-sensitive interaction stages. This solves the problem of unbalanced coarse-grained and fine-grained learning in the complete task demonstration trajectory of the task-adapted VLA model.

[0016] 2. This invention does not require additional online robot exploration, task rewards, or preference feedback. It can retain the coarse-grained operation capability of the basic strategy while focusing on correcting fine-grained execution errors. It is also applicable to visual-language-action models that use different action generation forms such as autoregressive action labeling, diffusion, or flow matching.

[0017] 3. This invention is not dependent on specific basic vision-language-action (VLA) models and can be applied to various vision-language-action models, thus having the advantage of good versatility. Attached Figure Description

[0018] Figure 1 This is a schematic diagram illustrating the basic principle of the method in an embodiment of the present invention.

[0019] Figure 2 This is a schematic diagram of the network structure in an embodiment of the present invention. Detailed Implementation

[0020] To enable those skilled in the art to better understand the technical solutions of the present invention, the technical solutions of the present invention will be further described in detail below with reference to the accompanying drawings in the embodiments of the present invention.

[0021] like Figure 1 As shown, the training and inference method of the embodied intelligence model in this embodiment includes the following steps: S101, Read or collect the complete task demonstration trajectory. The complete task demonstration trajectory is an offline training sample consisting of the robot's observations at various trajectory points and expert action blocks during the complete task process. The expert action block consists of one or more predefined actions. S102, adapts the pre-trained Visual-Language-Action (VLA) model to the task, enabling the VLA model to generate basic actions; S103, Freeze the model parameters of the vision-language-action model; S104. Based on the basic action blocks and multimodal context features output by the adapted visual-language-action model under the condition of parameter freezing, the motion residual expert (ARE) model is iteratively trained to obtain an embodied intelligent model composed of the visual-language-action model and the motion residual expert model for realizing the robot's action reasoning.

[0022] This embodiment provides a training and inference method for embodied intelligent models (BoostVLA). First, a pre-trained visual-language-action model is task-adapted using complete task demonstration trajectories to obtain a task-adapted basic visual-language-action model. Then, the parameters of the basic visual-language-action model are frozen, and an Action Residual Expert (ARE) model is trained using existing offline demonstration data. The VLA model is used to generate basic action blocks, and the ARE model is used to predict local action residuals based on the basic action blocks and their corresponding multimodal contextual features, thereby correcting the residual error between the basic action blocks and the expert action blocks. Furthermore, this embodiment improves the contribution of fine-grained action blocks to the training loss through granular residual training.

[0023] The method in this embodiment applies to the action output end of the basic vision-language-action strategy. It does not require replacing the original vision-language encoding method or action generation method of the basic strategy. Therefore, it can be applied to vision-language-action models that use autoregressive action labeling, diffusion generation, flow matching generation or other action generation paradigms.

[0024] S101, Read or acquire the complete task demonstration trajectory. The complete task demonstration trajectory is an offline training sample consisting of the robot's observations at various trajectory points and expert action blocks during the complete task. Each expert action block consists of one or more predefined actions. The observations at each trajectory point include visual observations, robot body state, and natural language task instructions. At time... Multimodal observation can be represented as: in, Indicates that one or more cameras are at time Visual observations collected; Indicates the state of the robot itself; The camera can be positioned on the robot's head, wrist, hand, or outside the workspace to represent natural language task commands. The robot's body state can include some or all of the following: joint position, joint velocity, end effector pose, gripper or suction cup state, and internal sensor information such as touch, force, or torque. Ambient temperature, humidity, or sound information can be used as additional environmental observation inputs, but are not limited to the robot's body state.

[0025] S102, Task adaptation is performed on the pre-trained visual-language-action model to enable it to generate basic actions. For current multimodal observation... The task-adapted visual-language-action model generates basic action blocks: While generating basic action blocks, multimodal contextual features corresponding to the current observation are obtained from the basic vision-language-action model. ; This invention provides a hierarchical key-value cache for each Transformer layer. The specific structure of the basic vision-language-action model is not limited in this invention. As an optional implementation, the basic vision-language-action model can be based on autoregressive, diffusion, flow matching, or other output-capable network structures.

[0026] S103, Freeze the model parameters of the VLA model after task adaptation. Specifically, after task adaptation is completed, the model parameters of the visual encoding module, language encoding module, multimodal fusion module, and action generation module in the VLA model are set to an untrainable state. During the subsequent iterative training of the action residual expert model, the frozen VLA model only performs forward computation on the multimodal observations in the offline training samples to output basic action blocks and their corresponding multimodal context features, without participating in gradient backpropagation and model parameter updates; the training optimizer only updates the trainable parameters of the action residual expert model. By fixing the generation process of basic action blocks and multimodal context features, the basic action blocks serve as stable action priors, avoiding drift of the residual supervision target due to changes in VLA model parameters during the training of the action residual expert model, while preserving the overall task execution capability obtained by the VLA model after task adaptation.

[0027] S104, based on the basic action blocks output by the frozen VLA model and their corresponding multimodal context features, iteratively train the action residual expert model. As an optional implementation method, such as... Figure 2As shown, the VLA model includes a visual-language model and a motion expert model; the motion residual expert model includes a motion projection module, a Transformer module, and a zero-initialization residual output head.

[0028] The visual-language-action (VLA) model is formed by freezing the input parameters of visual observations, natural language task instructions, and robot state. The visual-language model module in the VLA model extracts multimodal features from the input visual observations, natural language task instructions, and robot state to obtain multimodal contextual features. The action expert model in the VLA model generates basic action blocks based on the features output by the visual-language model. These basic action blocks are input to the action residual expert model and also propagated along a bypass to the action block synthesis position.

[0029] In the action residual expert model, the action projection module projects features onto the basic action block to obtain basic action features. The Transformer module extracts features based on the projected basic action features and the multimodal context features provided by the vision-language model to obtain residual features for predicting action residuals. The zero-initialized residual output head can be a linear layer with its weight and bias parameters initialized to zero, so that the action residual expert model outputs zero action residuals at the beginning of training. Therefore, in the initial training stage, the final action block obtained by adding the action residual block to the basic action block is consistent with the basic action block, avoiding significant disturbance to the existing action generation capability of the basic vision-language-action model caused by untrained action residuals. As iterative training progresses, the action residual expert model uses the basic action block as the action prior and gradually learns the residual action error between the basic action block and the expert action block based on the multimodal context features. Then, it adds the residual error element-by-element to the basic action block passed along the bypass to obtain the final action block.

[0030] During the iterative training of the action residual expert model, for the first [item name] in the offline demonstration data... Each expert action block is denoted as: ,in Indicates the time domain length of the action. This represents the action dimension for each control step. The multimodal observations corresponding to this expert action block are then used. Based on the frozen fundamental visual-language-action model Generate basic action blocks: in, This represents the basic strategy parameters for freezing. The difference between the expert action block and the basic action block is then defined as the residual supervision objective. in This indicates the stopping gradient operator, used to prevent gradients from propagating to the base vision-language-action model during the training of the action residual expert. In this way, the action residual expert only learns the errors of the base action, without having to relearn the overall mapping from complete multimodal observations to the action.

[0031] During the iterative training of the motion residual expert model, granularity-aware residual training is further employed. Robotic operation tasks typically involve actions of different granularities simultaneously. Coarse-grained actions refer to actions with a large range of motion and a wide tolerance range, such as some or all of arriving, carrying, moving, and repositioning; fine-grained actions refer to actions with a small range of motion and high requirements for position, orientation, contact state, or execution timing accuracy, such as some or all of contact, precise alignment, grasping closure, insertion, confined placement, and post-contact stabilization.

[0032] In the complete task demonstration trajectory, coarse-grained actions such as reaching the target area, transporting objects, and repositioning typically involve longer control times, thus the training samples corresponding to these coarse-grained action phases constitute a larger proportion of the offline demonstration data. Fine-grained operations such as contact, precise alignment, grasping closure, and insertion, on the other hand, typically have shorter durations, resulting in a smaller proportion of training samples related to these fine-grained actions. If the same residual loss weight is applied to all training samples in the complete demonstration trajectory, the larger proportion of coarse-grained action-related samples will have a significant cumulative contribution to the total residual loss. This means that the parameter optimization of the action residual expert model is primarily influenced by coarse-grained action samples, while the smaller proportion of fine-grained action samples contributes relatively little to parameter updates.

[0033] However, fine-grained actions are typically more sensitive to errors in position, posture, contact state, and execution timing. Local errors in fine-grained actions within basic action blocks can directly impact the overall task execution result. Therefore, this embodiment employs granularity-aware residual training while retaining all training samples from the complete task demonstration trajectory: different residual loss weights are assigned to different action blocks to increase the contribution of fine-grained action-related samples to the total residual loss and parameter updates, thereby mitigating the optimization bias caused by the imbalance between coarse-grained and fine-grained actions in the offline demonstration data.

[0034] The function expression for calculating the residual loss weight of the expert action block is as follows: ; in, Let i be the weight of the i-th expert action block. Greater than 1, For the first Does each expert action block belong to the category of fine motor skills indicators? If so, then... =1, otherwise It is 0.

[0035] During training, all training samples from the complete demonstration trajectory are retained, and only the parameters of the action residual expert model are updated. Therefore, the entire action block corresponding to the training sample whose input time falls within the critical extension interval of the interaction has a high optimization contribution, while other training samples still participate in training with basic weights. This strengthens the correction of local action errors in the accuracy-sensitive phase while preserving coarse-grained behavioral supervision in the complete task demonstration trajectory.

[0036] This embodiment eliminates the need for online robot rollouts, task rewards, or preference annotations. Instead, it utilizes existing offline demonstration data to train an action residual expert model. By freezing the base VLA model, performing zero initialization on the residual output head, and employing granularity-aware residual training, it can enhance the correction of action errors in contact-rich and precision-sensitive operation phases while maintaining the overall task execution capability of the basic strategy.

[0037] Through the above iterative training, an embodied intelligence model is obtained, consisting of a frozen vision-language-action model and a trained action residual expert model, which can be used to realize the robot's action reasoning.

[0038] Optionally, after step S104, the following robot action reasoning steps are also included: S201, using a vision-language-action model to adapt the robot's current observations to the task and obtain basic action blocks. and its corresponding multimodal context features ; S202, basic action block and its corresponding multimodal context features The pre-trained action residual expert model is input to predict action residuals. The action residual expert model uses basic action blocks as action priors and multimodal contextual features as conditional information to predict action residuals. It can be represented as: ; in, For motion residual expert models, The trainable parameters represent the motion residual expert; S203, basic action block and motion residuals The final action block of the robot at the current moment is obtained by adding elements one by one. For the purpose of implementing action reasoning in robots, it can be expressed as: ; When the vision-language-action model has accurately predicted a coarse-grained action, the motion residual expert model can output a correction amount close to zero; when the vision-language-action model has local errors in the contact, alignment, grasping, insertion, or restricted placement stages, the motion residual expert model outputs the corresponding position, posture, joint, or end effector correction amount.

[0039] To verify the improvement effect of the embodied intelligent model training and inference method (BoostVLA) of this embodiment on different basic vision-language-action models, comparative experiments were conducted on two robot operation simulation benchmarks, LIBERO and MetaWorld, and π0, π0.5, and SmolVLA were selected as vision-language-action models (basic models), respectively. For each basic model, task adaptation was first completed using the task demonstration trajectory, then the basic model was frozen and the action residual expert was trained according to the embodied intelligent model training and inference method (BoostVLA) of this embodiment, and optimization was performed using granular-aware residual training. The task success rate (%) was used as the evaluation index, and the results are shown in Tables 1 and 2.

[0040] Table 1: Comparative experimental results of BoostVLA on different vision-language-action models on the LIBERO benchmark.

[0041] Table 2: Comparative experimental results of BoostVLA on different vision-language-action models on the MetaWorld benchmark.

[0042] As shown in Tables 1 and 2, the embodied intelligence model training and inference method (BoostVLA) in this embodiment can improve the overall task success rate on three different basic vision-language-action models: π0, π0.5, and SmolVLA. Specifically, on the LIBERO benchmark, the average success rates of π0, π0.5, and SmolVLA increased from 85.45%, 96.85%, and 45.95% to 92.20%, 97.50%, and 58.95%, respectively; on the MetaWorld benchmark, the average success rates of the three basic models increased from 47.38%, 32.42%, and 55.13% to 62.30%, 36.95%, and 58.77%, respectively. Experimental results demonstrate that the embodied intelligence model training and inference method (BoostVLA) in this embodiment can adapt to different basic vision-language-action models and further improve the task execution success rate while preserving the basic action generation structure.

[0043] To verify the role of granularity-aware residual training in the embodied intelligence model training and inference method (BoostVLA) of this embodiment, ablation experiments were conducted while maintaining the same basic vision-language-action model and action residual expert structure. Uniform Residual Training (URT) applied the same residual loss weights to all action blocks in the complete task trajectory; Granularity-Aware Residual Training (GRT) assigned higher residual loss weights to fine-grained action blocks such as contact, alignment, and grasping, while retaining complete trajectory supervision. URT and GRT differed only in the residual loss weight settings. Experiments were conducted on the π0 and SmolVLA basic models, and the average task success rate (%) is shown in Table 3.

[0044] Table 3: Results of BoostVLA Granularity-Aware Residual Training Ablation Experiment

[0045] As shown in Table 3, when using URT, the training and inference method (BoostVLA) of the embodied intelligent model in this embodiment can already improve the task success rate on top of the basic vision-language-action model; after further using GRT, the task success rate continues to improve. For π0, the average success rate of LIBERO further increased from 91.35% to 92.20%, and MetaWorld increased from 60.78% to 62.30%; for SmolVLA, LIBERO increased from 54.78% to 58.95%, and MetaWorld increased from 57.26% to 58.77%. Since URT and GRT use the same basic model and action residual expert structure, only the action block loss weights are different, the above results show that the performance improvement of the training and inference method (BoostVLA) of the embodied intelligent model in this embodiment not only comes from the action residual expert, but also from granular-aware residual training, which can bring further performance gains.

[0046] Furthermore, this embodiment also provides a training and inference system for an embodied intelligence model, including a microprocessor and a memory interconnected, wherein the microprocessor is programmed or configured to execute the training and inference method of the embodied intelligence model. This embodiment also provides a computer-readable storage medium storing a computer program or instructions programmed or configured to execute the training and inference method of the embodied intelligence model via a processor. This embodiment also provides a computer program product, including a computer program or instructions programmed or configured to execute the training and inference method of the embodied intelligence model via a processor.

[0047] Those skilled in the art will understand that the technical solutions provided by this invention may take the form of a method, system, or computer program product. Therefore, this invention may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this invention may take the form of a computer program product embodied on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, produce an implementation of the flowchart... Figure 1 One or more processes and / or boxes Figure 1 The computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The functions specified in one or more boxes. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable apparatus for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0048] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A method for training and reasoning embodied intelligence models, characterized in that, Includes the following steps: S101, Read or collect the complete task demonstration trajectory. The complete task demonstration trajectory is an offline training sample consisting of the robot's observations at each trajectory point and expert action blocks during the complete task process. The expert action block consists of one or more predefined actions. S102, Task adaptation is performed on the pre-trained visual-language-action model to enable it to generate basic actions. The visual-language-action model includes a visual-language model and an action expert model. The visual-language model includes multiple Transformer layers. After the robot's observations are input into the visual-language model, the hierarchical key-value pairs (KV) generated by each Transformer layer are concatenated and output as multimodal context features. The final output features of the visual-language model are predicted by the action expert model to generate corresponding basic action blocks. The basic action blocks include multiple control steps within a preset action time domain length range, and each control step is a D-dimensional action. The basic action blocks are divided into coarse-grained action blocks and fine-grained action blocks. The actions of each control step under the coarse-grained action block are coarse-grained actions, and the actions of each control step under the fine-grained action block are fine-grained actions. The motion residual expert model includes: a motion projection module for projecting the input basic motion block; a multi-level Transformer module for extracting features from the projected basic motion block and multimodal context features; and a zero-initialized residual output head for regressing the motion residual based on the output features of the Transformer module. S103, Freeze the model parameters of the vision-language-action model; S104. The motion residual expert model is iteratively trained based on the basic action blocks and multimodal context features output by the vision-language-action model, thereby obtaining an embodied intelligence model composed of the vision-language-action model and the motion residual expert model for the purpose of realizing the robot's action reasoning.

2. The training and inference method for embodied intelligent models according to claim 1, characterized in that, When iteratively training the action residual expert model based on the basic action blocks output by the vision-language-action model under parameter freezing conditions and their corresponding multimodal context features, the loss function used includes the predicted action residuals and the deviation between the residual supervision target and the residual residuals.

3. The training and inference method for embodied intelligent models according to claim 1, characterized in that, The function expression for calculating the residual loss weight of the expert action block is as follows: ； in, Let i be the weight of the i-th expert action block. Greater than 1, For the first Does each expert action block belong to the category of fine motor skills indicators? If so, then... =1, otherwise It is 0.

4. The training and inference method for embodied intelligent models according to claim 1, characterized in that, The observation of each trajectory point includes the robot's visual observation at each trajectory point, the robot's state, and natural language task instructions. The robot's state includes some or all of the robot's joint positions, tactile sensation, temperature, humidity, force sensation, and hearing.

5. The training and inference method for embodied intelligent models according to claim 1, characterized in that, The expert action block can be a coarse-grained action or a fine-grained action. The coarse-grained action includes some or all of the actions of arrival, transport, movement and repositioning, while the fine-grained action includes some or all of the actions of contact, precise alignment, gripping and closing, insertion, confined placement and stabilization after contact.

6. The training and inference method for embodied intelligent models according to claim 1, characterized in that, Step S104 is followed by: S201, The robot's current observations are used to perform task adaptation using a vision-language-action model to obtain basic action blocks and their corresponding multimodal context features; S202, Input the basic action block and its corresponding multimodal context features into the trained action residual expert model to predict the action residual; S203, add the basic action blocks and action residuals element by element to obtain the robot's final action block at the current moment, which is used to implement the robot's actions.

7. A training and inference system for an embodied intelligence model, comprising a microprocessor and a memory interconnected, characterized in that, The microprocessor is programmed or configured to execute the training and inference method of the embodied intelligence model according to any one of claims 1 to 6.

8. A computer-readable storage medium storing a computer program or instructions, characterized in that, The computer program or instructions are programmed or configured to execute, via a processor, the training and reasoning method of the embodied intelligence model as described in any one of claims 1 to 6.

9. A computer program product, comprising a computer program or instructions, characterized in that, The computer program or instructions are programmed or configured to execute, via a processor, the training and reasoning method of the embodied intelligence model as described in any one of claims 1 to 6.

Images