Robot action generation method and system combining universal and special models

Abstract

The invention relates to a robot action generation method and system combining a universal model and a special model, and the method comprises the steps: constructing the universal model and the special model, carrying out the pre-training of the universal model, carrying out the parameter fine tuning of the pre-trained universal model, and training the special model based on the fine-tuned universal model; obtaining a task instruction and real-time visual information, inputting the task instruction and the real-time visual information into the fine-tuned universal model, and outputting an action sequence and task potential features; and acquiring real-time point cloud sensing data, inputting the real-time point cloud sensing data, the action sequence and the task potential features into the trained special model, and outputting continuous actions of the robot. Compared with the prior art, the robot action generation speed is increased, and the generalization of robot action generation is enhanced.

Description

Technical Field

[0001] The present invention relates to the field of robot control technology, and in particular to a robot motion generation method and system combining general and special models. Background Art

[0002] Realizing robots with multi-task operation and adaptive capabilities has always been a core goal in the field of robotics. Traditional robot learning methods usually learn strategies through data sets that specify robots and specific tasks. These operation strategies are called dedicated models. Dedicated models show high accuracy in specific scenarios and tasks, but often have limited generalization capabilities. As robots are increasingly used in open and multi-task scenarios, the demand for multi-task robots has surged.

[0003] In response to the needs of multi-task robots, general robot strategies have begun to develop, such as RT-2 and OpenVLA. These strategies attempt to apply Internet knowledge to robot control and leverage a wide range of heterogeneous datasets to enhance the versatility of robots across domains. General strategies integrate a large amount of cross-ontology data with a pre-trained large language model to implement functions such as common sense reasoning and instruction tracking in robot policy learning. General strategies are good at knowledge transfer and generalization in different scenarios, but there are still some limitations: 1) They cannot be directly deployed to new applications or environments without fine-tuning. The fine-tuning process requires more data and training than dedicated strategies. 2) Although general strategies are good at decision-making, their large model characteristics lead to extremely high inference latency. This key bottleneck makes them unsuitable for fine control in dynamic environments. The current approach of using lightweight models to address the above shortcomings will result in a significant drop in model performance.

[0004] Therefore, providing a robot motion generation method that can meet both the generalization of general models and the high performance of special models is a technical problem that needs to be solved. Summary of the invention

[0005] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and to provide a robot motion generation method and system that combines general and special models. The general model and the special model are effectively combined, the general model is used to process complex multimodal data, the point cloud data is used as the input of the special model and the output of the general model is used as the diffusion denoising condition, thereby retaining both the generalization of the general model and the accuracy of the special model.

[0006] The purpose of the present invention can be achieved by the following technical solutions:

[0007] According to a first aspect of the present invention, there is provided a method for generating robot motions combining general and special models, characterized in that the method comprises:

[0008] Constructing a general model and a special model, pre-training the general model, fine-tuning parameters of the pre-trained general model, and training the special model based on the fine-tuned general model;

[0009] Obtaining task instructions and real-time visual information, inputting the task instructions and real-time visual information into the fine-tuned general model, and outputting action sequences and task potential features;

[0010] Real-time point cloud perception data is acquired, and the real-time point cloud perception data, the action sequence and the task potential features are input into a trained dedicated model to output continuous actions of the robot.

[0011] As a preferred technical solution, the universal model is constructed based on a large visual language model.

[0012] As a preferred technical solution, the dedicated model is constructed based on a lightweight and scalable diffusion model.

[0013] As a preferred technical solution, the pre-training of the general model includes:

[0014] Acquiring robot operation data, wherein the robot operation data includes language instructions, image data and robot motion data;

[0015] Based on the language instructions and the image data, using the general model to extract language features and visual features, and aligning the language features and visual features;

[0016] Iterate the following steps until pre-training is complete:

[0017] Inputting the aligned language features and visual features into a universal model, outputting a first discretized action, and performing decoding processing on the first discretized action;

[0018] Based on the decoded first discretized action and robot action data, a first loss is calculated and back-propagation is performed, a first gradient is calculated, and parameters of the general model are updated based on the first gradient.

[0019] As a preferred technical solution, the fine-tuning method of the general model includes:

[0020] Obtain robot operation data for a specific task and obtain the parameters of the pre-trained general model as initial parameters;

[0021] Determine fine-tuning ranges of parameters involved in visual feature extraction, language feature extraction, and action decoding in the initial parameters, and freeze other parameters in the initial parameters;

[0022] Inputting the robot operation data under the specific task into the general model and outputting a second discretized action;

[0023] Based on the second discretized action and the robot action data in the robot operation data under the specific task, a second loss is calculated, and back propagation is performed to calculate the second gradient, and the parameters of the general model are updated based on the second gradient.

[0024] As a preferred technical solution, the training of the dedicated model includes:

[0025] Acquire a point cloud data set and robot operation data at a corresponding time, wherein the point cloud data set includes point cloud data, robot state body perception data, and robot motion data;

[0026] Based on the robot operation data at the corresponding moment, the fine-tuned general model is used to output the action sequence and the task potential features;

[0027] Based on the point cloud data set, action sequence and task potential features, a dedicated model is trained, the loss of the training process is calculated, and back propagation is performed to calculate the gradient, and the parameters of the dedicated model are updated based on the gradient.

[0028] As a preferred technical solution, the general model and the dedicated model collaborate asynchronously, and the method of the asynchronous collaboration is:

[0029] Obtain the task instructions and real-time visual information at the current time t1, and use the general model to output the action sequence and task potential features at time t1;

[0030] Get t1 to t n The real-time point cloud perception data at time t1 is used to output the time series from t1 to t2 based on the action sequence and task potential features at time t1. n Continuous action at all times.

[0031] As a preferred technical solution, the method for outputting the action sequence and task potential features at time t1 is:

[0032] Based on the visual information at time t1, the general model is used to extract visual features, and based on the task instructions at time t1, the general model is used to extract language features;

[0033] Project the visual features and language features into a unified latent space to generate the task potential features at time t1;

[0034] Based on the visual features and language features, a general model is used to generate a discretized action sequence according to the time step, and its expression is:

[0035] a t1 =g φ (a＜t1 ,c),

[0036] Among them, a t1 represents the action sequence at time t1, a ＜t1 represents the action sequence before time t1, c represents the visual features and language features, g φ (·) indicates general model processing.

[0037] As a preferred technical solution, the output t1 to t n The method of continuous action at the moment is:

[0038] Based on the t1 to t n Real-time point cloud perception data extraction at all times Real-time point cloud perception features;

[0039] The real-time point cloud perception features at each moment are aligned and fused with the action sequence and task potential features at time t1, and the diffusion denoising mechanism is used to generate the action at the corresponding moment. The expression is:

[0040]

[0041] in, Indicates t i The action at the moment and i∈[1,n], ∈ represents Gaussian noise, Indicates t i The real-time point cloud perception features at moment t are aligned and fused with the action sequence and task potential features at moment t1.

[0042] According to a second aspect of the present invention, there is provided a robot motion generation system combining general and special models, wherein the system is used to implement the above method.

[0043] Compared with the prior art, the present invention utilizes a general model to process multimodal data composed of vision and language, and uses the output of the general model as the diffusion denoising condition of the special model and combines it with point cloud data that can provide better spatial information to output the continuous actions of the robot, while retaining the generalization of the general model and the accuracy of the special model; and the present invention maintains asynchronous collaboration between the general model and the special model, that is, the general model infers once at the current moment, and the special model infers actions at the current moment and multiple moments in the future based on the inference result of the general model at the current moment, which reduces the resource consumption of the general model and can also ensure the accuracy of the generated actions. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 is a flow chart of the method of the present invention;

[0045] Figure 2 FIG. 4 is a hardware architecture diagram of the system of the present invention. DETAILED DESCRIPTION

[0046] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0047] Unless otherwise defined, the technical terms or scientific terms involved in this application should be understood by people with ordinary skills in the technical field to which this application belongs. The words "one", "a", "a", "the" and the like involved in this application do not indicate a quantitative limitation, and may represent the singular or plural. The terms "include", "comprise", "have" and any of their variations involved in this application are intended to cover non-exclusive inclusions; for example, a process, method, system, product or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units that are not listed, or may also include other steps or units inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The "multiple" involved in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there may be three relationships, for example, "A and / or B" can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the objects before and after are in an "or" relationship. The terms "first", "second", "third", etc. involved in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0048] This embodiment combines the generalization of a multimodal general model and the high efficiency of a dedicated model to improve the generalization and reasoning speed of the robot motion generation system. The dedicated model uses point cloud data as input. Compared with RGB image data, point cloud data provides better spatial information, which helps to improve the robot's operational capabilities.

[0049] Specifically, the process of this method is as follows Figure 1 As shown, including:

[0050] S1. Model construction:

[0051] S11. Based on a large visual language model such as OpenVLA, an autoregressive visual language action model with about 7B parameters is constructed as a general model. Specifically, the general model includes: a visual encoder, a large model (LLaMA-2) module and an action generation module, wherein the action generation module includes an action decoder.

[0052] S12. Pre-trained general model:

[0053] S121, obtaining robot operation data, including language instructions, RGB images and robot motion data;

[0054] S122. Use visual encoders (DINOv2 and ViT) to extract visual features based on RGB images, use a large language model to extract language features based on language instructions, and use a multi-layer perceptron to map the visual features to a latent space consistent with the language features for feature alignment.

[0055] S123. Input the aligned language features and visual features into the action generation module, output the first discretized action, and decode the first discretized action. The action range is mapped to a uniform distribution of [-1, 1] to ensure that the general model has clear physical constraints when generating actions.

[0056] S124. Based on the decoded first discretized action and robot action data, calculate the first loss and perform back propagation, calculate the first gradient, and based on the first gradient, use the AdamW optimizer to update the parameters of the general model.

[0057] The calculation expression of the first loss is:

[0058]

[0059] Among them, p is the input language prompt, a ＜i is the action sequence generated before step i, N a is the length of the action sequence.

[0060] Steps S123 to S124 are iterated until the pre-training is completed.

[0061] S13. General model fine-tuning:

[0062] S131. Obtain robot operation data for a specific task, such as robot operation data generated when folding clothes, and obtain parameters of the pre-trained general model as initial parameters to retain the general knowledge learned during the pre-training process.

[0063] S132: Determine the fine-tuning range of the parameters related to visual feature extraction, language feature extraction, and action decoding in the initial parameters, and freeze other parameters in the initial parameters.

[0064] S133. Input the robot operation data under the specific task into the general model and output the second discretized action.

[0065] S134. Based on the second discretized action and the robot action data in the robot operation data under the specific task, calculate the second loss and its calculation expression is consistent with that in step S124, and perform back propagation to calculate the second gradient, update the parameters of the general model based on the second gradient, and dynamically adjust the learning rate to avoid overfitting.

[0066] The calculation expression for calculating the second loss is as follows:

[0067]

[0068] Among them, p is the input language prompt, a ＜i is the action sequence generated before step i, N a is the length of the action sequence.

[0069] S14. Based on the diffusion model DiT, a lightweight and scalable dedicated model is constructed, and the dedicated model is a multimodal motion denoising model. The specific dedicated model includes: a lightweight point cloud visual encoder (PointNet++) and an action generator, wherein the action generator includes: a causal self-attention layer for processing time series actions, a cross-modal attention layer for fusing point cloud features and output features of the general model, a feedforward network for nonlinear feature transformation, and a position encoder for maintaining the sequential consistency of the time series.

[0070] S15. Dedicated model training:

[0071] S151. Obtain a point cloud data set and robot operation data at a corresponding moment, wherein the point cloud data set includes point cloud data, robot state body perception data, and robot motion data, and use a lightweight point cloud visual encoder to extract point cloud features based on the point cloud data set.

[0072] S152. Based on the robot operation data at the corresponding moment, the fine-tuned general model is used to output the action sequence and task potential features.

[0073] S153, training a dedicated model based on point cloud features, action sequences and task potential features, calculating the loss of the training process, and performing back propagation to calculate the gradient, and updating the parameters of the dedicated model based on the gradient. The expression for calculating the loss is:

[0074]

[0075] Among them, π θis a dedicated model; a0 is the real action; ∈ is Gaussian noise; t is the time step; c is the conditional input, including the action sequence and task latent features from the fine-tuned general model and the point cloud features.

[0076] S2, output action sequence and task potential features:

[0077] S21. Adjust the size of the RGB image at time t1 to a preset size, such as 224×224, and use a visual encoder to extract visual features after standardization. Based on the task instructions at time t1, that is, natural language task instructions, use a large model to extract language features, that is, semantic embedding.

[0078] S22, project the visual features and language features into a unified latent space to generate the task latent features at time t1.

[0079] S23. Based on visual features and language features, a general model is used to generate a discretized action sequence according to the time step using an autoregressive mechanism. The expression is:

[0080]

[0081] in, represents the action sequence at time t1, represents the action sequence before time t1, c represents the visual features and language features, g φ (·) indicates general model processing.

[0082] S3, output robot continuous action:

[0083] S31, based on t1 to t n Real-time point cloud perception data at all times, and use lightweight point cloud visual encoder to extract real-time point cloud perception features.

[0084] S32, the real-time point cloud perception features at each moment and the action sequence and task potential features at moment t1 are input into the shared latent space for alignment and fusion, and the action at the corresponding moment is generated using the diffusion denoising mechanism, which is expressed as:

[0085]

[0086] in, Indicates t i The action at the moment and i∈[1,n], ∈ represents Gaussian noise, Indicates t i The real-time point cloud perception features at moment t are aligned and fused with the action sequence and task potential features at moment t1.

[0087] Steps S2 and S3 use a fixed window mechanism for asynchronous collaboration, specifically:

[0088] A1. Obtain the task instructions and real-time visual information at the current time t1, and use the general model to output the action sequence and task potential features at time t1.

[0089] A2. Obtain real-time point cloud perception data from time t1 to t8, and use a dedicated model to output continuous actions from time t1 to t8 based on the action sequence and task potential features at time t1.

[0090] A3. When the general model updates the output action sequence and task latent features, the dedicated model performs step A2 using the updated output action sequence and task latent features.

[0091] A4. Repeat steps A1 to A3 until a complete robot action is generated.

[0092] This embodiment also provides a robot motion generation system combining general and special models for implementing the above method, and its hardware framework is as follows: Figure 2 As shown, it includes a controller, a teleoperation device, a depth camera and a robotic arm. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process described can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0093] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A robot motion generation method combining general and special models, characterized in that: The method includes: Constructing a general model and a special model, pre-training the general model, fine-tuning parameters of the pre-trained general model, and training the special model based on the fine-tuned general model; Obtaining task instructions and real-time visual information, inputting the task instructions and real-time visual information into the fine-tuned general model, and outputting action sequences and task potential features; Real-time point cloud perception data is acquired, and the real-time point cloud perception data, the action sequence and the task potential features are input into a trained dedicated model to output continuous actions of the robot.

2. A robot motion generation method combining general and special models according to claim 1, characterized in that: The general model is constructed based on a large model of visual language.

3. A robot motion generation method combining general and special models according to claim 1, characterized in that: The specialized model described is built upon a lightweight and scalable diffusion model.

4. A robot motion generation method combining general and special models according to claim 1, characterized in that: The pre-training of the general model includes: Acquiring robot operation data, wherein the robot operation data includes language instructions, image data and robot motion data; Based on the language instructions and the image data, using the general model to extract language features and visual features, and aligning the language features and visual features; Iterate the following steps until pre-training is complete: Inputting the aligned language features and visual features into a universal model, outputting a first discretized action, and performing decoding processing on the first discretized action; Based on the decoded first discretized action and robot action data, a first loss is calculated and back-propagation is performed, a first gradient is calculated, and parameters of the general model are updated based on the first gradient.

5. A robot motion generation method combining general and special models according to claim 4, characterized in that: The fine-tuning method of the general model includes: Obtain robot operation data for a specific task and obtain the parameters of the pre-trained general model as initial parameters; Determine fine-tuning ranges of parameters involved in visual feature extraction, language feature extraction, and action decoding in the initial parameters, and freeze other parameters in the initial parameters; Inputting the robot operation data under the specific task into the general model and outputting a second discretized action; Based on the second discretized action and the robot action data in the robot operation data under the specific task, a second loss is calculated, and back propagation is performed to calculate a second gradient, and the parameters of the general model are updated based on the second gradient.

6. A robot motion generation method combining general and special models according to claim 5, characterized in that: The training of the dedicated model includes: Acquire a point cloud data set and robot operation data at a corresponding time, wherein the point cloud data set includes point cloud data, robot state body perception data, and robot motion data; Based on the robot operation data at the corresponding moment, the fine-tuned general model is used to output the action sequence and the task potential features; Based on the point cloud data set, action sequence and task potential features, a dedicated model is trained, the loss of the training process is calculated, and back propagation is performed to calculate the gradient, and the parameters of the dedicated model are updated based on the gradient.

7. A robot motion generation method combining general and special models according to claim 1, characterized in that: The general model and the special model collaborate asynchronously, and the method of the asynchronous collaboration is: Obtain the task instructions and real-time visual information at the current time t1, and use the general model to output the action sequence and task potential features at time t1; Get t1 to t n The real-time point cloud perception data at time t1 is used to output the time series from t1 to t2 based on the action sequence and task potential features at time t1. n Continuous action at all times.

8. A robot motion generation method combining general and special models according to claim 7, characterized in that: The method for outputting the action sequence and task potential features at time t1 is: Based on the visual information at time t1, the general model is used to extract visual features, and based on the task instructions at time t1, the general model is used to extract language features; Project the visual features and language features into a unified latent space to generate the task potential features at time t1; Based on the visual features and language features, a general model is used to generate a discretized action sequence according to the time step, and its expression is: in, represents the action sequence at time t1, represents the action sequence before time t1, c represents the visual features and language features, g φ (·) indicates general model processing.

9. A robot motion generation method combining general and special models according to claim 7, characterized in that: The output t1 to t n The method of continuous action at the moment is: Based on the t1 to t n Real-time point cloud perception data extraction at all times Real-time point cloud perception features; The real-time point cloud perception features at each moment are aligned and fused with the action sequence and task potential features at time t1, and the diffusion denoising mechanism is used to generate the action at the corresponding moment. The expression is: in, Indicates t i The action at the moment and i∈[1,n], ∈ represents Gaussian noise, Indicates t i The real-time point cloud perception features at moment t are aligned and fused with the action sequence and task potential features at moment t1.

10. A robot motion generation system combining general and special models, characterized in that: The system is used to implement the method according to any one of claims 1 to 9.

Images