A robot action generation method based on continuous state decomposition and related devices

Abstract

The application belongs to the technical field of embodied intelligence, and discloses a robot action generation method based on continuous state decomposition and related devices; wherein the robot action generation method based on continuous state decomposition comprises the following steps: acquiring a language instruction of a user and a scene observation of a robot; analyzing the language instruction through a large language model to obtain a key target state value; generating an action through an actor model based on the language instruction, the scene observation and the key target state value, to obtain a robot action. 2 The application can solve the problem of difficult mapping between language instructions and target actions in current robot operation, can effectively process continuous states, can generate accurate robot operation actions in new states, and improves the generalization ability and operation precision of robots in diversified tasks.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of embodied intelligence, and particularly relates to a robot action generation method based on continuous state decomposition and a related device. BACKGROUND

[0002] The manipulation task of language conditions, especially the manipulation task based on continuous target states, mainly involves two aspects of the understanding of object states by robots and the execution of accurate target actions, which requires the robot system to maintain the mapping from language instructions to accurate target states in a continuous world. However, this process often faces the following two challenges, including:

[0003] 1) The robot needs to comprehensively understand the scene, which includes information such as geometric details, scene layout, and visual appearance; in order to accurately execute the task, the robot needs to be able to identify and locate various objects in the environment, which is crucial for subsequent decision-making and action execution.

[0004] 2) The robot needs to perform increasingly fine actions to meet the desired target state, which means that the robot not only needs to understand the language instructions, but also needs to have high flexibility and accuracy to be able to respond quickly in a complex and dynamic environment.

[0005] At present, with the increase of task complexity, the robot must consider the changes and uncertainties of the environment when executing target actions to achieve higher operation accuracy; however, the existing technical solutions mainly emphasize abstract language instructions, which assume discrete object states and large range of motion, ignoring the continuity and generalization of target states. SUMMARY

[0006] The present application aims to provide a robot action generation method based on continuous state decomposition and related devices to solve one or more of the above technical problems. The technical solution disclosed in the present application is a robot operation scheme that realizes continuous state decomposition through diffusion embedding, which solves the problem of mapping difficulty between language instructions and target actions in current robot operation, can effectively handle continuous states, and generate accurate robot operation actions in new states, improving the generalization ability and operation precision of robots in diversified tasks.

[0007] To achieve the above-mentioned purpose, the present application adopts the following technical solutions:

[0008] In a first aspect of the present application, a robot action generation method based on continuous state decomposition is provided, comprising the following steps:

[0009] Obtaining the language instructions of the user and the scene observation of the robot;

[0010] obtaining a key target state value through a large language model to analyze the language instruction;

[0011] generating an action through a trained SD 2 Actor model based on the language instruction, the scene observation and the key target state value, to obtain a robot action;

[0012] The SD 2 Actor model comprises:

[0013] a feature extraction module configured to extract language features of the language instruction and scene features of the scene observation;

[0014] a feature fusion module configured to fuse the language features, the scene features and the key target state value corresponding embedding to obtain fused features;

[0015] a diffusion model configured to guide a denoising process as a condition through the fused features, and generate refined robot actions through step-by-step denoising;

[0016] The SD 2 Actor model, some learnable embeddings are initialized, corresponding embeddings are retrieved through the language instruction, and the embeddings are fused with the language features and interacted with the scene features to serve as a condition input of the diffusion model, and corresponding robot actions are predicted through the diffusion model; wherein, an iterative updating manner of supervised training is adopted, and an overall loss function comprises an orthogonalization loss and a denoising loss of the diffusion model;

[0017] The key target state value corresponding embedding is obtained by retrieval if the key target state value is contained in the SD 2 Actor model training sample set; if the key target state value is not contained in the SD 2 Actor model training sample set, the key target state value is linearly decomposed through a large language model, and a linear combination of known state embeddings is obtained through linear combination to serve as the key target state value corresponding embedding.

[0018] Further improvements of the present application are as follows:

[0019] In the feature fusion module, the step of fusing the language features, the scene features and the key target state value corresponding embedding to obtain the fused features comprises:

[0020] firstly, self-attention calculation is performed on the language features and the key target state value corresponding embedding to obtain preliminary fusion features; and secondly, cross-attention calculation is performed on the preliminary fusion features and the scene features to obtain the fused features of the corresponding state under the scene.

[0021] The further improvement of the present application is that,

[0022] The overall loss function is represented as:

[0023]

[0024] In the formula, The overall loss function is represented as: The denoising loss is represented as: The orthogonalization loss is represented as: γ0 represents the weight coefficient of the orthogonalization loss;

[0025]

[0026] In the formula, And The position, rotation and switch state loss of the target action are represented as:

[0027]

[0028] In the formula, The orthogonal loss of the embedding between different tasks is represented as: The orthogonal loss between the embeddings within the same task is represented as.

[0029] The further improvement of the present application is that,

[0030] In the calculation process of the denoising loss,

[0031]

[0032] In the formula, f, The prediction network and the diffusion model network are represented as: f is used to predict the clamp state, Used to predict the translation and rotation; o represents the scene observation; e' represents the state embedding; The translation and rotation amount at the t time step are represented as: BCE represents the classification cross-entropy loss function; a open The clamp state is represented as: the clamp state is open or closed.

[0033] The further improvement of the present application is that,

[0034] In the calculation process of the orthogonalization loss,

[0035]

[0036] In the formula, γ1, γ2 represent the weight factor; E i The i embeddings within the task are represented as: E mean The average value of all embeddings of each task is represented as: I is the unit matrix; σ(·) represents the calculation of the spectral norm.

[0037] Further improvements of the present application are,

[0038] The step of obtaining a linear combination of known state embeddings as the embedding corresponding to the key target state value by linear decomposition of the key target state value by a large language model, and then linear combination,

[0039] s novel =λ1s1+λ2s2+…+λ i s i +…+λ m s m ;

[0040] λ1+λ2+…+λ i +…+λ m =1;

[0041] In the formula, s novel represents the new target state value, s novel corresponding embedding is not in the training sample set; λ i represents the i-th coefficient obtained by decomposition with a large language model; s i represents the known i-th state value, s i corresponding embedding is in the training sample set; m is the total number of state values;

[0042] e novel =λ1e1+λ2e2+…+λ i e i +…+λ m e m ;

[0043] In the formula, e novel represents the embedding corresponding to the new target state value obtained by embedding linear combination; e i represents the known i-th state embedding, which is retrieved from the model according to s i .

[0044] Further improvements of the present application are,

[0045] The robot action a novel generated by the diffusion model is represented as:

[0046]

[0047] In the formula, a t is Gaussian noise, α t represents the denoising coefficient, ∈ θ represents the diffusion network, σ t represents Gaussian distribution, and δ represents noise intensity.

[0048] In a second aspect, the present application provides a robot action generation system based on continuous state decomposition, comprising:

[0049] a data acquisition module configured to acquire a language instruction of a user and a scene observation of a robot;

[0050] a state value analysis module configured to analyze the language instruction by a large language model to obtain a key target state value;

[0051] an action generation module configured to generate an action by using a trained SD 2 Actor model based on the language instruction, the scene observation and the key target state value, to obtain a robot action;

[0052] wherein the SD 2 Actor model comprises:

[0053] a feature extraction module configured to extract language features of the language instruction and scene features of the scene observation;

[0054] a feature fusion module configured to fuse corresponding embedding of the language features, the scene features and the key target state value to obtain fused features;

[0055] a diffusion model configured to guide a denoising process by taking the fused features as a condition to generate a refined robot action by step-by-step denoising;

[0056] the SD 2 Actor model, some learnable embeddings are initialized, corresponding embeddings are retrieved by the language instruction, and the embeddings are fused with the language features and interacted with the scene features to be input to the diffusion model as a condition, and corresponding robot actions are predicted by the diffusion model; wherein an iterative updating manner of supervised training is adopted, and an overall loss function comprises an orthogonalization loss and a denoising loss of the diffusion model;

[0057] the key target state value corresponding embedding is obtained by retrieval if the key target state value is contained in a training sample set of the SD 2 Actor model, and the key target state value corresponding embedding is obtained by retrieval if the key target state value is not contained in the training sample set of the SD 2 Actor model, the key target state value is linearly decomposed by the large language model, and a linear combination of known state embeddings is obtained by linear combination to be the key target state value corresponding embedding.

[0058] In a third aspect, the present application provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the robot action generation method based on continuous state decomposition according to any one of the first aspect of the present application.

[0059] In a fourth aspect, the present application provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the robot action generation method based on continuous state decomposition according to any one of the first aspect of the present application.

[0060] Compared with the prior art, the present application has the following beneficial effects:

[0061] In the prior art, there is no attention to the target state to generate rough spatial actions, and the problem of how to determine consistent spatial actions according to specific state information is not concerned. The present application specifically discloses a robot operation method for realizing continuous state decomposition through diffusion embedding, which enhances the attention to state features by using a learnable embedding to represent continuous target state features, and promotes accurate action generation in continuous space. In addition, for solving the generalization problem of new states, the present application uses a large language model to analyze and extract state information in the language instruction, and performs linear decomposition and linear combination (affine combination) of embedding to obtain new state features. This method has the characteristics of high efficiency and convenience, and can be generalized to the entire continuous space. The above two improved parts can improve the generalization ability and operation precision of the robot in diversified tasks. In summary, the present application uses the relationship between object states and the relationship between states and language instructions to enhance state representation and promote fine action generation. The large language model is used to analyze the state information in the language instruction, and the corresponding action is generated through the diffusion model. The present application can effectively process continuous states and generate accurate robot operation actions in new states, thereby improving the generalization ability and operation precision of the robot in diversified tasks. BRIEF DESCRIPTION OF DRAWINGS

[0062] In order to more clearly illustrate the technical solutions in the present application or the prior art, the following will briefly introduce the drawings needed in the embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0063] Figure 1 is a flowchart of a robot action generation method based on continuous state decomposition in an embodiment of the present application;

[0064] Figure 2 is an SD 2Actor model processing logic schematic diagram;

[0065] Figure 3 is the general architecture schematic diagram of the embodiment of the application, which generalizes new states through state decomposition;

[0066] Figure 4 is the training and inference architecture schematic diagram of the robot operation method of the embodiment of the application, which generalizes new states through state decomposition;

[0067] Figure 5 is the detailed structure schematic diagram of the state decomposition method in the embodiment of the application;

[0068] Figure 6 is the schematic diagram of the orthogonal loss constraint between embeddings in the embodiment of the application;

[0069] Figure 7 is the result visualization schematic diagram of the ARNOLDBenchmark in the embodiment of the application;

[0070] Figure 8 is the schematic diagram of a robot action generation system based on continuous state decomposition in the embodiment of the application. DETAILED DESCRIPTION

[0071] In order to make the purpose, technical scheme and advantages of the present application clearer, the technical scheme in the present application will be described clearly and completely below in combination with the drawings in the embodiment of the present application; obviously, the described embodiment technical scheme is a part of the embodiments of the present application, and is not all the embodiments.

[0072] Based on the technical scheme disclosed in the embodiment of the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0073] Please refer to Figure 1 The embodiment of the present application provides a robot action generation method based on continuous state decomposition, which comprises the following steps:

[0074] Step 1, obtaining the language instruction of the user and the scene observation of the robot;

[0075] Step 2, analyzing the language instruction through a large language model to obtain a key target state value;

[0076] Step 3, based on the language instruction, the scene observation and the key target state value, using the trained SD 2 Actor model to generate actions, obtain the corresponding robot action of the language instruction and the scene observation;

[0077] Wherein, the SD 2 The Actor model comprises:

[0078] The feature extraction module is configured to extract language features of the language instruction and scene features of the scene observation.

[0079] The feature fusion module is configured to fuse the language features, the scene features and the corresponding embedding of the state value to obtain fused features. In an exemplary preferred technical solution, when performing feature fusion, first, the embedding corresponding to the language features and the key target state value is subjected to self-attention calculation to obtain preliminary fused features. Then, the preliminary fused features and the scene features are subjected to cross-attention calculation to obtain the fused features of the corresponding state under the scene. In a further explanatory technical solution, if the key target state value is included in the SD 2 The training sample set of the Actor model, the embedding corresponding to the key target state value is directly obtained through retrieval. If the key target state value is not included in the SD 2 The training sample set of the Actor model, the embedding corresponding to the key target state value is directly obtained through retrieval. If the key target state value is not included in the SD

[0080] The diffusion model is configured to use the fused features as an external condition to guide the noise reduction process, and generate refined robot actions through step-by-step noise reduction.

[0081] The technical scheme disclosed in the embodiments of the present application applies state decomposition and state combination to the scheme of improving the operation performance of the robot, provides a brand-new idea for the robot to process continuous states, and through the organic combination of state decomposition and state combination, the robot can more flexibly cope with complex and changeable environments and tasks, and improve the overall performance and adaptability of the robot system. Specifically, in state decomposition, a large language model (LLM) is used to analyze the state information in the language instruction. This decomposition method not only improves the accuracy of state recognition, but also enhances the understanding ability of the robot to complex language instructions, so that the robot can more accurately execute the instructions of the user. In state combination, the state features obtained through linear combination decomposition are used to generate actions that adapt to new states. This combination method can fully utilize the information of known states to quickly generate actions that adapt to new states, thereby improving the response speed and operation precision of the robot. Through the unified framework of state decomposition and combination, the robot can better adapt to unseen or changed states, and significantly improve its generalization ability in continuous states, which means that the robot can perform outstanding performance in a wider range of scenarios and tasks, and reduce the dependence on specific environments and tasks.

[0082] In an embodiment of the present application, a robot action generation method for continuous state decomposition through diffusion embedding is provided to realize fine operation of the robot, specifically including the following steps:

[0083] Step 1, a set of demonstration data of robot operation is given, which includes scene observation, language instruction, and robot's own perception. In order to facilitate training, the key frames are extracted as training data; the main framework of the present application adopts the structure of diffusion model to capture the distribution of continuous actions. In the training stage, scene features and language features are extracted by CLIP; in order to generate fine actions, the present application models the target state of the object using a learnable embedding, and performs self-attention with the language features to obtain preliminary fusion features; then, the preliminary fusion features and the scene features are cross-attention to obtain secondary fusion features as external conditions to guide the diffusion model to generate more fine actions.

[0084] Step 2, in order to force the model to better distinguish different target states, the present application introduces an orthogonalization loss, adds a soft interval between the embeddings, and forces them to maintain a corresponding distance, so that each embedding has a unique corresponding state value, aligning the embedding space and the continuous action space; finally, the denoising loss and the orthogonalization loss function of the diffusion model are used to realize the iterative optimization of the network in the training stage.

[0085] Step 3, under the supervision of step 1 and step 2, the trained embedding and diffusion model is obtained, then when encountering an instruction containing a new state in the inference stage, the application uses a large language model to analyze and extract the corresponding target state, and further linearly decomposes the instruction state, then uses linear combination to fuse to obtain new features, which are used as new state conditions and interact with language features and scene features and input into the diffusion model, so that the diffusion model is generalized to new actions; finally, by reasonably using the decomposition and combination of the state, the correct understanding of each new state is realized in the inference stage, and the corresponding action is generated.

[0086] In a specific exemplary technical solution, in step 1, the specific steps of using the diffusion model and embedding include:

[0087] Step 1.1, a set of operation task demonstrations are given, which include language instructions l, scene observations o and agent's ontology perception c. These demonstration data usually come from actual operations of multiple tasks, through these demonstrations, the robot can learn the mapping relationship between language instructions and action execution. In order to convert these data into effective training data that the model can learn, key frames are extracted from each operation task, these key frames are time points in the scene that contain main information, they can effectively represent the changes of object state and the key steps of the task, reducing redundant data and improving training efficiency.

[0088] Step 1.2, in order to better handle the continuous action state distribution, the framework of the application adopts a diffusion model structure, which iteratively denoises the noise-added action a, which can be approximated as generating complex target data from simple noise through step-by-step denoising; moreover, the diffusion model can model the action space with continuity and high complexity, especially when facing diversified target states in tasks, it can effectively infer the action transition between different states.

[0089] Step 1.3, in order to improve the accuracy and refinement level of action generation, the application introduces a learnable embedding to model the target state of the object; unlike traditional methods, the state of the object is not just a simple descriptive feature, but a rich state information represented by a learnable embedding. These embeddings can be fused with various features of the object, including scene information, language instructions and ontology perception information, etc., and these information are input into the diffusion model as conditions.

[0090] Explanatorily, the state information and language features are self-attentioned, and after cross-attention with the scene features, the final state fusion features are obtained; the specific process is as follows:

[0091]

[0092] In the formula, represents the splicing operation, WK and W V denote the linear transformation matrix of Key and Value; K ⊕ and V ⊕ denote the linear transformation matrix of Key and Value; denote the projection of state representation to the feature space of language, composed of multi-layer MLP; finally, each group of K, V is fused with the scene, and the noise is predicted through the multi-layer transformer of diffusion model.

[0093] In one specific embodiment of the present application, in step 2, the subject framework of action generation has been obtained, in order to enhance the modeling of different target states by the model and to distinguish them, the present application adopts an orthogonalization constraint to limit the distance between the embeddings, which specifically includes the following steps:

[0094] Step 2.1, the main goal of the present application is to distinguish the states of the task, but the distance between the state embeddings within the task and between the tasks should be slightly different, in order to ensure this, the present application imposes different constraints on the states within the task and the states between the tasks. Specifically, all states belonging to the same task are taken out respectively, the inner product of the transpose of the task embedding with itself is calculated, and the difference between the result and the unit matrix is calculated, which represents the correlation between the embeddings, and the non-linear relationship is strengthened through the activation function to obtain the loss within the task.

[0095] Step 2.2, for the state calculation of different tasks, since the number of states is large, in order to simplify the network calculation, the mean of all states within the task is calculated, and the loss of state value is calculated using them;

[0096] The specific formula is as follows:

[0097]

[0098] In the formula, γ1 and γ2 are weight factors (hyperparameters) that control the influence of the loss on the training process; E i denote the i-th embedding within the task; E mean denote the mean value of all embeddings of each task; I is the unit matrix; σ(E) denotes the spectral norm of E; denote the orthogonalization loss; denote the orthogonalization loss between tasks and within tasks.

[0099] In one specific embodiment of the present application, in step 3, the state decomposition acts on each new state, the purpose of which is to construct and fuse new state representations by reasoning the combination features between states, which specifically includes the following steps:

[0100] Step 3.1, in the reasoning stage, a large language model (for example, optional ChatGPT) is used to analyze and extract the target state information in the new instruction. The large language model automatically identifies the key target state value s in the task by parsing the input natural language instruction. novel After obtaining the embedding of the target state, the next step is to perform linear decomposition of the state. Specifically, the target state in the instruction is usually not a simple, fully visible state, but needs to be approximated by known object states and environmental information; therefore, this problem is handled by decomposing s novel into a linear combination of multiple known state embeddings {s1, s2…s K}.

[0101] The specific formula is as follows:

[0102] s novel =λ1s1+λ2s2+…+λ i s i +…+λ m s m ;

[0103] λ1+λ2+…+λ i +…+λ m =1;

[0104] In the formula, s novel represents the new target state value, s novel corresponding embedding is not in the training sample set; λ i represents the i-th coefficient obtained by decomposition with a large language model; s i represents the i-th known state value, s i corresponding embedding is in the training sample set; m is the total number of state values.

[0105] Step 3.2, use a set of known state embeddings {e1, e2…e K}, these states have learned corresponding actions through the diffusion model in the training stage, by weighting combination of these known state embeddings, we get the approximate representation e novel of the target state, this weighted combination is realized by affine combination, where the weight coefficient λ i of each known state embedding is provided by the large model. The specific formula is as follows:

[0106] e novel =λ1e1+λ2e2+…+λ i e i +…+λ m e m ;

[0107] In the formula, e noveldenotes the embedding of the new goal state value obtained by embedding linear combination; e i denotes the known i-th state embedding, according to s i is retrieved from the model.

[0108] In this way, not only the information of the existing state can be utilized, but also the generation of the new goal state can be ensured to conform to the learned pattern during training.

[0109] Step 3.3, the embedding e of the new goal state is obtained by the affine combination described above novel , which further enhances the representation ability of the goal state by weighting each state feature, so that it can cover a wider state space. When the diffusion model receives this new state condition, it can predict the action a corresponding to the goal state according to the learned action generation rule novel .

[0110]

[0111] where a t is Gaussian noise, α t denotes the de-noising coefficient, ∈ θ represents the diffusion network, σ t denotes Gaussian distribution, and δ represents noise intensity.

[0112] The above method of the embodiment of the application realizes effective promotion in a new environment and task by combining the state decomposition and combination process with the action generation model.

[0113] Please refer to Figures 2 to 4 , the robot operation method based on state decomposition provided by the embodiment of the application comprises the following steps:

[0114] Step 1, the known language instruction l, scene observation o and the body perception c of the robot. We extract the action and observation of adjacent key frames as a set of training data, that is, according to the observation at the current time and the action to be predicted at the next time.

[0115] In order to fully combine the scene and instruction information, CLIP is used as the visual and language encoder to obtain respective feature encodings, and cross attention is performed thereon for feature fusion. In order to effectively model the action distribution, the diffusion model method is adopted. Specifically, the above features are taken as conditions to gradually de-noise the real noised action, and in this process, the instruction, the sampled scene feature and the action are bidirectionally interacted to obtain a more comprehensive feature. The action feature is subjected to a plurality of MLPs to obtain the corresponding predicted noise.

[0116] Step 2, in the training phase, to enhance the modeling of state information and generate refined actions, we initialize some learnable embeddings, retrieve the corresponding embeddings through state values, and fuse them with language features and interact with scene features. The specific approach is as follows:

[0117]

[0118] where ⊕ denotes the concatenation operation, W K and W V represent the linear transformation matrices of Key and Value, K ⊕ and V ⊕ represent the linear transformation matrices of Key and Value, denotes the projection of state representation to the feature space of language, composed of multiple layers of MLP. The concatenated state features are sent to the diffusion model for further refinement of action generation.

[0119] Step 3, in the inference process, referring to Figure 5 , a large language model is used to understand and extract the target state information in the new instruction. By analyzing the input natural language instruction, the language model can automatically identify and extract the key target state value s novel . After obtaining the embedding representation of the target state, we next perform linear decomposition on the state. Specifically, we represent the target state as a weighted linear combination of multiple known states {s1, s2…s m} to better model complex target states. Specifically, this process can be represented by the following formula:

[0120] s novel =λ1s1+λ2s2+…+λ m s m ;

[0121] λ1+λ2+…+λ m =1;

[0122] where λ i represents the decomposition coefficients of the state, and we make these coefficients sum to 1, so that our state combination is also in the same embedding space.

[0123] Step 4, according to {s1, s2…s m}, we retrieve a corresponding set of known state embeddings {e1, e2…e m}, which have learned corresponding state features through the diffusion model in the training phase. By linearly combining these known state embeddings according to the decomposition coefficients, we obtain an approximate representation e novel of the target state. The specific mathematical expression of this process is as follows:

[0124]

[0125] where e i is the embedding of the ith known state, λ i is the decomposition coefficient corresponding to each state embedding, representing the contribution degree of different known states to the target state. Through this affine combination, we can effectively approximate the new state e novel to provide conditional input for subsequent action generation.

[0126] Step 5, in the network optimization training process, the above state features and diffusion model are jointly trained end-to-end. At the same time, in order to increase the distinguishability of state information and the corresponding relationship with the action, for all states in the same task, referring to Figure 6 , we first calculate the inner product of the transpose of the task embedding and itself, and take the difference between the result and the unit matrix as the correlation measure between embeddings. Then, we strengthen this nonlinear relationship through the activation function, so as to obtain the loss within the task.

[0127]

[0128] Through the above steps, we ensure the effective distinction of states within and between tasks, and enable the model to better learn the features of different task states, while enhancing the model's generalization ability in multiple task environments.

[0129] Loss of combining diffusion model to model action which can be represented as:

[0130]

[0131] where, and represent the loss of the position, rotation and switch state of the target action respectively; BCE represents the classification cross-entropy loss function; f and represent the prediction network and the diffusion model network.

[0132] The average success rate is used to evaluate the proposed method, and the state decomposition model of the embodiment of the application is evaluated on ARNOLD. The dataset provides camera input from 5 perspectives, and the resolution of each input is 128x128 by default. The Franka Emika Panda manipulator with 7 degrees of freedom is used in the application, and a parallel gripper is equipped to perform tasks. The dataset contains 40 different objects and 20 diverse scenes, covering 8 tasks, and the target state of the task has different variability. Each task is divided into a training set, a test set, a novel set and an any set in the original way. The test set only contains the target state observed during the training process, the novel set contains a new, unobserved target state, and the any set contains multiple target states (including observed and unobserved target states), covering the entire operating state space. In this process, the novel and any sets are mainly used to verify the generalization ability of the model. When a task instance meets the success condition for 2 seconds, the task is considered successful. This requires that the deviation between the current state and the target state be within an acceptable threshold.

[0133] Table 1. Results of basic performance, new state performance and continuous state performance on ARNOLD benchmark

[0134]

[0135] Table 1 is the experimental results of different methods used in ARNOLD simulator. As can be seen from Table 1, whether it is basic performance, new state generalization performance or continuous state generalization performance, the state decomposition-based robot operation method proposed in the application has obvious advantages. Figure 7 is the visualization result of the method of the application on ARNOLD experiment, and the experimental result shows the effectiveness of the relationship state and action alignment ability of the application in continuous scenes. In summary, compared with other methods, the method of the application has obtained greater improvement in state generalization robot operation tasks.

[0136] The embodiment of the application discloses a robot operation method for continuous state decomposition through diffusion embedding. The technology combines diffusion models and large language models (LLMs) to achieve precise mapping of language instructions to robot actions. The technical solution includes state modeling, state decomposition and combination, action generation, and orthogonalization constraints, which can effectively process and generate transition actions for continuous states. SD 2The Actor supports zero-shot learning, does not need task-specific fine-tuning, can directly generate robot operation actions applicable to new states, has the characteristics of multi-modal fusion and efficient state decomposition and reorganization. The application technology is mainly applied to robot operation tasks, human-computer interaction and education and research fields, and is particularly suitable for diversified object processing in dynamic environments. 2 The Actor significantly improves the generalization ability and operation precision of the robot in unobserved states, reduces the dependence on large-scale training data, improves the task execution efficiency, and enhances the practical applicability of the model. The continuous state decomposition method based on diffusion embedding disclosed in the embodiment of the application is the first work that uniformly applies state decomposition and state combination to improve the robot operation performance. The state information in the language instruction is parsed by a large language model, the new state is decomposed into multiple known states, the decomposed state features are combined through an affine combination, and actions suitable for the new state are generated. The combination of the two can significantly improve the generalization ability and operation precision of the robot in continuous states, can be directly integrated into the existing robot operation system, and can be trained end-to-end through standard action generation and state prediction loss.

[0137] The following is an apparatus embodiment of the application, which can be used to perform the method embodiment of the application. For details not disclosed in the apparatus embodiment, please refer to the method embodiment of the application.

[0138] Please refer to Figure 8 In the embodiment of the application, a robot action generation system based on continuous state decomposition is provided, comprising:

[0139] A data acquisition module is configured to acquire a language instruction of a user and a scene observation of a robot;

[0140] A state value analysis module is configured to parse the language instruction by a large language model to obtain a key target state value;

[0141] An action generation module is configured to generate an action based on the language instruction, the scene observation and the key target state value by using a trained SD 2 Actor model to obtain a robot action;

[0142] The SD 2 Actor model comprises:

[0143] A feature extraction module is configured to extract language features of the language instruction and scene features of the scene observation;

[0144] A feature fusion module is configured to fuse the corresponding embeddings of the language features, the scene features and the key target state value to obtain fused features;

[0145] a diffusion model configured to generate refined robot actions by progressively denoising using the post-fusion features as conditional guidance for a denoising process;

[0146] the SD 2 In the training process of the Actor model, some learnable embeddings are initialized, and the corresponding embeddings are retrieved through language instructions, fused with language features, and interacted with scene features to serve as the conditional input of the diffusion model, and the corresponding robot actions are predicted through the diffusion model; wherein the iterative updating method of supervised training is adopted, and the overall loss function includes the orthogonalization loss and the denoising loss of the diffusion model;

[0147] The obtaining step of the embedding corresponding to the key target state value is that if the key target state value is contained in the SD 2 The training sample set of the Actor model, the embedding corresponding to the key target state value is directly obtained through retrieval; if the key target state value is not contained in the SD 2 The training sample set of the Actor model, the embedding corresponding to the key target state value is directly obtained through retrieval; if the key target state value is not contained in the SD

[0148] In an embodiment of the present application, a computer device is provided, which includes a processor and a memory, the memory is used to store a computer program, the computer program includes program instructions, and the processor is used to execute the program instructions stored in the computer storage medium. The processor can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc., which are the computing core and control core of the terminal, and are suitable for implementing one or more instructions, and are specifically suitable for loading and executing one or more instructions in the computer storage medium to implement a corresponding method flow or a corresponding function; the processor in the embodiment of the present application can be used to execute the operation of the robot action generation method based on continuous state decomposition.

[0149] In an embodiment of the present application, a storage medium, specifically a computer readable storage medium (Memory) is provided, which is a memory device in a computer device, for storing programs and data. It can be understood that the computer readable storage medium herein can include an internal storage medium in the computer device, and of course can include an extended storage medium supported by the computer device. The computer readable storage medium provides a storage space, which stores an operating system of the terminal. Moreover, one or more instructions suitable for being loaded and executed by the processor are also stored in the storage space, and the instructions can be one or more computer programs (including program codes). It should be noted that the computer readable storage medium herein can be a high-speed RAM (Random Access Memory) memory, or a non-volatile memory such as at least one disk memory. The one or more instructions stored in the computer readable storage medium can be loaded and executed by the processor to implement the corresponding steps of the robot action generation method based on continuous state decomposition in the above embodiments.

[0150] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, optical storage, etc.) containing computer-usable program code.

[0151] The present application is described with reference to the flowcharts and / or block diagrams according to the methods, devices (systems), and computer program products of the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing apparatus to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing apparatus generate a means for implementing the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions specified in one or more flows and / or blocks Figure 1 The means for implementing the functions specified in one or more flows and / or blocks.

[0152] These computer program instructions can also be stored in a computer readable memory capable of directing the computer or other programmable data processing apparatus to work in a specific manner, so that the instructions stored in the computer readable memory produce a product including instruction means, which implements the functions specified in the flowcharts and / or block diagrams. Figure 1one or more processes and / or blocks Figure 1 the function specified in the one or more blocks.

[0153] These computer program instructions can also be loaded into computer or other programmable data processing devices, so that a series of operation steps are performed on the computer or other programmable data processing devices to generate computer-implemented processes, so that the instructions executed on the computer or other programmable data processing devices provide processes for implementing the flow Figure 1 one or more processes and / or blocks Figure 1 the function specified in the one or more blocks.

[0154] Finally, it should be noted that: the above examples are only used to illustrate the technical solutions of the present application, but not to limit it, although the above embodiments of the present application have been described in detail, those skilled in the art should understand: the specific embodiments of the present application can be modified or replaced by the same, without departing from the spirit and scope of the present application, any modification or equivalent replacement, which should be covered within the scope of protection of the claims of the present application.

Claims

1. A method for robot motion generation based on continuous state decomposition, characterized by, The method comprises the following steps: obtaining a user's language instruction and a robot's scene observation; parsing the language instruction through a large language model to obtain a key target state value; Based on the language instruction, the scene observation and the key target state value, the trained SD 2 Actor model generates an action to obtain a robot action; wherein the SD 2 The actor model comprises: a feature extraction module for extracting language features of the language instruction and scene features of the scene observation; a feature fusion module for fusing the language features, the scene features, and the key target state value corresponding embedding to obtain fused features; a diffusion model for guiding the noise reduction process as a condition with the fused features as a condition, and generating refined robot actions through step-by-step noise reduction. The SD 2 In the training process of the Actor model, some learnable embeddings are initialized, corresponding embeddings are retrieved through language instructions, and the embeddings are fused with language features and interacted with scene features to serve as conditional inputs of the diffusion model. The corresponding robot action is predicted through the diffusion model. In the training process of the Actor model, some learnable embeddings are initialized, corresponding embeddings are retrieved through language instructions, and the embeddings are fused with language features and interacted with scene features to serve as conditional inputs of the diffusion model. The corresponding robot action is predicted through the diffusion model. The obtaining step of the key target state value corresponding embedding is that if the key target state value is included in the SD 2 Actor model, the key target state value corresponding embedding is directly obtained by retrieval; if the key target state value is not included in the SD 2 Actor model, the key target state value is linearly decomposed by the large language model, and then a linear combination of the known state embeddings is obtained by linear combination to serve as the key target state value corresponding embedding.

2. The method of claim 1, wherein the feature fusion module comprises the following steps: first, self-attention calculation is performed on the embedding corresponding to the language features and the key target state value to obtain preliminary fused features; and then cross-attention calculation is performed on the preliminary fused features and the scene features to obtain fused features corresponding to the state under the scene.

3. The method of claim 1, wherein the overall loss function is represented as:

4. The method of claim 3, wherein in the calculation process of the denoising loss, 5. The method of claim 3, wherein in the calculation process of the orthogonalization loss, In the formula, represents the overall loss function; represents the denoising loss; represents the orthogonalization loss; γ0represents the weight coefficient of the orthogonalization loss; wherein and respectively represent the position, rotation and switch state loss of the target action; wherein denotes the orthogonal loss embedded between different tasks; denotes the orthogonal loss between embeddings within the same task.

6. The method of claim 1, wherein in the step of linearly decomposing the key target state value through a large language model, and then obtaining a linear combination of known state embeddings through linear combination and taking it as the embedding corresponding to the key target state value, 7. The method of claim 6, wherein the method comprises: where f, denotes the prediction network and the diffusion model network, f is used to predict the gripper state, for predicting translation and rotation; o denotes the scene observation; e ′ denotes the state embedding; denotes the translation and rotation amount at time step t; BCE denotes the classification cross-entropy loss function; a open denotes the gripper state, the gripper state is open or closed. a data acquisition module for acquiring a user's language instruction and a robot's scene observation; a state value analysis module for parsing the language instruction through a large language model to obtain a key target state value; where γ1, γ2 represent weight factors; E i denotes the i-th embedding within a task; E mean denotes the average of all embeddings per task; I is the identity matrix; σ(·) denotes the computation of the spectral norm. a feature extraction module for extracting language features of the language instruction and scene features of the scene observation; a feature fusion module for fusing the language features, the scene features, and the key target state value corresponding embedding to obtain fused features; s novel = λ1s1+ λ2s2+... + λ i s i +... + λ m s m ; λ1+λ2+…+λ i +…+λ m = 1; where s novel denotes the new target state value, s novel The corresponding embedding is not in the training sample set; λ i represents the i-th coefficient obtained by decomposing the large language model; s i represents the i-th state value known, s i corresponding to the embedding in the training sample set; m is the total number of state values; e novel = λ1e1+ λ2e2+ … +λ i e i +…+λ m e m ; where e novel represents the embedding of the new target state value obtained by embedding linear combination; e i represents the known i-th state embedding, according to s i is retrieved from the model. a diffusion model for guiding the noise reduction process as a condition with the fused features as a condition, and generating refined robot actions through step-by-step noise reduction. Robot action a generated by diffusion model novel is represented as: In the formula, a t is a Gaussian noise, α t represents a de-noising coefficient, ∈ θ represents a diffusion network, σ t represents a Gaussian distribution, and δ represents a noise intensity.

8. A continuous state decomposition based robot motion generation system, characterized by, The processor executes the program to realize the method of claim 1 to 7. The computer program is executed by the processor to realize the method of claim 1 to 7. ​ An action generation module is configured to generate an action based on the language instruction, the scene observation and the key target state value by using the trained SD 2 Actor model to obtain a robot action. The SD 2 The Actor model comprises: ​ ​ ​ The SD 2 In the training process of the Actor model, some learnable embeddings are initialized, the corresponding embeddings are retrieved through language instructions, are fused with language features, and are interacted with scene features to serve as conditional inputs of the diffusion model, and corresponding robot actions are predicted through the diffusion model; wherein, an iterative updating manner of supervised training is adopted, and the overall loss function includes an orthogonalization loss and a denoising loss of the diffusion model. The obtaining step of the key target state value corresponding embedding is that if the key target state value is included in the SD 2 Actor model training sample set, the key target state value corresponding embedding is directly obtained by retrieval; if the key target state value is not included in the SD 2 Actor model training sample set, the key target state value is linearly decomposed by a large language model, and a linear combination of known state embeddings is obtained by linear combination as the key target state value corresponding embedding.

9. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, ​ 10. A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, ​

Images