A robot interaction control method and server based on diffusion strategy

Abstract

The application discloses a kind of robot interaction control method and server based on diffusion strategy, belong to robot control technical field.The existing robot control model lacks accurate understanding to the state of each time of robot future, cannot understand robot action.A kind of robot interaction control method based on diffusion strategy of the application, by constructing scene understanding model, scene interaction prediction model, diffusion strategy decision model, the change of scene is obtained by predicting robot action, and interaction scene change information;Then interaction scene change information is handled, and the control instruction of robot can be directly generated, so as to be able to accurately understand the state of each time of robot future, and can accurately understand robot action, so as to realize the control from scene end to robot action end, improve the accuracy and reliability of robot control, improve robot response speed, simplify control process, so that robot can adapt to various scenes.

Description

TECHNICAL FIELD

[0001] The application relates to a robot interaction control method and server based on a diffusion strategy and belongs to the technical field of robot control. BACKGROUND

[0002] A robot control method and device, robot, electronic device and storage medium are disclosed in Chinese Patent Application (Publication No. CN120002673A) and belong to the field of artificial intelligence. The robot control method comprises obtaining a first color depth image of a target scene at a current time and execution action information of a robot; inputting the first color depth image and the execution action information into a dynamics prediction module of a target world model to obtain a predicted scene flow of the target scene at a next time output by the dynamics prediction module; inputting the predicted scene flow and the first color depth image into a diffusion module of the target world model to obtain a second color depth image of the target scene at the next time output by the diffusion module, which is used to determine a control strategy of the robot at the next time.

[0003] The above scheme can predict the color depth image of the target scene at the next time using the world model, but the scheme mainly involves the prediction of the target scene and lacks accurate understanding of the state of the robot at each future time, cannot understand the robot action, and thus cannot directly generate the control instruction of the robot and cannot directly realize the control from the scene end to the robot action end. Therefore, the robot control process is complicated and the response efficiency of the robot is low.

[0004] The information disclosed in the background technology is only used to understand the background of the inventive concept, and thus it can include information that does not constitute the prior art. SUMMARY

[0005] In view of the above problems or one of the above problems, the purpose of the present application is to provide a robot interaction control method and server based on a diffusion strategy, which can identify the scene features involved in the task by constructing a scene understanding model, a scene interaction prediction model and a diffusion strategy decision model to obtain scene perception data; then design the robot action according to the scene perception data and predict the change of the robot action on the scene to obtain interaction scene change information; then process the interaction scene change information to directly generate the control instruction of the robot, so as to accurately understand the state of the robot at each future time and accurately understand the robot action, thereby realizing the control from the scene end to the robot action end, improving the accuracy and reliability of the robot control, improving the response speed of the robot, simplifying the control process, and enabling the robot to adapt to various scenes.

[0006] To solve the above problems or one of the above problems, the second technical solution of the present application is to provide a robot interaction control method and server based on diffusion strategy, which is designed based on interactive environment, can receive robot actions and simulate the dynamic changes of the environment through video generation mode, and predict the dynamic changes of the robot state, so that the robot decision process has the ability to predict the future state and the ability to upgrade in time and space dimensions, and can accurately understand the physical law and action, so as to accurately generate control instructions, thereby improving the robot control instruction generation ability.

[0007] To achieve one of the above purposes, the first technical solution of the present application is:

[0008] A robot interaction control method based on diffusion strategy, comprising the following steps:

[0009] Step one, obtaining task instruction information to be executed;

[0010] Step two, using a pre-created scene understanding model, based on the task instruction information, identifying the scene features involved in the task to obtain scene perception data;

[0011] Step three, using a pre-created scene interaction prediction model, designing robot actions according to the scene perception data, and predicting the changes of the scene caused by the robot actions to obtain interaction scene change information;

[0012] Step four, using a pre-created diffusion strategy decision model to process the interaction scene change information to generate control instructions for the robot.

[0013] The present application constructs a scene understanding model, a scene interaction prediction model and a diffusion strategy decision model, identifies the scene features involved in the task to obtain scene perception data, designs robot actions according to the scene perception data, predicts the changes of the scene caused by the robot actions to obtain interaction scene change information, thereby accurately recognizing the state of the robot at each future time and accurately understanding the robot actions, then processes the interaction scene change information to directly generate control instructions for the robot, thereby realizing control from the scene end to the robot action end, improving the accuracy and reliability of robot control, improving the response speed of the robot, simplifying the control process, and enabling the robot to adapt to various scenes.

[0014] As a preferred technical measure:

[0015] Step two, using a pre-created scene understanding model, based on the task instruction information, identifying the scene features involved in the task to obtain scene perception data, the method is as follows:

[0016] According to the task instruction information, collect robot operation expert data and Internet video data about the task scene;

[0017] Process the robot operation expert data and Internet video data to obtain a plurality of historical observation data;

[0018] Train the world model using the plurality of historical observation data, and the training target is to minimize the training loss amount, so that the world model can learn the general physical law of environmental change when the robot performs specific task operation, and the generalization ability of the world model is transferred to the robot operation scene to obtain a scene understanding model;

[0019] Use the scene understanding model to identify the scene features involved in the task to obtain scene perception data.

[0020] As a preferred technical measure:

[0021] Step three, using the pre-created scene interaction prediction model, according to the scene perception data, designing robot action, and predicting the change of robot action to the scene, the method for obtaining interaction scene change information is as follows:

[0022] According to the scene perception data, the robot action is designed to obtain the action information to be executed by the robot; At the same time, the historical observation information is obtained, which includes observation image data and robot joint angle state;

[0023] Based on the action information, observation image data and robot joint angle state, the scene understanding model is trained to form a scene interaction prediction model;

[0024] Using the scene interaction prediction model, the change of robot action to the scene is predicted to obtain the interaction scene change information.

[0025] As a preferred technical measure:

[0026] The method for training the scene understanding model based on the action information, observation image data and robot joint angle state to form a scene interaction prediction model is as follows:

[0027] Based on a plurality of operation tasks, the robot is remotely operated to obtain an original training data set; The original training data set includes: observation image data of the robot, action information, robot joint angle state and text instructions of the corresponding task;

[0028] Sampling the original training data set to obtain a generation target, which includes a plurality of video frames and robot state;

[0029] Using an image encoder, a plurality of video frames are encoded into a hidden space representation;

[0030] The hidden space representation and the robot state are forwardly added with noise for multiple time steps to train the noise network, and two trained noise networks are obtained, including noise network one and noise network two;

[0031] The hidden space representation and the robot state are converted into video pure noise and state pure noise, and are input into the two trained noise networks to perform denoising for multiple time steps, and finally generate a noise-free predicted video hidden space representation and a predicted robot state quantity for representing the prediction of the robot and the environment;

[0032] The predicted video hidden space representation is restored into a clear video frame image through the decoder, so as to complete the construction of the scene interaction prediction model and realize the understanding of the dynamic physical law of the robot interacting with the environment.

[0033] As a preferred technical measure:

[0034] The method for training the two noise networks by forwardly adding noise to the hidden space representation and the robot state for multiple time steps is as follows:

[0035] Based on the hidden space representation, the policy parameter and the standard normal distribution noise one, a hidden space forward noise expression is constructed;

[0036] Based on the robot state, the policy parameter and the standard normal distribution noise two, a state forward noise expression is constructed;

[0037] The policy parameter is a parameter for controlling the noise intensity in the diffusion process to change with time;

[0038] Based on the hidden space forward noise expression, a noise network one is constructed for fitting the standard normal distribution noise one actually added at each time step;

[0039] Based on the state forward noise expression, a noise network two is constructed for fitting the standard normal distribution noise two actually added at each time step;

[0040] A loss function is set to jointly train the noise network one and the noise network two, and the training target is to minimize the loss quantity, so as to obtain the two trained noise networks.

[0041] As a preferred technical measure:

[0042] Step four, the method for generating the control instruction of the robot by using the diffusion policy decision model created in advance is as follows:

[0043] According to the task instruction information, the historical observation information and the text instruction are obtained;

[0044] The scene interaction prediction model is trained based on historical observation information and text instructions to form a diffusion strategy decision model;

[0045] The diffusion strategy decision model is used to generate noise-free video latent space representations and predicted robot actions for a plurality of time steps in the future;

[0046] The predicted video frame latent space representations are restored into clear video frame images by the decoder;

[0047] The predicted robot actions are converted into control instructions for controlling the robot target joint angle position.

[0048] As a preferred technical measure:

[0049] The method for training the scene interaction prediction model based on historical observation information and text instructions to form a diffusion strategy decision model is as follows:

[0050] Based on the video generation model U-Net, an action head generation unit is set up;

[0051] Each layer of down-sampling and up-sampling in the action head generation unit takes the interaction scene change information output by the scene interaction prediction model as input condition, and uses the understanding of the scene interaction prediction model for future state changes to generate corresponding actions;

[0052] The historical observation information is input into the scene interaction prediction model to obtain the interaction scene change information;

[0053] The interaction scene change information and the text instructions are input into the action head generation unit to obtain the robot actions, forming a diffusion strategy decision model with a parallel generation architecture.

[0054] As a preferred technical measure:

[0055] The method for inputting the interaction scene change information and the text instructions into the action head generation unit to obtain the robot actions is as follows:

[0056] The interaction scene change information is obtained, which includes video frame images predicted for the future;

[0057] The video frame images are encoded into latent space representations by using an image encoder;

[0058] The latent space representations and the text instructions are forwardly added with noise for a plurality of time steps to train the noise network, obtaining two trained noise networks, including noise network three and noise network four;

[0059] The latent space representation and the text instruction are converted into video pure noise and instruction pure noise, and are input into the two trained noise networks, de-noising is performed for multiple time steps, and finally a noise-free predicted video latent space representation and a predicted robot action are generated.

[0060] To achieve one of the above purposes, the second technical solution of the application is:

[0061] A robot interaction control method based on a diffusion strategy includes the following contents:

[0062] Collect robot operation expert data and Internet video data, and train the world model, so that the world model can learn the general physical law of environmental change when the robot performs specific task operations, and be used to migrate the generalization ability of the world model to the robot operation scene.

[0063] Collect robot expert data for specific tasks, and construct a scene interaction prediction model; the scene interaction prediction model is used to understand the physical law of robot action and environmental interaction, and its input is historical observation images, robot joint angle states and robot actions to be executed, and its output is a video of future environmental dynamic changes, to represent the action result of the robot and the environment.

[0064] The scene interaction prediction model is loaded into the diffusion strategy architecture and coupled with an action generation unit to form a diffusion strategy decision model, which is used to generate robot actions.

[0065] Each layer of the action generation unit receives the vector features output by the corresponding layer of the scene interaction prediction model, uses the understanding of the robot action and environmental change physical law of the scene interaction prediction model, and outputs the robot action in combination with the text instruction.

[0066] The scene interaction prediction model and the diffusion strategy decision model are alternately trained, and finally a diffusion strategy decision model capable of responding to human task instructions, predicting future environmental dynamic changes, and generating robot control instructions is obtained.

[0067] The application is designed based on an interactive environment, can receive robot actions and simulate the dynamic changes of the environment through video generation, and predict the dynamic changes of the robot state, so as to generate synthetic data for robot control. At the same time, the video generation and action generation of the application are performed in parallel, which is different from the existing control architecture performed in sequence, can effectively save interaction time, and improve the response efficiency of the robot.

[0068] Further, the scene interaction prediction model of the present application can more deeply understand the dynamic physical law of the robot ontology state, action and environmental interaction, so that the robot can efficiently understand the changes caused by its own action to the environment, thereby serving as a simulator to improve the efficiency of data generation and reduce the number of interactions of the robot in the actual environment; and through interaction with the diffusion strategy decision model, autonomous learning and self-improvement are carried out to improve the learning efficiency, so that long-term planning and prediction can be carried out to improve the accuracy and robustness of execution.

[0069] Further, the diffusion strategy decision model of the present application is an end-to-end embedded strategy architecture that can be directly applied to the decision-making of robot operation, so that the robot decision-making process has the ability to predict future states, and has the ability to upgrade in time and space dimensions, and can accurately understand physical laws and actions, thereby accurately generating control instructions to improve the control instruction generation ability of the robot.

[0070] To achieve one of the above purposes, the third technical solution of the present application is:

[0071] A server comprising:

[0072] one or more processing units;

[0073] a storage device for storing one or more programs;

[0074] When the one or more programs are executed by the one or more processing units, the one or more processing units implement the above-mentioned diffusion strategy-based robot interaction control method.

[0075] Compared with the prior art, the present application has the following beneficial effects:

[0076] The present application constructs a scene understanding model, a scene interaction prediction model and a diffusion strategy decision model to identify the scene features involved in the task and obtain scene perception data; then, according to the scene perception data, robot actions are designed and the changes of the robot actions to the scene are predicted to obtain interaction scene change information; then, the interaction scene change information is processed to directly generate control instructions for the robot, so that the future state of the robot at each time can be accurately known and the robot action can be accurately understood, thereby realizing control from the scene end to the robot action end, improving the accuracy and reliability of robot control, improving the response speed of the robot, simplifying the control process, and enabling the robot to adapt to various scenes.

[0077] Furthermore, the present application is designed based on an interactive environment, which can receive robot actions and simulate the dynamic changes of the environment through a video generation method, and predict the dynamic changes of the robot ontology state, thereby generating synthetic data for the robot to learn.

[0078] Further, the scene interaction prediction model of the present application can more deeply understand the dynamic physical law of the robot ontology state, action and environment interaction, so that the robot can efficiently understand the changes caused by its own action to the environment, thereby serving as a simulator to improve the efficiency of data generation and reduce the number of interactions of the robot in the actual environment; and through interaction with the diffusion strategy decision model, autonomous learning and self-improvement are carried out to improve the learning efficiency, so that long-term planning and prediction can be carried out to improve the accuracy and robustness of execution.

[0079] Further, the diffusion strategy decision model of the present application is an end-to-end embedded strategy architecture, which can be directly applied to the decision of robot operation, so that the robot decision process has the ability to predict future states, and has the ability to upgrade in time and space dimensions, and can accurately understand the physical law and action, so that control instructions can be accurately generated to improve the control instruction generation ability of the robot. BRIEF DESCRIPTION OF DRAWINGS

[0080] Figure 1 It is a flowchart of the robot interaction control method of the present application;

[0081] Figure 2 It is a schematic diagram of the scene interaction prediction model of the present application;

[0082] Figure 3 It is a schematic diagram of the interaction between the scene interaction prediction model and the diffusion strategy decision model of the present application;

[0083] Figure 4 It is a schematic diagram of training the scene interaction prediction model of the present application;

[0084] Figure 5 It is a schematic diagram of the diffusion strategy decision model of the present application;

[0085] Figure 6 It is a schematic diagram of using the diffusion strategy decision model of the present application for real machine execution. DETAILED DESCRIPTION

[0086] In order to enable personnel in the art to better understand the present application scheme, the technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor should be within the scope of protection of the present application. The present application covers any substitution, modification, equivalent method and scheme defined by the claims within the essence and scope of the present application.

[0087] As shown in Figure 1 The first embodiment of the robot interaction control method based on the diffusion strategy of the application is as follows:

[0088] A robot interaction control method based on a diffusion strategy comprises the following steps:

[0089] Step one, obtain the task instruction information to be executed;

[0090] Step two, use the pre-created scene understanding model to identify the scene features involved in the task based on the task instruction information, and obtain the scene perception data;

[0091] Step three, use the pre-created scene interaction prediction model to design the robot action according to the scene perception data, and predict the change of the scene caused by the robot action, and obtain the interaction scene change information;

[0092] Step four, use the pre-created diffusion strategy decision model to process the interaction scene change information, and generate the control instruction of the robot.

[0093] The second embodiment of the robot interaction control method based on the diffusion strategy of the application is as follows:

[0094] A robot interaction control method based on a diffusion strategy comprises the following steps:

[0095] Step one, obtain the task instruction information to be executed through the pre-created task acquisition model;

[0096] Step two, use the pre-created scene understanding model to identify the environment and objects involved in the task based on the task instruction information, and obtain the environment perception data;

[0097] Step three, use the pre-created scene interaction prediction model to design the robot action according to the environment perception data, and predict the change of the environment and objects caused by the robot action, and obtain the interaction simulation information;

[0098] Step four, use the pre-created diffusion strategy decision model to process the interaction simulation information, and generate the control instruction of the robot.

[0099] The third embodiment of the robot interaction control method based on the diffusion strategy of the application is as follows:

[0100] A robot interaction control method based on a diffusion strategy comprises the following steps:

[0101] The first step is to collect robot operation expert data and Internet video data, train the world model, enable the world model to learn the general physical law of environmental change when the robot performs specific task operation, and migrate the generalization ability of the world model to the robot operation scene.

[0102] The second step is to collect robot expert data of specific tasks to construct a scene interaction prediction model.

[0103] The third step is to load the scene interaction prediction model constructed in the second step into the diffusion strategy architecture, and couple the action generation unit to form a diffusion strategy decision model for generating robot actions.

[0104] Each layer of the action generation unit receives the vector features output by the corresponding layer of the scene interaction prediction model, uses the understanding of the physical law of robot action and environmental change of the scene interaction prediction model, and outputs the robot action combined with the text instruction.

[0105] The fourth step is to alternately train the scene interaction prediction model in the second step and the diffusion strategy decision model in the third step, and finally obtain a diffusion strategy decision model capable of responding to human task instructions, predicting future environmental dynamic changes, and generating robot control instructions.

[0106] The present application is based on the design under the interactive environment, which can receive robot actions and simulate the dynamic changes of the environment through video generation method, and predict the dynamic changes of the robot state (such as joint angle true value), so as to generate synthetic data for robot learning.

[0107] The training of the scene interaction prediction model enables the model to more deeply understand the dynamic physical law of the interaction between the robot state, action and environment; the diffusion strategy decision model is an end-to-end embedded strategy architecture, which can be directly applied to the decision of robot operation to improve the robot's control instruction generation ability.

[0108] In the embodiment, the construction method of the scene understanding model is as follows:

[0109] Based on the task instruction, large-scale robot operation expert data and Internet video data are collected.

[0110] The world model is trained by using expert data and Internet video data. The training target is to minimize the training loss amount, so that the world model can learn the general physical law of environment change when the robot performs specific task operation, and then the model generalization ability is migrated to the robot operation scene, so as to obtain the scene understanding model and realize the construction of the scene understanding model.

[0111] As shown in Figure 2 , in the embodiment, the construction method of the scene interaction prediction model is as follows:

[0112] The learning process of the scene interaction prediction model is similar to the process of gradually constructing the understanding of the physical law of the external environment by the human brain (for example, holding a water bottle, the water bottle will not fall, and the water bottle will fall freely when the hand is loosened). Therefore, the scene interaction prediction model predicts the dynamic change of the object and the environment caused by the robot action in the form of video output, and the accuracy of the prediction reflects the accuracy of the model's understanding of the physical law of the environment. At the same time, the understanding of the physical law by the scene interaction prediction model is a general understanding, so a good scene interaction prediction model should show good generalization performance in unseen scenes.

[0113] The scene interaction prediction model is constructed based on the scene understanding model or the world model, which is a real world simulator. The simulator can receive environment state (such as image frame, robot body joint angle state, etc.), action (such as robot action, camera motion, etc.) and text instruction, and then output the result of the dynamic change of the environment in the form of video as the corresponding response to the action or text instruction. From the modeling point of view, first, define an observation space , an action space and an instruction space , corresponding to the three possible input modalities mentioned above.

[0114] The goal of the scene interaction prediction model is to predict a set of future video frames and the state of the robot at the future time , which is expressed as follows:

[0115] ,

[0116] Where represents video frames, represents robot states.

[0117] These video frames can effectively capture the dynamic change of the environment caused by the action. Therefore, the training of the scene interaction prediction model can be defined as the training of an observation prediction model , where represents the state of the robot at the time to history information of the time point; represents robot action sequence of the time point, including L actions, corresponding to L future video frames and states; represents the text instruction from the first time point. However, in actual applications, the history information of the last time points can be selected to reduce the learning difficulty of the model, and experiments have proved that time point data is sufficient to obtain a better environment state representation.

[0118] In addition to single-step interaction with the diffusion strategy decision model, the most significant advantage of the scene interaction prediction model is that it can perform long-term interaction. As shown in Figure 3 , based on the previous prediction results, sampling is performed from the observation prediction model , and then the next round of interaction is performed based on the most time point information in the sampling. This process can generate high-fidelity synthetic data for robot learning, significantly expanding the size of the robot dataset for large-scale base model training. When training the scene interaction prediction model with large-scale data, the model structure remains unchanged while having sufficient scene generalization ability, task generalization ability, action generalization ability, and cross-ontology generalization ability.

[0119] In this embodiment, the interaction mode of the scene interaction prediction model is defined as , and the text instruction is omitted (note: in actual applications , a full 0 vector will be set as input), and the original intention is to make the model focus on understanding the changes caused by robot actions to the environment and its own state. The architecture corresponding to the scene interaction prediction model is shown in Figure 4 . This model uses U-Net as the backbone network, and its input includes: the last time point history image, the last n time point robot history state, and the future time point action; the output includes: the future time point video frame, reflecting the execution result of the action in the environment.

[0120] Unlike existing world models that only output future videos, the scene interaction prediction model includes a video generation model and a state header generation unit, which simultaneously predicts the future state of the robot using a state header generation unit. More importantly, the video generation model and the state header generation unit use a parallel mode, as shown by the light blue arrow in Figure 4 , the output of each intermediate layer of the video generation model is used as the input of the corresponding layer of the state header generation unit, and both modules generate simultaneously, saving inference time.

[0121] The training objective of this scenario interaction prediction model is to minimize a joint diffusion loss. Specifically, firstly, data can be collected from teleoperations of the robot performing different tasks (e.g., folding clothes, wiping tables, washing dishes) as the original training dataset. This dataset includes: observed images of the robot, robot actions, robot states, and corresponding text instructions for the tasks. The overall distribution of this dataset is represented as follows: After training begins, historical observation information is first sampled from the dataset. and robot movements , express The robot's movements, and The goal of time generation ,in express One video frame, express Each robot state is then analyzed. The observed video is then encoded into a latent space representation. ,in , For the image encoder, sample another time step. Representing the latent space of video and robot status conduct The positive noise addition at each time step can be expressed as follows:

[0122]

[0123] .

[0124] in: This represents the result after adding noise at the k-th step to the latent space representation of a video frame; This represents the result of adding noise to the robot's state vector at the k-th step; Represents a standard normal distribution; Equivalent to the latent space representation of the original data ; , From linearly increase to 0.02 is used as a strategy parameter to control the change of noise intensity over time during the diffusion process; It is standard normal distribution noise 1, which represents the standard normal distribution noise with respect to the observed video; It is standard normal distribution noise 2, which represents the standard normal distribution noise with respect to the robot's state.

[0125] Therefore, the goal is to train two noise networks to accurately fit the actual noise added at each time step k. and Two noise networks include noise network one about observed video and noise network two about robot state The joint training objective is to minimize the loss function as follows The expression is as follows:

[0126]

[0127]

[0128] wherein, represents expectation; represents a discrete uniform distribution between [1, K]; represents the maximum number of noise addition steps; represents the diffusion loss amount of predicted video noise, represents the diffusion loss amount of state noise.

[0129] In inference, based on the current historical observation and robot action instruction The trained two noise networks are used to generate future video frames and robot states, and the generation process is actually a reverse denoising process, which can be expressed as:

[0130]

[0131]

[0132]

[0133] ,

[0134] wherein, represents the result of the k-1 step in the reverse denoising process of the video frame hidden space representation from pure noise; represents the result of the k-1 step in the reverse denoising process of the robot state vector from pure noise.

[0135] The and input in the form of pure noise are denoised for K steps, and finally the noise-free predicted video hidden space representation and the predicted robot state quantity are generated. The predicted video frame image can be restored into a clear video frame image through the decoder , and the expression is as follows:

[0136] .

[0137] Because the generation of videos represents a prediction of the future state of robots and the environment, this predictive ability reflects, on the one hand, the robot's understanding of the dynamic physical laws of environmental interaction.

[0138] The method for constructing the diffusion strategy decision model is as follows:

[0139] For humans, understanding the physical world helps them make flexible and efficient behavioral decisions in daily life. Therefore, how to leverage the understanding of spatial physical laws by scene interaction prediction models to empower robot decision-making processes is a key challenge in current research on general artificial intelligence. To address this challenge, this embodiment proposes a diffusion strategy decision-making model (see...). Figure 5 The architecture consists of a world model and an action head generation unit built on the video generation model U-Net.

[0140] Each downsampling and upsampling layer in the action head generation unit uses the output features of the corresponding U-Net layer in the world model as a condition. Figure 5 (Light blue arrow in the image). The core concept of this design is that, during the motion generation process, the denoising at each time step can fully utilize the internal features of each layer of the world model that predict future state changes (i.e., the understanding of spatial physical laws) to enhance the rationality and accuracy of the robot's motion output. In addition, this architecture can realize the parallel generation of video and motion, solving the time-consuming problem of sequential generation of video and motion in existing methods.

[0141] In this embodiment, the working mode of the diffusion strategy decision model is defined as "decision mode" (distinguished from the "interaction mode" in the scenario interaction prediction model). In this mode, the input of the diffusion strategy decision model includes: historical observation information. and text commands The corresponding prediction model can be expressed as Note that because of the action The target value is no longer used as a conditional input; for ease of understanding, it is omitted (set to 0 in practical applications). In this mode, the diffusion policy decision model generates predictions about the future based on current observations and textual instructions (e.g., task description: stack the three wooden blocks in front of you in the order of red, yellow, and green). Similarly, two diffusion policy decision models can be trained to generate future videos and control actions, respectively.

[0142] Specifically, historical observation information is sampled from the dataset. and text instructions (For example: stack the three wooden blocks in front of you in the order of red, yellow, and green), and Video frame generation target at any time Action-generated targets . Wherein represents video frames, represents robot action instructions. First, the observed video is encoded into the hidden space representation , wherein , is an image encoder. Then, a time step is sampled , and the robot action is subjected to forward noise addition for k time steps, which can be respectively represented as:

[0143]

[0144] .

[0145] Wherein: represents the result of the k-th step noise addition of the video frame hidden space representation; represents the result of the k-th step noise addition of the robot action vector; is equivalent to the hidden space representation of the original data ; , can be linearly increased from 0.01 to 0.02 and used as a control strategy parameter for the noise intensity change over time in the diffusion process; is a standard normal distribution noise three, which represents a standard normal distribution noise about the observed video; is a standard normal distribution noise four, which represents a standard normal distribution noise about the robot action. Here, and in the scene interaction prediction model are consistent in definition.

[0146] Therefore, the goal is to train two noise networks to accurately fit the actual noise and added at each time step k; the two noise networks include noise network three about the observed video and noise network four about the robot action.

[0147] The expression of noise network three is as follows:

[0148]

[0149] The expression of noise network four is as follows:

[0150]

[0151] The embodiment directly embeds the world model intoFigure 4 the diffusion strategy decision model, so here and the scene interaction prediction model are the same network, except that the action is no longer an input condition, so it is omitted for ease of understanding. The training objective of the entire embedded network is to minimize the loss function as follows:

[0152]

[0153]

[0154] wherein, represents the diffusion loss of the predicted video noise, represents the diffusion loss of the action noise.

[0155] During inference, based on the current historical observation and the text instruction , the two trained noise networks can be used to generate future video frames and robot states. The generation process is actually a reverse denoising process, which can be represented as:

[0156]

[0157]

[0158]

[0159] ,

[0160] wherein, represents the result of the k-1 step in the reverse denoising process from pure noise for the video frame hidden space representation; represents the result of the k-1 step in the reverse denoising process from pure noise for the robot action vector.

[0161] The and input in the form of pure noise are denoised for K steps, and finally the noise-free future L time step video hidden space representation and the predicted robot action are generated. The predicted video frame hidden space representation can be restored into a clear video frame image through the decoder , and its expression is as follows:

[0162] ,

[0163] The predicted action is the robot target joint angle position, which can be input to the underlying controller to realize the movement of the robotic arm.

[0164] The diffusion strategy decision model in this embodiment can also be constructed based on the trained scene interaction prediction model. The diffusion strategy decision model no longer receives specific execution actions, but instead receives text instructions, because actions need to be generated and text instructions are issued by humans. The diffusion strategy decision model generates future videos according to the text instructions. Since the scene interaction prediction model has been trained, the diffusion strategy decision model can fully understand the robot behavior actions, and therefore can further optimize action generation. Just like humans in the process of action, they fully understand their own limb movements and what state they can achieve.

[0165] In this embodiment, the backbone architecture of the world model is constructed based on the video generation model U-Net. The construction principle can be directly applied to another mainstream video generation architecture DiT, and the specific contents include the following:

[0166] Since the autoregressive model Transformer generates images or videos by frame-by-frame generation, the generation of each frame depends on the result of the previous frame, which means that the generation process is linear and there is cumulative error. Due to the characteristics of frame-by-frame generation, it is difficult to guarantee the temporal consistency. Especially when generating long time series, the error may accumulate step by step, causing objects to disappear or motion to jump, which makes the autoregressive model perform poorly when generating dynamic scenes.

[0167] Therefore, the world model of this embodiment is based on the video generation model U-Net or the video generation architecture DiT, and adopts a global attention mechanism for construction, so that multiple frames of images are generated at the same time, especially having an advantage in spatiotemporal modeling. The video generation model U-Net can establish global dependency between pixels at different positions through spatial self-attention mechanism, effectively capturing spatial information. In addition, the video generation model U-Net can also combine the temporal attention mechanism to model the attention mechanism between frames in the time dimension, ensuring the temporal consistency of the video. The video generation architecture DiT adopts global self-attention mechanism in time and space, which can model spatiotemporal information in a global range, and can maintain high-quality images while avoiding the temporal and spatial errors caused by frame-by-frame generation.

[0168] The environmental change law generated by the robot in the process of work is extremely complex, which involves the motion change of the robot itself and the motion change of the object caused by the interaction (contact, grabbing, placing, etc.) with different categories of objects, especially the complex interaction of flexible objects. The video generation capability of the world model can capture such dynamic changes, and thus reflect the understanding of the model for the spatial dynamic physical law.

[0169] The limitations of the autoregressive model Transformer in generating frame-by-frame videos are that when dealing with complex interactions between objects, there are problems of inconsistent timing and loss of spatial information. The video generation model U-Net and DiT effectively avoid this problem by generating all video frames in parallel and using a global self-attention mechanism, which better ensures temporal consistency and can capture complex physical laws in space.

[0170] The fourth embodiment of the robot interaction control method based on the diffusion strategy of the application is as follows:

[0171] A robot interaction control method based on a diffusion strategy includes the following steps:

[0172] Step 1: World model pre-training, which includes the following:

[0173] Collect large-scale robot operation expert data and Internet video data, and train or fine-tune a video generation model The training goal is to minimize the , so that the model can learn the general physical laws of environmental changes when the robot performs specific task operations, and then transfer the generalization ability of the video generation model to the robot operation scene.

[0174] Step 2: Scene interaction prediction model training, which includes the following:

[0175] Collect robot expert data for specific downstream tasks to train the scene interaction prediction model. In this mode, the input of the scene interaction prediction model is the historical observation information (images and robot joint angle state) and the action to be performed by the robot, and the model output is the video of the future environmental dynamic change and the future state of the robot.

[0176] For example: In the task of folding clothes, the current observation is that the robot has grabbed the right sleeve of a piece of clothing, and a series of action instructions will make the robot move to the left. The generated video should reflect this movement and the sleeve of the clothes will move with the robot.

[0177] The model's prediction of the future reflects the results of the interaction between the robot's action and the environment, so the model's prediction ability reflects its understanding of the physical laws of the interaction between the robot's action and the environment. Thus, the scene interaction prediction model learns what kind of environmental changes its own actions will cause and what state it will reach, achieving an understanding of its own action and environmental interaction.

[0178] Step 3: Diffusion strategy decision model training, which includes the following:

[0179] The second step of the trained scene interaction prediction model is loaded into the architecture of the diffusion strategy decision model, and the training of the robot action generation is continued. At this time, the input of the world model includes: historical observation information (image and robot joint angle state) and text instruction (such as: stack the three wooden blocks in front of you in the order of red, yellow and green), and the output is the video prediction frame and the robot action instruction for the future environmental change.

[0180] The action head generation unit of the model is generated in parallel with the scene interaction prediction model to save inference time, that is, each layer of the action head generation unit receives the vector features output by the corresponding layer of the scene interaction prediction model, fully utilizes the understanding of the physical law of the robot action and the environmental change learned by the scene interaction prediction model in the second step of training, and further optimizes the output of the strategy to the robot action.

[0181] In the fourth step, the scene interaction prediction model and the diffusion strategy decision model are alternately trained, and finally a diffusion strategy decision model that can respond to human task instructions, predict future environmental dynamic changes, and generate robot control instructions is obtained, so that the diffusion strategy decision model has the ability to generate future frames according to language instructions and can accurately understand the influence of robot action on the physical world.

[0182] As shown in Figure 5 , Figure 6 The fifth embodiment of the robot interaction control method based on the diffusion strategy of the present application is shown in the following:

[0183] A robot interaction control method based on a diffusion strategy, which uses a trained diffusion strategy decision model for real machine execution, includes the following contents:

[0184] Receive historical observation information, text instructions and noise, and input them into the diffusion strategy decision model for processing, and finally output control instructions to control the robot arm. At the same time, the model's prediction of the future can be visualized from the inference video. A specific embodiment of the application for controlling the robot to control the robot arm is as follows:

[0185] The method based on the present application controls the robot arm, which includes the following contents:

[0186] The robot first acquires the historical observation information about the current scene at the last n time points through the camera; at the same time, the historical state information is obtained through the encoders of the robot joints;

[0187] The robot receives the text instruction: stack the wooden blocks in the order of red, yellow and green;

[0188] According to the generation mechanism of the diffusion strategy decision model, a set of noise video frames needs to be input for the generation of future videos, and a set of action noises needs to be input for the generation of future actions;

[0189] The generation process is the denoising process introduced in the construction process of the diffusion strategy decision model, which includes the following contents:

[0190] The trained noise network is applied to gradually denoise the input standard normal distribution noise to generate clear future images and future actions (the action is the position of the joint angle of the robot arm);

[0191] Finally, the position of the joint angle of the robot arm is sent to the underlying controller to realize the movement of the robot arm.

[0192] A specific embodiment of the application for controlling a robot to sort goods is as follows:

[0193] The method of the application for controlling a robot to sort goods includes the following contents:

[0194] Task: The robot needs to put the items on the corresponding shelf position;

[0195] Understanding: The robot needs to observe the environment through the camera and sensor, identify the position, shape, shelf position, space, etc. of the items;

[0196] Prediction: The robot needs to predict whether the target object will fall off when grasped from the current angle, whether it will touch the surrounding objects during the movement to the shelf to cause the object to overturn, etc.

[0197] Decision: Based on the above prediction, the diffusion strategy decision model outputs the control instruction for safe and efficient grasping and placing;

[0198] A specific embodiment of the application for controlling a robot to take food materials is as follows:

[0199] The method of the application for controlling a robot to take food materials includes the following contents:

[0200] Household scenario: taking food materials from the refrigerator;

[0201] Task: Take a box of yogurt placed behind the refrigerator;

[0202] Understanding: The robot needs to identify the category, shape, and placement position of each layer of items in the refrigerator, and perceive which are fragile and which are liquids, etc.

[0203] Prediction: It needs to predict whether the current refrigerator door opening space is sufficient for action execution when taking the yogurt behind. If the robot arm directly reaches in to grasp the yogurt, will it knock down other objects? When the yogurt is taken out, will the action knock over the egg box in front?

[0204] Decision: Based on the above prediction, the diffusion strategy decision model outputs control instructions for safe and efficient grabbing and placing.

[0205] Therefore, the diffusion strategy decision model of the present application enables the robot decision-making process to have the ability to predict future states, and the ability to upgrade in time and space dimensions (such as video generation), and can accurately understand physical laws and actions, so as to accurately generate control instructions.

[0206] The scene interaction prediction model of the present application enables the robot to efficiently understand the changes caused by its own actions to the environment, thereby serving as a simulator to improve the efficiency of data generation and reduce the number of interactions of the robot in the actual environment; and through interaction with the diffusion strategy decision model, autonomous learning and self-improvement are carried out to improve the learning efficiency, so as to make long-term planning and prediction and improve the accuracy and robustness of execution.

[0207] A server embodiment applying the method of the present application:

[0208] A server, comprising:

[0209] one or more processing units;

[0210] a storage device for storing one or more programs;

[0211] When the one or more programs are executed by the one or more processing units, the one or more processing units implement the above-mentioned diffusion strategy-based robot interaction control method.

[0212] The storage device is an internal memory or an external memory or a cache memory or other special memory. The processing unit has signal processing capability and can be a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a ready-to-program gate array or other programmable logic device.

[0213] A device embodiment applying the method of the present application:

[0214] An electronic device is provided with a computer readable storage medium, and the computer readable storage medium stores a computer program, which, when executed by a processing unit, implements the above-mentioned diffusion strategy-based robot interaction control method.

[0215] The computer readable storage medium refers to a physical carrier capable of storing computer-recognizable data, instructions or programs, which needs to meet the core characteristics of "being readable by a computer" (i.e. data exists in the form of electric, magnetic, optical signals, which can be converted into binary information that can be processed by a computer through corresponding devices). The physical carrier is a magnetic storage medium, an optical storage medium, a semiconductor storage medium or other storage medium.

[0216] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, a computer program product. Therefore, the present application can adopt a completely hardware embodiment, a completely software embodiment, an embodiment combining software and hardware aspects. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer usable storage media (including but not limited to disk storage, optical storage, etc.) containing computer usable program code.

[0217] The model in the present application is an object that constitutes an objective description of the structure of a form by means of a physical or virtual representation. The object is not equal to the object, and is not limited to the physical and virtual. It can be a data processing function, a software program, a processing mode, a use method, an operation method, a work flow, an application process, an electronic hardware, a circuit module, a processing system, a system imitation or a simulation object.

[0218] Finally, it should be noted that: the above-described embodiments are merely specific embodiments of the present application, used to illustrate the technical solutions of the present application, and are not limiting. The protection scope of the present application is not limited thereto. Although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacements to some technical features, within the technical range disclosed by the present application. These modifications or replacements do not cause the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and any modification or equivalent replacement that does not deviate from the spirit and scope of the present application should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A robot interaction control method based on diffusion strategy, characterized in that: comprising the following steps: step one, obtaining task instruction information to be executed; step two, using a pre-created scene understanding model, based on the task instruction information, identifying the scene features involved in the task to obtain scene perception data; step three, using a pre-created scene interaction prediction model, designing robot actions according to the scene perception data, and predicting the changes of the scene caused by the robot actions to obtain interaction scene change information; step four, using a pre-created diffusion strategy decision model to process the interaction scene change information to generate control instructions for the robot; the diffusion strategy decision model is formed as follows: based on a video generation model U-Net, setting an action head generation unit; each layer of down-sampling and up-sampling in the action head generation unit takes the interaction scene change information output by the scene interaction prediction model as input condition, and uses the understanding of the scene interaction prediction model on the future state change to generate corresponding actions; inputting historical observation information into the scene interaction prediction model to obtain interaction scene change information; inputting the interaction scene change information and text instructions into the action head generation unit to obtain robot actions, and forming a diffusion strategy decision model with a parallel generation architecture.

2. The robot interaction control method based on diffusion strategy according to claim 1, characterized in that: step two, using a pre-created scene understanding model, based on the task instruction information, identifying the scene features involved in the task to obtain scene perception data, is as follows: according to the task instruction information, collecting robot operation expert data and internet video data about the task scene; processing the robot operation expert data and internet video data to obtain a plurality of historical observation data; using the plurality of historical observation data to train a world model, and the training target is to minimize the training loss amount, so that the world model can learn the general physical law of environmental change when the robot performs specific task operations, and be used to transfer the generalization ability of the world model to the robot operation scene to obtain the scene understanding model; using the scene understanding model to identify the scene features involved in the task to obtain scene perception data.

3. The robot interaction control method based on diffusion strategy according to claim 1, characterized in that: step three, using a pre-created scene interaction prediction model, based on the scene perception data, designing robot actions, and predicting the changes of the scene caused by the robot actions to obtain interaction scene change information, is as follows: according to the scene perception data, designing robot actions to obtain action information to be executed by the robot; at the same time, obtaining historical observation information, which includes observation image data and robot joint angle state; based on the action information, observation image data and robot joint angle state, training the scene understanding model to form a scene interaction prediction model; using the scene interaction prediction model to predict the changes of the scene caused by the robot actions to obtain interaction scene change information.

4. The robot interaction control method based on diffusion strategy according to claim 3, characterized in that: ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ The method for training the scene interaction prediction model based on the action information, the observation image data and the robot joint angle state is as follows: Remote operation is performed on the robot based on a plurality of operation tasks to obtain an original training data set; the original training data set includes observation image data of the robot, action information, robot joint angle states and text instructions corresponding to the tasks; The original training data set is sampled to obtain a generation target, which includes a plurality of video frames and robot states; The plurality of video frames are encoded into a latent space representation by using an image encoder; The latent space representation and the robot states are forwardly added with noise for a plurality of time steps to train a noise network, and two trained noise networks, including a noise network one and a noise network two, are obtained; The latent space representation and the robot states are converted into video pure noise and state pure noise, and are input into the two trained noise networks to perform denoising for a plurality of time steps, and finally generate a noise-free predicted video latent space representation and a predicted robot state quantity, which are used to represent the prediction of the robot and the environment; The predicted video latent space representation is restored into clear video frame images by using a decoder, so as to complete the construction of the scene interaction prediction model and realize the understanding of the dynamic physical law of the robot interacting with the environment.

5. The robot interaction control method based on the diffusion strategy according to claim 4, wherein: The method for training the noise network by forwardly adding noise to the latent space representation and the robot states for a plurality of time steps is as follows: An expression for forwardly adding noise to the latent space is constructed based on the latent space representation, a strategy parameter and standard normal distribution noise one; An expression for forwardly adding noise to the state is constructed based on the robot state, the strategy parameter and standard normal distribution noise two; The strategy parameter is a parameter for controlling the noise intensity in the diffusion process to change with time; The noise network one is constructed based on the expression for forwardly adding noise to the latent space, and is used to fit the standard normal distribution noise one actually added at each time step; The noise network two is constructed based on the expression for forwardly adding noise to the state, and is used to fit the standard normal distribution noise two actually added at each time step; A loss function is set to jointly train the noise network one and the noise network two, and the training target is to minimize the loss quantity, so as to obtain the two trained noise networks.

6. The robot interaction control method based on the diffusion strategy according to claim 1, wherein: In step four, the diffusion strategy decision model created in advance is used to process the interaction scene change information to generate the control instructions of the robot, and the method is as follows: The historical observation information and the text instructions are obtained according to the task instruction information; The scene interaction prediction model is trained based on the historical observation information and the text instructions to form the diffusion strategy decision model; The diffusion strategy decision model is used to generate noise-free video latent space representations of a plurality of future time steps and predicted robot actions; The predicted video frame latent space representation is restored into clear video frame images by using a decoder. The predicted robot action is converted into a control instruction for controlling the robot target joint angle position.

7. The robot interaction control method based on diffusion strategy according to claim 6, characterized in that: The method for obtaining the robot action by inputting the interaction scene change information and the text instruction into the action head generation unit is as follows: Obtain the interaction scene change information, which includes the video frame image for predicting the future; Use the image encoder to encode the video frame image into the latent space representation; The latent space representation and the text instruction are forwardly added with noise for multiple time steps to train the noise network, and two trained noise networks are obtained, including noise network three and noise network four; The latent space representation and the text instruction are converted into video pure noise and instruction pure noise, and are input into the two trained noise networks to denoise for multiple time steps, and finally generate a noise-free predicted video latent space representation and a predicted robot action.

8. A robot interaction control method based on diffusion strategy, characterized in that: It includes the following contents: Collect robot operation expert data and Internet video data, train the world model, and enable the world model to learn the general physical law of environmental change when the robot performs specific task operations, so as to migrate the generalization ability of the world model to the robot operation scene; Collect robot expert data of specific tasks to construct a scene interaction prediction model; the scene interaction prediction model is used to understand the physical law of robot action and environmental interaction, and its input is historical observation image, robot joint angle state and robot action to be executed, and its output is a video representing the dynamic change of the future environment, to represent the action result of the robot and the environment; Load the scene interaction prediction model into the diffusion strategy architecture and couple it with the action generation unit to form a diffusion strategy decision model for generating robot action; the method for forming the diffusion strategy decision model is as follows: Based on the video generation model U-Net, set the action head generation unit; Each layer of down-sampling and up-sampling in the action head generation unit takes the interaction scene change information output by the scene interaction prediction model as the input condition, uses the understanding of the scene interaction prediction model on the future state change to generate the corresponding action; Input the historical observation information into the scene interaction prediction model to obtain the interaction scene change information; Input the interaction scene change information and the text instruction into the action head generation unit to obtain the robot action, and form a diffusion strategy decision model with a parallel generation architecture; Each layer network of the action generation unit receives the vector features output by the corresponding layer of the scene interaction prediction model, uses the understanding of the scene interaction prediction model on the physical law of robot action and environmental change, and outputs the robot action combined with the text instruction; The scene interaction prediction model and the diffusion strategy decision model are alternately trained, and finally a diffusion strategy decision model capable of responding to human task instructions, predicting future environmental dynamic changes, and generating robot control instructions is obtained.

9. A server, characterized in that: It includes: One or more processing units; A storage device for storing one or more programs; When the one or more programs are executed by the one or more processing units, the one or more processing units are caused to implement the method of claim 1-8.

Images