Robot control and model training method and device, equipment and storage medium

Abstract

The invention provides a robot control and model training method and device, equipment and a storage medium, and can be applied to various fields of robots, model training and the like. The method comprises the steps of obtaining description information of a target task to be executed by a robot and a first environment image when the robot is in an initial state; based on the description information of the target task, performing subtask division on the target task to obtain description information of N subtasks; for the ith sub-task in the N sub-tasks, determining three-dimensional voxel movement track information when the robot executes the ith sub-task based on the first environment image and the description information of the ith sub-task; the three-dimensional voxel movement track information of the ith sub-task is processed by controlling the prediction model to obtain the prediction control information of the ith sub-task, and the robot is controlled to execute the ith sub-task based on the prediction control information of the ith sub-task, so that the control effect of the robot is improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computer technology, and in particular to a robot control and model training method, device, equipment and storage medium. Background Art

[0002] With the rapid development of robotics technology, various robot applications have emerged, such as industrial robots, storage robots, household robots, and medical assistance robots, etc. In the development of robots, how to effectively control robots has always been a key technical problem that technicians in this field need to solve.

[0003] In view of the outstanding performance of neural network technology in various fields, technicians have proposed various robot control methods based on neural networks. However, the current robot control methods based on neural networks still have the problem of unsatisfactory control effects. Summary of the invention

[0004] The present application provides a robot control and model training method, device, equipment and storage medium, which can improve the control effect of the robot.

[0005] In a first aspect, the present application provides a robot control method, comprising:

[0006] Acquire description information of a target task to be performed by the robot, and a first environment image when the robot is in an initial state;

[0007] Based on the description information of the target task, divide the target task into subtasks to obtain description information of N subtasks, where N is a positive integer greater than 1;

[0008] For an i-th subtask among the N subtasks, based on the first environment image and the description information of the i-th subtask, determine the three-dimensional voxel movement trajectory information when the robot performs the i-th subtask, where i is a positive integer less than or equal to N;

[0009] The three-dimensional voxel movement trajectory information of the i-th subtask is processed by the control prediction model to obtain the prediction control information of the i-th subtask, and based on the prediction control information of the i-th subtask, the robot is controlled to perform the i-th subtask.

[0010] In a second aspect, the present application provides a model training method, comprising:

[0011] Acquire description information of a training task to be performed by the robot, and a first environment image when the robot is in an initial state;

[0012] Based on the description information of the training task, divide the training task into subtasks to obtain description information of N subtasks, where N is a positive integer greater than 1;

[0013] For an i-th subtask among the N subtasks, based on the first environment image and the description information of the i-th subtask, determine the three-dimensional voxel movement trajectory information when the robot performs the i-th subtask, where i is a positive integer less than or equal to N;

[0014] Processing the three-dimensional voxel movement trajectory information of the i-th subtask through the control prediction model to obtain prediction control information of the i-th subtask;

[0015] A model loss of the control prediction model is determined, and the control prediction model is trained based on the model loss.

[0016] In a third aspect, the present application provides a robot control device, comprising:

[0017] An acquisition unit, used to acquire description information of a target task to be performed by the robot, and a first environment image when the robot is in an initial state;

[0018] A division unit, configured to divide the target task into subtasks based on the description information of the target task to obtain description information of N subtasks, where N is a positive integer greater than 1;

[0019] a trajectory determination unit, configured to determine, for an i-th subtask among the N subtasks, three-dimensional voxel movement trajectory information of the robot when performing the i-th subtask based on the first environment image and the description information of the i-th subtask, where i is a positive integer less than or equal to N;

[0020] A prediction control unit is used to process the three-dimensional voxel movement trajectory information of the i-th subtask by controlling the prediction model, obtain the prediction control information of the i-th subtask, and control the robot to perform the i-th subtask based on the prediction control information of the i-th subtask.

[0021] In some embodiments, the trajectory determination unit is specifically used to perform spatial grid division on the first environment image to obtain a first spatial grid image; based on the description information of the i-th subtask, determine the three-dimensional voxel movement trajectory information of the robot when performing the i-th subtask in the first spatial grid image.

[0022] In some embodiments, the trajectory determination unit is specifically used to project the first environment image onto a top-view image to obtain a first top-view environment image; determine the depth value of each pixel in the first environment image; divide the first top-view environment image into multiple grids on the top-view plane based on the coordinate values ​​of the pixel points in the first top-view environment image; and divide the first top-view environment image into multiple layers in the depth direction based on the depth value of each pixel point in the first environment image to obtain the first spatial grid image.

[0023] In some embodiments, the trajectory determination unit is specifically used to obtain the initial posture information of the robot when performing the i-th subtask; based on the description information of the i-th subtask and the initial posture information of the robot when performing the i-th subtask, determine the three-dimensional voxel movement trajectory information of the robot when performing the i-th subtask in the first spatial grid image.

[0024] In some embodiments, the trajectory determination unit is specifically used to process the description information of the i-th subtask, the initial posture information of the robot when performing the i-th subtask, and the first spatial grid image through a trajectory prediction model to obtain the three-dimensional voxel movement trajectory information of the robot when performing the i-th subtask.

[0025] In some embodiments, the three-dimensional voxel movement trajectory information of the i-th subtask includes marking information of the first grids occupied by the trajectory of the i-th subtask on the first overhead environment image, and the number of grids corresponding to each first grid in the depth direction.

[0026] In some embodiments, the division unit is specifically used to obtain subtask division prompt words, and the subtask division prompt words are used to describe the division control information of the subtask; based on the description information of the target task and the subtask division prompt words, the target task is divided into subtasks to obtain the description information of the N subtasks.

[0027] In some embodiments, the division unit is specifically configured to divide the target task into subtasks based on the first environment image, the description information of the target task and the subtask division prompt words to obtain the description information of the N subtasks.

[0028] In some embodiments, a prediction control unit is used to determine the noise control information of the i-th subtask; taking the three-dimensional voxel movement trajectory information of the i-th subtask as the control condition, the noise control information of the i-th subtask is denoised K times by controlling the prediction model to obtain the prediction control information of the i-th subtask, where K is a positive integer.

[0029] In some embodiments, a prediction control unit is used to obtain an initial environment image when the robot performs the i-th subtask, and initial posture information when the robot performs the i-th subtask; using the three-dimensional voxel movement trajectory information of the i-th subtask, the initial environment image when the robot performs the i-th subtask, and the initial posture information when the robot performs the i-th subtask as control conditions, the noisy control information of the i-th subtask is denoised K times through the control prediction model to obtain the predicted control information of the i-th subtask.

[0030] In some embodiments, the control prediction model includes a denoising module and a prediction control unit, which is used for performing the j-th denoising process in the K denoising processes, taking the three-dimensional voxel movement trajectory information of the i-th subtask, the initial environment image when the robot performs the i-th subtask, and the initial posture information when the robot performs the i-th subtask as control conditions, and performing the j-th denoising process on the j-1 denoising control information of the i-th subtask through the denoising module to obtain the j-th noise prediction value, where j is a positive value less than or equal to K. integer, if the j is 1, the j-1th denoising control information is the noise addition control information of the ith subtask; based on the j-th noise prediction value and the j-1th denoising control information of the ith subtask, obtain the j-th denoising control information of the ith subtask; use the j-th denoising control information as the new j-1th denoising control information, return and repeat the above steps to obtain the K-th denoising control information of the ith subtask; determine the K-th denoising control information as the prediction control information of the ith subtask.

[0031] In some embodiments, the prediction control unit is used to determine the noise value obtained by random sampling as the noise addition control information of the i-th subtask.

[0032] In a fourth aspect, the present application provides a model training device, comprising:

[0033] An acquisition unit, used to acquire description information of a training task to be performed by the robot, and a first environment image when the robot is in an initial state;

[0034] A division unit, configured to divide the training task into subtasks based on the description information of the training task to obtain description information of N subtasks, where N is a positive integer greater than 1;

[0035] a trajectory determination unit, configured to determine, for an i-th subtask among the N subtasks, three-dimensional voxel movement trajectory information of the robot when performing the i-th subtask based on the first environment image and the description information of the i-th subtask, where i is a positive integer less than or equal to N;

[0036] A prediction unit, configured to process the three-dimensional voxel movement trajectory information of the i-th subtask by controlling the prediction model to obtain prediction control information of the i-th subtask;

[0037] A training unit is used to determine the model loss of the control prediction model and train the control prediction model based on the model loss.

[0038] In some embodiments, the prediction unit is specifically used to obtain the real control information of the i-th subtask; perform noise processing on the real control information to obtain the noisy control information of the i-th subtask; take the three-dimensional voxel movement trajectory information of the i-th subtask as the control condition, and perform K denoising processes on the noisy control information of the i-th subtask through the control prediction model to obtain the predicted control information of the i-th subtask, where K is a positive integer.

[0039] In some embodiments, the prediction unit is specifically used to obtain the initial environment image when the robot performs the i-th subtask, and the initial posture information of the robot when performing the i-th subtask; using the three-dimensional voxel movement trajectory information of the i-th subtask, the initial environment image when the robot performs the i-th subtask, and the initial posture information when the robot performs the i-th subtask as control conditions, the noisy control information of the i-th subtask is denoised K times through the control prediction model to obtain the predicted control information of the i-th subtask.

[0040] In some embodiments, the control prediction model includes a denoising module. In some embodiments, the prediction unit is specifically used for performing the j-th denoising process in the K denoising processes, taking the three-dimensional voxel movement trajectory information of the i-th subtask, the initial environment image when the robot performs the i-th subtask, and the initial posture information when the robot performs the i-th subtask as control conditions, and performing the j-th denoising process on the j-1 denoising control information of the i-th subtask through the denoising module to obtain the j-th noise prediction value, wherein j is less than or a positive integer of K, if the j is 1, the j-1th denoising control information is the noise addition control information of the ith subtask; based on the j-th noise prediction value and the j-1th denoising control information of the ith subtask, the j-th denoising control information of the ith subtask is obtained; the j-th denoising control information is used as the new j-1th denoising control information, and the above steps are repeated to obtain the K-th denoising control information of the ith subtask; the K-th denoising control information is determined as the prediction control information of the ith subtask.

[0041] In some embodiments, the training unit is specifically used to determine the model loss of the control prediction model based on the noise prediction value in the K denoising processes and the noise added value when the real control information is subjected to K noise addition processes.

[0042] In a fifth aspect, the present application provides an electronic device, comprising a processor and a memory, wherein the memory is used to store a computer program, and the processor is used to call and run the computer program stored in the memory to execute the method of the first aspect or the second aspect.

[0043] In a sixth aspect, a chip is provided for implementing the method in each implementation of the first aspect. Specifically, the chip includes: a processor for calling and running a computer program from a memory, so that a device equipped with the chip executes the method in the first or second aspect.

[0044] In a seventh aspect, a computer-readable storage medium is provided for storing a computer program, wherein the computer program enables a computer to execute the method of the first aspect or the second aspect.

[0045] In an eighth aspect, a computer program product is provided, comprising computer program instructions, wherein the computer program instructions enable a computer to execute the method of the first aspect or the second aspect.

[0046] In a ninth aspect, a computer program is provided, which, when executed on a computer, enables the computer to execute the method of the first aspect or the second aspect.

[0047] In summary, the present application obtains the description information of the target task to be performed by the robot, and the first environment image when the robot is in the initial state; based on the description information of the target task, the target task is divided into subtasks to obtain the description information of N subtasks; for the ith subtask among the N subtasks, based on the above-mentioned first environment image and the description information of the ith subtask, the three-dimensional voxel movement trajectory information of the robot when performing the ith subtask is determined; and then the three-dimensional voxel movement trajectory information of the ith subtask is processed through the control prediction model to obtain the prediction control information of the ith subtask, and based on the prediction control information of the ith subtask, the robot is controlled to perform the ith subtask. It can be seen from this that the embodiment of the present application can improve the robustness of the robot operation and the execution success rate of long sequence tasks by dividing the target task into multiple subtasks for separate execution. In addition, for each subtask, the embodiment of the present application determines the three-dimensional voxel movement trajectory information when the robot performs the subtask, and then based on the three-dimensional voxel movement trajectory information, the control information when the robot performs the subtask can be accurately determined. In this way, the robot can accurately execute the subtask based on the accurately determined control information, thereby improving the control effect of the robot. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0049] Figure 1 A schematic diagram of an implementation environment provided for an embodiment of the present application;

[0050] Figure 2 A flowchart of a model training method provided in one embodiment of the present application;

[0051] Figure 3 A schematic diagram of an application scenario of an embodiment of the present application;

[0052] Figure 4 A schematic diagram of task division through the task division model;

[0053] Figure 5 Another schematic diagram of task partitioning through the task partitioning model;

[0054] Figure 6 A schematic diagram of a subtask divided;

[0055] Figure 7 A schematic diagram for obtaining a first spatial grid image;

[0056] Figure 8 A schematic diagram of the movement trajectory information of three-dimensional voxels;

[0057] Fig. 9 A specific schematic diagram of the three-dimensional voxel movement trajectory information of the subtask;

[0058] Fig.10 A schematic diagram for predicting control information through a control prediction model;

[0059] Fig.11 Schematic diagram of K times noise addition;

[0060] Fig.12 Schematic diagram of K-times denoising;

[0061] Fig.13 Another schematic diagram for predicting control information through a control prediction model;

[0062] Fig.14 Another schematic diagram of predicting control information by using a control prediction model;

[0063] Fig.15 A schematic diagram of multiple denoising by controlling the prediction model;

[0064] Fig.16 A network diagram of a denoising module;

[0065] Fig.17 A data processing schematic diagram of a denoising module;

[0066] Fig.18 A schematic diagram of a flow chart of a robot control method provided in one embodiment of the present application;

[0067] Fig.19 Schematic diagram of several tasks involved in the embodiments of this application;

[0068] Fig. 20 A schematic diagram of the robot control effect;

[0069] Fig.21 Another schematic diagram of the robot control effect;

[0070] Fig. 22 is a schematic block diagram of a robot control device provided in one embodiment of the present application;

[0071] Fig.23 is a schematic block diagram of a model training device provided in one embodiment of the present application;

[0072] Fig.24 It is a schematic block diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0073] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0074] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application described here can be implemented in an order other than those illustrated or described here. In an embodiment of the present invention, "B corresponding to A" means that B is associated with A. In one implementation, B can be determined according to A. However, it should also be understood that determining B according to A does not mean that B is determined only according to A, but B can also be determined according to A and / or other information. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or server that includes a series of steps or units does not have to be limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices. In the description of the present application, unless otherwise specified, "multiple" refers to two or more than two.

[0075] The robot control method provided in the embodiment of the present application can be applied to various fields such as robots and model training, and can improve the control effect of the robot.

[0076] With the rapid development of neural network technology, various robot control methods based on neural networks have been proposed. For example, the robot control method based on the diffusion model, the diffusion model is a generative model that uses a probabilistic framework to gradually convert random noise into samples in the target distribution. The current robot control method based on the diffusion model is to apply the diffusion model to the robot control strategy, and use the diffusion model to predict the action sequence based on observation. However, this diffusion strategy mostly focuses on short-term strategies, and often cannot complete long-sequence tasks, and the control effect is not ideal.

[0077] In order to solve the above technical problems, the embodiment of the present application proposes a robot control method, which obtains the description information of the target task to be performed by the robot and the first environment image when the robot is in the initial state; based on the description information of the target task, the target task is divided into subtasks to obtain the description information of N subtasks; for the i-th subtask among the N subtasks, based on the above first environment image and the description information of the i-th subtask, the three-dimensional voxel movement trajectory information of the robot when performing the i-th subtask is determined; and then through the control prediction model, the three-dimensional voxel movement trajectory information of the i-th subtask is processed to obtain the prediction control information of the i-th subtask, and based on the prediction control information of the i-th subtask, the robot is controlled to perform the i-th subtask. It can be seen from this that the embodiment of the present application can improve the robustness of the robot operation and the execution success rate of long sequence tasks by dividing the target task into multiple subtasks for separate execution. In addition, for each subtask, the embodiment of the present application determines the three-dimensional voxel movement trajectory information when the robot performs the subtask, and then based on the three-dimensional voxel movement trajectory information, the control information when the robot performs the subtask can be accurately determined. In this way, the robot can accurately execute the subtask based on the accurately determined control information, thereby improving the control effect of the robot.

[0078] The implementation environment of the robot control method provided in the embodiment of the present application is introduced below.

[0079] Figure 1 A schematic diagram of an implementation environment provided for an embodiment of the present application, such as Figure 1 As shown, the implementation environment includes: a camera 101, a robot 102 and a server 103.

[0080] The camera 101 is connected to the server 103 via a wired or wireless connection, and the robot 102 is also connected to the server 103 via a wired or wireless connection.

[0081] In an embodiment of the present application, the camera 101 is used to collect the current environment image, that is, the image of the environment in which the robot is currently located, and send the collected environment image to the server 103. In some embodiments, the robot 102 is connected to the camera 101 by wire or wirelessly. In this way, the camera 101 can start the image acquisition function based on the instruction sent by the robot 102. For example, when the robot 102 is started, an image acquisition instruction is sent to the camera 101, and the camera 101 can collect the environment image in which the robot 102 is currently located based on the image acquisition instruction sent by the robot 102.

[0082] In the embodiment of the present application, the control information of the robot is predicted by a control prediction model, so before controlling the robot, the control prediction model needs to be trained first.

[0083] In some embodiments, the model training process of the embodiment of the present application is completed by the server 103. Specifically, the server 103 obtains the description information of the training task to be performed by the robot, and the first environment image when the robot is in the initial state. In some embodiments, the description information of the above training task can be input by an object (such as a user), and the first environment image is collected by the camera 101. Then, based on the description information of the training task, the server 103 divides the training task into subtasks to obtain the description information of N subtasks, where N is a positive integer greater than 1; for the i-th subtask among the N subtasks, based on the first environment image and the description information of the i-th subtask, the three-dimensional voxel movement trajectory information when the robot performs the i-th subtask is determined, where i is a positive integer less than or equal to N; through the control prediction model, the three-dimensional voxel movement trajectory information of the i-th subtask is processed to obtain the prediction control information of the i-th subtask, and at the same time, the model loss of the control prediction model is determined, and the control prediction model is trained based on the model loss. It can be seen that, during model training, the embodiment of the present application can improve the robustness of robot operation and the success rate of executing long sequence tasks by dividing the training task into multiple subtasks for separate execution. In addition, for each subtask, the embodiment of the present application determines the three-dimensional voxel movement trajectory information when the robot performs the subtask, and then trains the control prediction model with the three-dimensional voxel movement trajectory information as the control condition, so that the control prediction model can accurately predict the control information when the robot performs the subtask, and then can improve the robot's accurate execution of the subtask, thereby improving the control effect of the robot.

[0084] After obtaining the trained control prediction model based on the above model training method, the server 103 can save the trained control prediction model locally for controlling the robot. In some embodiments, the server 103 can also send the trained control prediction model to the robot 102 for saving, so that the robot 102 can complete its own control based on the trained control prediction model.

[0085] In some embodiments, if the robot control method of the embodiment of the present application is executed by the server 103, the server 103 first obtains the description information of the target task to be executed by the robot 102, and the first environment image when the robot 102 is in the initial state. For example, the server 103 obtains the description information of the target task input by the object, and obtains the first environment image of the robot 102 in the initial state taken by the camera 101. Then, based on the description information of the target task, the server 103 divides the target task into subtasks to obtain the description information of N subtasks, where N is a positive integer greater than 1; for the i-th subtask among the N subtasks, based on the first environment image and the description information of the i-th subtask, the three-dimensional voxel movement trajectory information when the robot performs the i-th subtask is determined, where i is a positive integer less than or equal to N; the server 103 processes the three-dimensional voxel movement trajectory information of the i-th subtask through the control prediction model, obtains the prediction control information of the i-th subtask, and sends the prediction control information of the i-th subtask to the robot 102. The robot 102 is controlled to perform the i-th subtask based on the predicted control information of the i-th subtask.

[0086] In some embodiments, when the robot control method of the embodiment of the present application is executed by the robot 102, the robot 102 first obtains the trained control prediction model from the server 103. During the robot control process, the robot 102 first obtains the description information of the target task to be performed by the robot 102, and the first environment image when the robot 102 is in the initial state. For example, the robot 102 obtains the description information of the target task input by the object, and obtains the first environment image of the robot 102 in the initial state taken by the camera 101. Next, the robot 102 divides the target task into subtasks based on the description information of the target task to obtain description information of N subtasks, where N is a positive integer greater than 1; for the i-th subtask among the N subtasks, based on the first environment image and the description information of the i-th subtask, the three-dimensional voxel movement trajectory information when the robot performs the i-th subtask is determined, where i is a positive integer less than or equal to N; the robot 102 processes the three-dimensional voxel movement trajectory information of the i-th subtask through the control prediction model to obtain prediction control information of the i-th subtask, and controls the robot 102 to perform the i-th subtask based on the prediction control information of the i-th subtask.

[0087] As can be seen from the above, the embodiment of the present application can improve the robustness of the robot operation and the success rate of executing long sequence tasks by dividing the target task into multiple subtasks for separate execution. In addition, for each subtask, the embodiment of the present application determines the three-dimensional voxel movement trajectory information when the robot executes the subtask, and then based on the three-dimensional voxel movement trajectory information, the control information when the robot executes the subtask can be accurately determined, so that the robot can accurately execute the subtask based on the accurately determined control information, thereby improving the control effect of the robot.

[0088] In some embodiments, the robot 102 of the embodiments of the present application may be any type of robot, such as an industrial robot, a housekeeping robot, a medical robot, etc. Optionally, the robot 102 may also be a robotic arm.

[0089] In some embodiments, the above-mentioned server 103 may be one or more. When there are multiple servers, there are at least two servers for providing different services, and / or there are at least two servers for providing the same service, such as providing the same service in a load balancing manner, which is not limited in the embodiments of the present application. Among them, the above-mentioned server may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. The server can also become a node of the blockchain.

[0090] It should be noted that the implementation environment of the embodiments of the present application includes but is not limited to Figure 1 shown.

[0091] The technical solutions of the embodiments of the present application are described in detail below through some embodiments. The following embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments.

[0092] First combine Figure 2 , the training process of the control prediction model provided in the embodiment of the present application is introduced.

[0093] Figure 2 The flowchart of the model training method provided in one embodiment of the present application is shown in FIG. The execution subject of the embodiment of the present application may be a device having a model training function, such as a model training device. The model training device may be the above-mentioned Figure 1 The server 103, or robot 102 shown, may also be Figure 1The system is composed of the server 103 or the robot 102 shown. For the convenience of description, the following embodiments take electronic devices as an example to illustrate the method of the embodiments of the present application.

[0094] like Figure 2 As shown, the model training method of the embodiment of the present application includes:

[0095] S101, obtaining description information of a training task to be performed by the robot and a first environment image when the robot is in an initial state.

[0096] In the embodiment of the present application, a control prediction model is proposed, which can predict the control information of the robot performing the subtask based on the three-dimensional voxel movement trajectory information of the robot performing the subtask. Before using the control prediction model to control the robot, the control prediction model needs to be trained first.

[0097] During the training process, a training data set is first obtained, which includes description information of each training task in a plurality of training tasks. The description information of the training task can be understood as description information describing the training task. For example, the description information of the training task can be "opening the bottle cap of the bottle", so that the robot can perform the training task based on the description information of the training task, that is, opening the bottle cap.

[0098] The embodiments of the present application do not limit the specific manner in which the electronic device obtains the descriptive information of the training task. In one possible implementation, an object (such as a user) can input the descriptive information of the training task. In another possible implementation, the electronic device can generate the descriptive information of the training task by itself, or download the descriptive information of multiple training tasks from the network. In another possible implementation, the electronic device can collect common tasks in the current application field. For example, if the robot is used in the housekeeping field, it can collect common tasks in the housekeeping field. For another example, if the robot is used in the medical field, it can collect common tasks in the medical field.

[0099] In the embodiment of the present application, when the control prediction model is trained using the training tasks in the training data set, multiple rounds of training are usually performed. In each round of training, a batch of training tasks are selected for model training, and the process of training the control prediction model using the description information of each training task in this batch of training tasks is basically the same. For the sake of ease of description, one training task is used as an example for explanation.

[0100] During model training, in addition to obtaining the descriptive information of the training task, the electronic device also needs to obtain the environmental image of the robot in its initial state when performing the training task. For ease of description, this environmental image is recorded as the first environmental image. In an embodiment of the present application, the first environmental image is the environmental image when the robot has not yet performed the training task and is preparing to perform the training task, that is, the environmental image when the robot is in its initial state. For example, assuming that the training task is "opening the bottle cap of the bottle", the first environmental image of the robot in its initial state is as follows: Figure 3 As shown, a robot (i.e., a robotic arm) and a bottle are placed on a table, a bottle cap is provided on the top of the bottle, and the robot is at a certain distance from the bottle.

[0101] The embodiment of the present application does not limit the specific manner in which the electronic device obtains the first environment image. In a possible implementation, the embodiment of the present application sets a camera, which is used to collect the environment image of the robot at the current moment. Optionally, the camera is a camera independent of the robot, and optionally, the camera can also be set on the robot. In some embodiments, the camera is communicatively connected to the electronic device, so that the camera sends the collected first environment image directly to the electronic device. In some embodiments, the camera is not communicatively connected to the electronic device, but is communicatively connected to the robot, and the robot is communicatively connected to the electronic device, so that the camera can send the collected first environment image to the robot, and the robot then forwards the first environment image to the electronic device.

[0102] In an embodiment of the present application, after the electronic device obtains description information of the training task to be performed by the robot and a first environment image when the robot is in an initial state, the following step S102 is executed.

[0103] S102: Based on the description information of the training task, divide the training task into subtasks to obtain description information of N subtasks.

[0104] Wherein, N is a positive integer greater than 1.

[0105] In the embodiment of the present application, in order to enable the robot to effectively execute the long sequence task, the long sequence task is decomposed into a series of more manageable and concise subtasks. For each individual subtask, the control prediction model generates control information for the robot end effector to execute the subtask, thereby improving the robot's execution effect on the long sequence task.

[0106] Specifically, the electronic device divides the training task into multiple subtasks, for example, N subtasks, based on the description information of the training task obtained above, and then obtains the description information of each of the N subtasks.

[0107] The embodiment of the present application does not limit the specific manner in which the electronic device divides the training task into subtasks based on the description information of the training task to obtain the description information of N subtasks.

[0108] In some embodiments, when the electronic device divides the training task into subtasks, it divides only based on the description information of the training task. For example, the description information of the training task is "open the bottle cap on the bottle", from which it can be known that the training task includes at least two subtasks: grabbing the bottle and opening the bottle cap. In this way, the electronic device can divide the training task into two subtasks, and the description information of the two subtasks is "grabbing the bottle" and "opening the bottle cap" respectively.

[0109] In some embodiments, the electronic device divides the training task into N subtasks through the following steps S102-A and S102-B:

[0110] S102-A, obtaining a subtask division prompt word, where the subtask division prompt word is used to describe the subtask division control information;

[0111] S102-B, based on the description information of the training task and the subtask division prompt words, divide the training task into subtasks to obtain description information of N subtasks.

[0112] In this implementation, in order to accurately decompose a complex and long sequence of training tasks into multiple short-term, manageable subtasks, the subtask division prompt words are also obtained. The subtask division prompt words are used to describe the subtask division control information, which usually starts and ends with grasping or releasing an object, indicating the complete interaction between the robot and the object. Then, based on the description information of the training task and the subtask division prompt words, the training task is divided into subtasks to obtain the description information of N subtasks. Through such task division, repeated or reverse trajectories in subtasks can be avoided, thereby minimizing potential ambiguity in the subsequent trajectory generation process.

[0113] In one example, the embodiment of the present application can divide the training task into N subtasks by using a trained task division model. The task division model can be a large model, such as a VLM model, which is trained to divide the tasks. Specifically, Figure 4 As shown, the electronic device inputs the description information of the training task and the subtask division prompt words into the task division model. The task division model divides the training task into N subtasks based on the description information of the training task and the subtask division prompt words, and outputs the description information of each of the N subtasks.

[0114] In some embodiments, in order to further improve the accuracy of the division of subtasks, the first environment image is also considered in the task division process. At this time, the above S102-B may include: based on the first environment image, the description information of the training task and the subtask division prompt words, the training task is divided into subtasks to obtain the description information of N subtasks. In this implementation, when the electronic device divides the training task into subtasks, it can accurately divide the training task according to the description information of the training task, the first environment image and the subtask division prompt words, so that the subtasks based on the accurate division are defined as discrete stages of the robot operation task, and then the robot is controlled to accurately execute these subtasks, thereby accurately executing the long sequence of training tasks.

[0115] In one example, if the embodiment of the present application divides the training task into N subtasks through the trained task division model, then Figure 5 As shown, the electronic device inputs the description information of the training task, the subtask division prompt word and the first environment image into the task division model. The task division model M divides the training task T into N subtasks based on the description information of the training task T, the subtask division prompt word P and the first environment image I, and outputs the description information of each of the N subtasks, that is, M(T,I,P)={S i} i＝1,…,N , where S i Represents the description information of the i-th subtask among N subtasks.

[0116] For example, assuming that the description information of the training task is "put an item into a drawer", the electronic device divides the training task into subtasks based on the first environment image, the description information of the training task and the subtask division prompt words through the task division model, and obtains the description information of the four subtasks: "grab the bottom drawer handle", "pull out the drawer", "grab the item" and "put the item into the bottom drawer". For another example, Figure 6 As shown, assuming that the description information of the training task is "open the bottle cap on the bottle", the electronic device divides the training task into subtasks through a task division model based on the first environment image, the description information of the training task and the subtask division prompt words, and obtains the description information of the multiple subtasks as: "grab the bottle", "open the bottle cap", etc.

[0117] In this step, the electronic device divides the training task into N subtasks based on the above steps, and then performs the following step S103.

[0118] S103 . For the ith subtask among the N subtasks, based on the first environment image and the description information of the ith subtask, determine the three-dimensional voxel movement trajectory information when the robot performs the ith subtask.

[0119] Wherein, i is a positive integer less than or equal to N.

[0120] In the embodiment of the present application, after the electronic device divides the training task into N subtasks, it executes each subtask in sequence based on the execution order of the N subtasks. For ease of description, the i-th subtask among the N subtasks is taken as an example for description.

[0121] In an embodiment of the present application, for the ith subtask among the N subtasks, the electronic device generates a series of three-dimensional coordinates when the robot performs the ith subtask given a two-dimensional first environment image. Specifically, the electronic device determines the three-dimensional voxel movement trajectory information when the robot performs the ith subtask based on the first environment image and the description information of the ith subtask.

[0122] The embodiment of the present application does not limit the specific manner in which the electronic device determines the three-dimensional voxel movement trajectory information when the robot performs the i-th subtask based on the first environment image and the description information of the i-th subtask.

[0123] In some embodiments, the electronic device converts the two-dimensional first environment image into a three-dimensional space, and then in the three-dimensional space, based on the description information of the i-th subtask, determines the pixel points that the robot must pass through to perform the i-th subtask, and then determines the three-dimensional coordinate values ​​of these pixel points as the three-dimensional voxel movement trajectory information when the robot performs the i-th subtask.

[0124] In some embodiments, the above S103 includes the following steps S103-A and S103-B:

[0125] S103-A, performing spatial grid division on the first environment image to obtain a first spatial grid image;

[0126] S103-B, based on the description information of the ith subtask, determine the three-dimensional voxel movement trajectory information when the robot performs the ith subtask in the first spatial grid image.

[0127] In this implementation, the electronic device divides the first environment image into spatial grids to obtain a first spatial grid image.

[0128] The embodiment of the present application does not limit the specific manner in which the electronic device divides the first environment image into spatial grids to obtain the first spatial grid image.

[0129] In a possible implementation manner, the electronic device projects the first environment image into a three-dimensional space, and then divides the first environment image projected into a grid to obtain a first space grid image.

[0130] In another possible implementation, the electronic device projects the first environment image onto the top view to obtain the first top view environment image. At the same time, the depth value of each pixel in the first environment image is determined. In this way, the electronic device divides the first top view environment image into multiple grids on the top view plane based on the coordinate values ​​of the pixels in the first top view environment image, and divides the first top view environment image into multiple layers in the depth direction based on the depth value of each pixel in the first environment image, to obtain the first spatial grid image.

[0131] In one example, if Figure 7 As shown, in order to maintain visual consistency, the electronic device uses the desktop texture to fill the shadows caused by the occluders in the first environment image. In addition, a height map is created to represent the height of each pixel in the first environment image. Then, the electronic device projects the first environment image onto the top view to obtain the first top view environment image, and then divides the first top view environment image into an m×n grid, and further divides the vertical space into k layers. Figure 7 FIG. 4 shows the first spatial grid image obtained when m, n, k is set to 6. Figure 7 In the figure, the grids obtained by dividing on the top view are marked, for example, marked as A1, B1, C1, D1, E1, F1, etc. In the vertical direction, the heights of the grids are marked, for example, the height of grid A1 is 6, that is, there are 6 grids in the height direction.

[0132] In an embodiment of the present application, the electronic device performs spatial grid division on the first environment image, and after obtaining the first spatial grid image, based on the description information of the i-th subtask, determines the three-dimensional voxel movement trajectory information of the robot when performing the i-th subtask in the first spatial grid image. For example, the electronic device uses a trained trajectory prediction model, and based on the description information of the i-th subtask, determines the three-dimensional voxel movement trajectory information of the robot when performing the i-th subtask in the first spatial grid image.

[0133] In some embodiments, when determining the three-dimensional voxel movement trajectory information when the robot performs the i-th subtask, the electronic device also considers the initial posture information when the robot performs the i-th subtask. Specifically, the electronic device first obtains the initial posture information when the robot performs the i-th subtask. In one example, if i is greater than 1, the electronic device obtains the posture information when the robot completes the i-1-th subtask, and then uses the posture information when the robot completes the i-1-th subtask as the initial posture information when the robot performs the i-th subtask. In one example, if i is equal to 1, the electronic device determines the posture information of the robot in the initial state as the initial posture information when the robot performs the i-th subtask. Then, based on the description information of the i-th subtask and the initial posture information when the robot performs the i-th subtask, the electronic device determines the three-dimensional voxel movement trajectory information when the robot performs the i-th subtask in the first spatial grid image.

[0134] In one example, the electronic device processes the description information of the ith subtask, the initial position information of the robot when performing the ith subtask, and the first spatial grid image through a trajectory prediction model to obtain the three-dimensional voxel movement trajectory information of the robot when performing the ith subtask. Figure 8 As shown, the electronic device inputs the description information of the ith subtask, the initial posture information of the robot when performing the ith subtask, and the first spatial grid image into the trajectory prediction model, and the trajectory prediction model predicts the three-dimensional voxel movement trajectory information when the robot performs the ith subtask. That is, M(P i ,V,S i )=T i , where T i Represents the 3D voxel moving trajectory information when the robot performs the i-th subtask. Fig. 9 As shown in FIG. 1 , assuming that the description information of the ith subtask is “grasp the bottle”, the electronic device inputs the description information of the ith subtask, the initial posture information of the robot when performing the ith subtask, and the first spatial grid image into the trajectory prediction model. The trajectory prediction model predicts the three-dimensional voxel movement trajectory information of the ith subtask, that is, the grids that the robot needs to pass through in the top view when performing the ith subtask are C2, D2, and E2, respectively. When moving to the grid E2, it moves down 3 grids in the height direction. Therefore, the three-dimensional voxel movement trajectory information of the robot performing the ith subtask is obtained as follows: Fig. 9 As shown, the heights of grids C2 and D2 are 6 respectively, and the height direction of grid E2 moves from 6 to 4 in sequence.

[0135] The embodiment of the present application does not limit the representation form of the three-dimensional voxel movement trajectory information during the i-th subtask. Fig. 9As shown, the three-dimensional voxel movement trajectory information for the i-th subtask includes the marking information of the first grid occupied by the trajectory of the i-th subtask on the first overhead environment image, and the number of grids corresponding to each first grid in the depth direction. In another example, the trajectory generated by the trajectory prediction model consists of a series of coordinates, which are represented as an m×n×k matrix. The corresponding voxels in the matrix are marked according to the order of appearance in the trajectory sequence, and the non-trajectory voxels are marked as 0. This voxel-based spatial trajectory representation provides a compact and spatially informative representation for the downstream control prediction model.

[0136] In the embodiment of the present application, after the electronic device determines the three-dimensional voxel movement trajectory information when the robot performs the i-th subtask based on the above method, it executes the following step S104.

[0137] S104 . Process the three-dimensional voxel movement trajectory information of the i-th subtask through the control prediction model to obtain prediction control information of the i-th subtask.

[0138] In the embodiment of the present application, the electronic device can determine the three-dimensional voxel movement trajectory information of each of the N subtasks according to the method of S103. Then, the three-dimensional voxel movement trajectory information of each of the N subtasks is processed by the control prediction model to obtain the prediction control information of each subtask.

[0139] In an embodiment of the present application, for each of the N subtasks, the electronic device predicts the control information of each subtask respectively through the control prediction model. For ease of description, in an embodiment of the present application, the control information predicted by the control prediction model is recorded as the predicted control information of the subtask. For example, for the i-th subtask among the N subtasks, the electronic device performs control information prediction processing based on the three-dimensional voxel trajectory information of the i-th subtask through the control prediction model to obtain the predicted control information of the i-th subtask.

[0140] The embodiment of the present application does not limit the specific method in which the electronic device processes the three-dimensional voxel movement trajectory information of the i-th subtask through the control prediction model to obtain the predicted control information of the i-th subtask.

[0141] In some embodiments, Fig.10 As shown, the electronic device inputs the three-dimensional voxel trajectory information of the i-th subtask into the control prediction model to perform control information prediction processing to obtain the predicted control information of the i-th subtask.

[0142] In some embodiments, the above S104 includes the following steps S104-A to S104-C:

[0143] S104-A, obtaining the real control information of the i-th subtask;

[0144] S104-B, performing noise processing on the real control information to obtain the noisy control information of the i-th subtask;

[0145] S104-C, taking the three-dimensional voxel movement trajectory information of the i-th subtask as the control condition, through the control prediction model, the noisy control information of the i-th subtask is denoised K times to obtain the predicted control information of the i-th subtask, where K is a positive integer.

[0146] In this implementation, forward diffusion and reverse diffusion techniques are used to obtain the predicted control information of the ith subtask. The forward diffusion process adds noise to the real control information of the ith subtask, gradually converting it into featureless noise control information. Reverse diffusion is the process of subtracting the estimated noise from the noisy control information.

[0147] In the embodiment of the present application, in the model training stage, each of the N subtasks is labeled, that is, the real control information of each subtask is determined. For each subtask, for example, the ith subtask, the real control information of the ith subtask is subjected to noise processing to obtain the noisy control information of the ith subtask. This process can be understood as a forward diffusion process.

[0148] In the embodiment of the present application, the real control information of the i-th subtask is subjected to noise processing, and the specific method of obtaining the noise control information of the i-th subtask is not limited.

[0149] In a possible implementation, the real control information of the i-th subtask may be noised once. For example, assuming that the control information of the i-th subtask is a 7-dimensional vector, each element of the 7-dimensional vector is multiplied by a preset noise value to obtain the noised control information of the i-th subtask.

[0150] In one possible implementation, Fig.11As shown, the real control information of the i-th subtask can be denoised multiple times, for example, K times. For example, assume that the real control information of the i-th subtask is a 7-dimensional vector, denoted as real control information A, and assume that K is equal to 5. Use noise value B1 to denoise the real control information A to obtain noisy control information A1. Then, use noise value B2 to denoise the noisy control information A1 to obtain noisy control information A2. Use noise value B3 to denoise the noisy control information A2 to obtain noisy control information A3. Use noise value B4 to denoise the noisy control information A3 to obtain noisy control information A4. Use noise value B5 to denoise the noisy control information A4 to obtain noisy control information A5. Optionally, the above noise value is obtained by random sampling. Optionally, the above noise value is a preset noise value.

[0151] In the embodiment of the present application, the electronic device performs noise processing on the real control information of the ith subtask, and after obtaining the noise control information of the ith subtask, executes step S104-C, and uses the three-dimensional voxel movement trajectory information of the ith subtask as the control condition, and performs K denoising processing on the noise control information of the ith subtask through the control prediction model, and obtains the predicted control information of the ith subtask. This process can be understood as a reverse diffusion process.

[0152] In the embodiment of the present application, the electronic device uses the three-dimensional voxel movement trajectory information of the i-th subtask as a control condition, and performs K denoising processes on the noisy control information of the i-th subtask through a control prediction model, and the specific method of obtaining the predicted control information of the i-th subtask is not limited.

[0153] In some embodiments, Fig.12 As shown, the electronic device inputs the three-dimensional voxel movement trajectory information of the i-th subtask and the noise control information of the i-th subtask into the control prediction model, so that the control prediction model uses the three-dimensional voxel movement trajectory information of the i-th subtask as the control information, performs K denoising processes on the noise control information of the i-th subtask, and obtains the predicted control information of the i-th subtask.

[0154] In some embodiments, the above S104-C includes the following steps S104-C1 and S104-C2:

[0155] S104-C1, obtaining an initial environment image when the robot performs the i-th subtask, and initial posture information when the robot performs the i-th subtask;

[0156] S104-C2, taking the three-dimensional voxel movement trajectory information of the i-th subtask, the initial environment image when the robot performs the i-th subtask, and the initial posture information when the robot performs the i-th subtask as control conditions, the noisy control information of the i-th subtask is denoised K times through the control prediction model to obtain the predicted control information of the i-th subtask.

[0157] In this implementation, the control conditions for the control prediction model to denoise the noisy control information of the i-th subtask include, in addition to the three-dimensional voxel movement trajectory information of the i-th subtask, the initial environment image when the robot performs the i-th subtask and the initial posture information when the robot performs the i-th subtask.

[0158] The initial environment image when the robot performs the i-th subtask can be understood as the image of the environment in which the robot is located when preparing to perform the i-th subtask. The initial posture information when the robot performs the i-th subtask can be understood as the current posture information of the robot when performing the i-th subtask.

[0159] For example, if the ith subtask is the first subtask among N subtasks, the initial environment image when the robot performs the ith subtask is the first environment image when the robot is in the initial state, and the initial posture information when the robot performs the ith subtask is the posture information when the robot is in the initial state. For another example, if the ith subtask is not the first subtask among N subtasks, the initial environment image when the robot performs the ith subtask is the environment image when the robot completes the i-1th subtask, and the initial posture information when the robot performs the ith subtask is the posture information when the robot completes the i-1th subtask.

[0160] In this implementation, if Fig.13 As shown, the electronic device inputs the three-dimensional voxel movement trajectory information of the ith subtask, the initial environment image when the robot performs the ith subtask, the initial posture information when the robot performs the ith subtask, and the noise control information of the ith subtask into the control prediction model, so that the control prediction model uses the three-dimensional voxel movement trajectory information of the ith subtask, the initial environment image when the robot performs the ith subtask, and the initial posture information when the robot performs the ith subtask to perform K denoising processes on the noisy control information of the ith subtask to obtain the predicted control information of the ith subtask.

[0161] The embodiment of the present application does not limit the specific grid structure of the control prediction model.

[0162] In some embodiments, the control prediction model includes a denoising module. In this case, the above S104-C1 includes the following steps S104-C11 to S104-C14:

[0163] S104-C11, for the j-th denoising process in the K-th denoising processes, the three-dimensional voxel movement trajectory information of the i-th subtask, the initial environment image when the robot performs the i-th subtask, and the initial posture information when the robot performs the i-th subtask are used as control conditions, and the j-th denoising process is performed on the j-1 denoising control information of the i-th subtask through the denoising module to obtain the j-th noise prediction value, where j is a positive integer less than or equal to K, and if j is 1, the j-1 denoising control information is the noise control information of the i-th subtask;

[0164] S104-C12, based on the j-th noise prediction value and the j-1-th denoising control information of the i-th subtask, obtain the j-th denoising control information of the i-th subtask;

[0165] S104-C13, taking the j-th denoising control information as the new j-1-th denoising control information, returning to repeat the above steps, and obtaining the K-th denoising control information of the i-th subtask;

[0166] S104-C14, determine the Kth denoising control information as the prediction control information of the i-th subtask.

[0167] In this implementation, the control prediction model includes a denoising module, which uses the three-dimensional voxel movement trajectory information of the i-th subtask, the initial environment image when the robot performs the i-th subtask, and the initial posture information when the robot performs the i-th subtask as control conditions, and performs multiple denoising processes on the noise-added control information of the i-th subtask. Among them, the denoising module is used to predict the noise value of each denoising process. In each denoising, the noise value currently predicted by the denoising module is subtracted from the previous denoising control information to generate new denoising control information and input it into the denoising module for further denoising. Repeat multiple times until the noise added to the control information of the i-th subtask is removed. In each denoising process, in addition to inputting the three-dimensional voxel movement trajectory information of the i-th subtask, the denoising module also inputs the initial environment image when the robot performs the i-th subtask, and the initial posture information when the robot performs the i-th subtask, so that the generated predicted control information of the i-th subtask is more in line with the actual situation, thereby improving the prediction accuracy of the control information. When the robot is controlled to perform the i-th subtask based on the accurately predicted control information, the execution effect of the task can be improved.

[0168] The following introduces the specific process of using a denoising module, taking the three-dimensional voxel movement trajectory information of the i-th subtask, the initial environment image when the robot performs the i-th subtask, and the initial posture information when the robot performs the i-th subtask as control conditions, performing multiple denoising processes on the noisy control information of the i-th subtask, and obtaining the predicted control information of the i-th subtask.

[0169] like Fig.14 As shown, the denoising process using the denoising module is an iterative process. For example, the denoising module uses the three-dimensional voxel movement trajectory information of the i-th subtask, the initial environment image when the robot performs the i-th subtask, and the initial posture information when the robot performs the i-th subtask as control conditions to perform the j-1 denoising control information of the i-th subtask for the j-th time to obtain the j-th noise prediction value. Then, based on the j-th noise prediction value and the j-th denoising control information of the i-th subtask, the j-th denoising control information of the i-th subtask is obtained. Next, the j-th denoising control information is used as the new j-1-th denoising control information. Through the denoising module, the three-dimensional voxel movement trajectory information of the i-th subtask, the initial environment image when the robot performs the i-th subtask, and the initial posture information when the robot performs the i-th subtask are used as control conditions to denoise the new j-th denoising control information to obtain a new j-th noise prediction value, and return to execute. After repeating the denoising operation K times, the K-th denoising control information can be obtained, and then the K-th denoising control information is determined as the prediction control information of the i-th subtask.

[0170] For example, suppose K is equal to 20, such as Fig.15As shown, first, the electronic device inputs the noise control information of the ith subtask, the three-dimensional voxel movement trajectory information of the ith subtask, the initial environment image when the robot performs the ith subtask, and the initial posture information when the robot performs the ith subtask into the denoising network for the first denoising, and obtains the first noise prediction value predicted by the denoising module. Based on the first noise prediction value and the noise control information of the ith subtask, the first denoising control information of the ith subtask is obtained. Next, the first denoising control information of the ith subtask, the three-dimensional voxel movement trajectory information of the ith subtask, the initial environment image when the robot performs the ith subtask, and the initial posture information when the robot performs the ith subtask are input into the denoising network for the second denoising, and obtains the second noise prediction value predicted by the denoising module. Based on the second noise prediction value and the first denoising control information of the ith subtask, the second denoising control information of the ith subtask is obtained. Next, the second denoising control information of the ith subtask, the three-dimensional voxel movement trajectory information of the ith subtask, the initial environment image when the robot performs the ith subtask, and the initial posture information when the robot performs the ith subtask are input into the denoising network for the third denoising, and the third noise prediction value predicted by the denoising module is obtained. Based on the third noise prediction value and the second denoising control information of the ith subtask, the third denoising control information of the ith subtask is obtained. Repeat the execution, and input the 19th denoising control information of the ith subtask, the three-dimensional voxel movement trajectory information of the ith subtask, the initial environment image when the robot performs the ith subtask, and the initial posture information when the robot performs the ith subtask into the denoising network for the 20th denoising, and the 20th noise prediction value predicted by the denoising module is obtained. Based on the 20th noise prediction value and the 19th denoising control information of the ith subtask, the 20th denoising control information of the ith subtask is obtained. Then, the 20th denoising control information of the ith subtask is determined as the prediction control information of the ith subtask.

[0171] Each denoising process in the K denoising processes is described in detail below. As can be seen from the above, each denoising process is basically the same. For ease of description, the jth denoising process is taken as an example for description.

[0172] From the above, it can be seen that during the j-th denoising process, the three-dimensional voxel movement trajectory information of the ith subtask, the initial environment image when the robot performs the ith subtask, and the initial posture information when the robot performs the ith subtask are used as control conditions. The j-th denoising control information of the ith subtask is subjected to the j-th denoising process through the denoising module to obtain the j-th noise prediction value.

[0173] In some embodiments, the above Fig.16As shown, the denoising module includes multiple network layers (for example, including M network layers). In this embodiment, the electronic device fuses the feature information of the output of each network layer in the denoising module with the three-dimensional voxel movement trajectory information of the i-th subtask, the initial environment image when the robot performs the i-th subtask, and the initial posture information when the robot performs the i-th subtask. It should be noted that before the fusion, the electronic device can extract features of the initial environment image when the robot performs the i-th subtask to obtain the image feature information of the initial environment image, and then fuse the feature information of the output of each network layer with the three-dimensional voxel movement trajectory information of the i-th subtask, the image feature information of the initial environment image when the robot performs the i-th subtask, and the initial posture information when the robot performs the i-th subtask.

[0174] In some embodiments, Fig.16 As shown, the electronic device fuses the feature information output by some network layers of the denoising module with at least the three-dimensional voxel movement trajectory information of the i-th subtask, fuses the feature information output by some network layers of the denoising module with at least the initial environment image when the robot performs the i-th subtask, and fuses the feature information output by some network layers of the denoising module with at least the initial posture information when the robot performs the i-th subtask, to obtain the j-th noise prediction value.

[0175] In one example, the feature information output by each network layer of the denoising network is fused with the three-dimensional voxel movement trajectory information of the i-th subtask. Furthermore, the feature information output by some network layers in the denoising network is fused with the initial environment image when the robot performs the i-th subtask, and the feature information output by some network layers is fused with the initial posture information when the robot performs the i-th subtask. Fig.17 As shown, the electronic device fuses the feature information 1 output by the first network layer of the denoising module with the three-dimensional voxel movement trajectory information of the i-th subtask and the initial environment image when the robot performs the i-th subtask, and uses the fused feature information as the input feature information of the second network layer. The output feature information 2 of the second network layer is fused with the three-dimensional voxel movement trajectory information and initial posture information of the i-th subtask, and used as the input feature information of the third network layer. The output feature information 3 of the third network layer is fused with the three-dimensional voxel movement trajectory information of the i-th subtask and the initial environment image when the robot performs the i-th subtask, and used as the input feature information of the fourth network layer. And so on, the feature information output by the last network layer of the denoising module is obtained as the j-th noise prediction value.

[0176] The embodiment of the present application does not limit the specific method of fusing the output feature information of the network layer in the denoising module with at least one of the three-dimensional voxel movement trajectory information of the i-th subtask, the initial environment image when the robot performs the i-th subtask, and the initial posture information when the robot performs the i-th subtask.

[0177] In an embodiment of the present application, in order to facilitate the description of the process of fusing the output feature information of the network layer in the denoising module with at least one of the three-dimensional voxel movement trajectory information of the i-th subtask, the initial environment image when the robot performs the i-th subtask, and the initial posture information when the robot performs the i-th subtask, the k-th network layer in the denoising module is taken as an example for explanation.

[0178] Specifically, for the kth network layer in the denoising module, the output feature information of the kth network layer is fused with at least one of the three-dimensional voxel movement trajectory information of the ith subtask, the initial environment image when the robot performs the ith subtask, and the initial pose information when the robot performs the ith subtask to obtain the input feature information of the k+1th network layer in the denoising module.

[0179] The embodiment of the present application does not limit the specific network structure of the denoising module, for example, a U-Net network may be used.

[0180] In an embodiment of the present application, for the kth network layer of the denoising module, the electronic device fuses the output feature information of the kth network layer with at least one of the three-dimensional voxel movement trajectory information of the ith subtask, the initial environment image when the robot performs the ith subtask, and the initial posture information when the robot performs the ith subtask, to obtain the input feature information of the k+1th network layer in the denoising module. For example, the electronic device fuses the output feature information of the kth network layer with the three-dimensional voxel movement trajectory information of the ith subtask, and uses the fused feature information as the input feature information of the k+1th network layer in the denoising module. For another example, the electronic device fuses the output feature information of the kth network layer with the three-dimensional voxel movement trajectory information of the ith subtask and the initial environment image when the robot performs the ith subtask, and uses the fused feature information as the input feature information of the k+1th network layer in the denoising module. For another example, the electronic device fuses the output feature information of the kth network layer with the three-dimensional voxel movement trajectory information of the ith subtask, the initial environment image when the robot performs the ith subtask, and the initial pose information when the robot performs the ith subtask, and uses the fused feature information as the input feature information of the k+1th network layer in the denoising module.

[0181] The embodiment of the present application does not limit the specific manner in which the electronic device fuses the output feature information of the kth network layer with at least one of the three-dimensional voxel movement trajectory information of the ith subtask, the initial environment image when the robot performs the ith subtask, and the initial pose information when the robot performs the ith subtask to obtain the input feature information of the k+1th network layer in the denoising module.

[0182] In some embodiments, the electronic device fuses the output feature information of the kth network layer with at least one of the three-dimensional voxel movement trajectory information of the ith subtask, the initial environment image when the robot performs the ith subtask, and the initial pose information when the robot performs the ith subtask through one or more convolutional layers to obtain the input feature information of the k+1th network layer in the denoising module.

[0183] The embodiment of the present application does not limit the specific manner in which the output feature information of the kth network layer is fused with at least one of the three-dimensional voxel movement trajectory information of the ith subtask, the initial environment image when the robot performs the ith subtask, and the initial posture information when the robot performs the ith subtask. For example, the output feature information of the kth network layer is added or multiplied, or spliced ​​with at least one of the three-dimensional voxel movement trajectory information of the ith subtask, the initial environment image when the robot performs the ith subtask, and the initial posture information when the robot performs the ith subtask to obtain the input feature information of the k+1th network layer. In some embodiments, the fused feature information can also be subjected to other processing (such as convolution or other processing) as the input feature information of the k+1th network layer. In some embodiments, convolution or other processing may be performed before or after at least one of the output feature information of the kth network layer, the three-dimensional voxel movement trajectory information of the i-th subtask, the initial environment image when the robot performs the i-th subtask, and the initial posture information when the robot performs the i-th subtask before fusion is performed, and the embodiments of the present application do not limit this.

[0184] In some embodiments, the electronic device uses an attention mechanism to fuse the output feature information of the kth network layer with at least one of the three-dimensional voxel movement trajectory information of the ith subtask, the initial environment image when the robot performs the ith subtask, and the initial pose information when the robot performs the ith subtask, to obtain the input feature information of the k+1th network layer.

[0185] For example, the text feature information of the prompt word is used as the key value of the attention module, the three-dimensional voxel movement trajectory information of the i-th subtask is used as the representative value of the attention module, and the initial environment image and initial posture information when the robot performs the i-th subtask are used as the query value of the attention module. They are input into the attention module for fusion processing to obtain the input feature information of the k+1-th network layer.

[0186] The above is an introduction to the specific process of fusing the output feature information of the kth network layer in the denoising module by the electronic device with at least one of the three-dimensional voxel movement trajectory information of the i-th subtask, the initial environment image when the robot performs the i-th subtask, and the initial posture information when the robot performs the i-th subtask as the input feature information of the k+1th network layer. Based on the above steps, the electronic device can determine the feature information output by the last network layer of the denoising module.

[0187] In one example, the electronic device may determine the feature information output by the last network layer of the denoising module as the j-th noise prediction value.

[0188] Next, the above steps S104 to S11 are executed to obtain the j-th denoising control information of the ith subtask based on the j-th noise prediction value and the j-1-th denoising control information of the ith subtask.

[0189] For example, the difference between the j-1th denoising control information of the ith subtask and the jth noise prediction value is determined as the jth denoising control information of the ith subtask.

[0190] For another example, the jth noise prediction value of the electronic device is multiplied by the first parameter to obtain a first value, and then the j-1th denoising control information of the i-th subtask is subtracted from the first value to obtain a second value. The second value is multiplied by the second parameter to obtain the jth denoising control information of the i-th subtask.

[0191] Exemplarily, the electronic device obtains the j-th denoising control information of the i-th subtask according to the following formula (1):

[0192]

[0193] The j-1th denoising control information of the service, γ is the first parameter, is the first value, α is the second parameter, is the second value, N(0,σ 2 I) is Gaussian distribution.

[0194] In this equation, the embodiment of the present application makes a key modification to the original diffusion strategy: the noise predictor ∈ θ Now consider not only the observation Ot and denoising step k, also considering the trajectory The parameters θ are now optimized under the condition of this additional trajectory information.

[0195] The electronic device uses the j-th denoising control information of the i-th subtask determined above as the new j-1-th denoising control information, returns to execute the above S104-C11 steps, repeats the execution K times, obtains the K-th denoising control information of the i-th subtask, and then uses the K-th denoising control information as the prediction control information of the i-th subtask.

[0196] S105: Determine the model loss of the control prediction model, and train the control prediction model based on the model loss.

[0197] In an embodiment of the present application, the electronic device determines the prediction control information of the i-th subtask through the control prediction model based on the above steps. Referring to the above method, the electronic device can determine the prediction control information of each of the N subtasks included in the training task. Then, the electronic device determines the model loss of the control prediction model.

[0198] The embodiments of the present application do not limit the specific manner in which the electronic device determines the model loss of the control prediction model.

[0199] In some embodiments, the electronic device can determine the prediction loss of the control prediction model for the i-th subtask based on the difference between the predicted control information and the actual control information of the i-th subtask. Referring to this method, the prediction loss of the control prediction model for each of the N subtasks can be determined. Then, based on the prediction losses of the N subtasks, the prediction loss of the control prediction model for the training task is determined. Similarly, the electronic device can determine the prediction loss of each training task in a batch during this round of training, and then determine the model loss of the control prediction model based on the prediction loss of each training task in the batch, and then update the parameters of the control prediction model based on the model loss. In this way, after multiple rounds of training, a trained control prediction model is obtained.

[0200] In some embodiments, it can be seen from the above that the electronic device uses K real noise values, and performs K noise addition processes on the real control information of the ith subtask to obtain the noisy control information of the ith subtask. Then, using the three-dimensional voxel movement trajectory information of the ith subtask, the initial environment image when the robot performs the ith subtask, and the initial posture information when the robot performs the ith subtask as control conditions, the noisy control information of the ith subtask is denoised K times through the control prediction model to obtain the predicted control information of the ith subtask. In each denoising process, the control prediction model predicts a noise value. In this way, K noise preset values ​​can be obtained through K denoising processes. Based on this, the electronic device can determine the model loss of the control prediction model based on the noise prediction value in the K denoising processes and the noise value when the real control information is subjected to K noise addition processes.

[0201] Exemplarily, the electronic device determines the model loss L of the control prediction model through the following formula (2):

[0202]

[0203] Among them, MSE is the mean square error, ∈ k is the true noise value, The preset noise value predicted by the control prediction model.

[0204] After the electronic device determines the model loss of the control prediction model based on the above steps, it updates the parameters in the control prediction model based on the model loss. At the same time, it determines whether the training end condition of the control prediction model is met, where the training end condition may include that the loss meets the preset loss, or the number of trainings reaches the preset number of times. If the model training end condition is not met at present, a new batch of training samples is selected, and the control prediction model with updated parameters is trained again using this batch of training samples in the same way as above, and the execution is repeated until the model training end condition is met, and finally the trained control prediction model is obtained.

[0205] The model training method provided in the embodiment of the present application obtains the description information of the training task to be performed by the robot, and the first environment image when the robot is in the initial state; based on the description information of the training task, the training task is divided into subtasks to obtain the description information of N subtasks; for the ith subtask among the N subtasks, based on the above-mentioned first environment image and the description information of the ith subtask, the three-dimensional voxel movement trajectory information of the robot when performing the ith subtask is determined; and then the three-dimensional voxel movement trajectory information of the ith subtask is processed through the control prediction model to obtain the prediction control information of the ith subtask, and based on the prediction control information of the ith subtask, the robot is controlled to perform the ith subtask. It can be seen from this that the embodiment of the present application can improve the robustness of the robot operation and the execution success rate of long sequence tasks by dividing the training task into multiple subtasks for separate execution. In addition, for each subtask, the embodiment of the present application determines the three-dimensional voxel movement trajectory information when the robot performs the subtask, and then based on the three-dimensional voxel movement trajectory information, the control information when the robot performs the subtask can be accurately determined. In this way, the robot can accurately execute the subtask based on the accurately determined control information, thereby improving the control effect of the robot.

[0206] The above introduces the training process of the control prediction model. The following introduces the process of control prediction based on the trained control prediction model.

[0207] Fig.18 This is a flow chart of a robot control method provided in an embodiment of the present application. The execution subject of the embodiment of the present application can be a device with a robot control function, such as a robot control device. The robot control device can be the above Figure 1 The server shown can also be Figure 1 The server and the robot shown in the figure are composed of a system. For the convenience of description, the following embodiments take electronic devices as an example to illustrate the method of the embodiments of the present application.

[0208] like Fig.18 As shown, the robot control method of the embodiment of the present application includes:

[0209] S201, obtaining description information of a target task to be performed by a robot and a first environment image when the robot is in an initial state.

[0210] The embodiment of the present application does not limit the specific manner in which the electronic device obtains the description information of the target task.

[0211] In a possible implementation, the description information of the target task is input by an object (such as a user). For example, the object inputs the description information of the target task to the robot by language or keystrokes.

[0212] In a possible implementation, the description information of the target task is a preset task, or a task automatically generated by the robot based on the current environment. For example, if there is something dropped on the ground, the robot can automatically generate a task to pick up the dropped object.

[0213] In the embodiment of the present application, the method of obtaining the first environment image when the robot is in the initial state includes at least the following:

[0214] Mode 1: If the execution subject of the embodiment of the present application is a robot, after the robot obtains the description information of the target task, it sends a shooting request to the camera in response to the description information of the target task. Based on the shooting request, the camera shoots a first environment image when the robot is in an initial state, and sends the shot first environment image to the robot.

[0215] Mode 2: If the execution subject of the embodiment of the present application is a server, after the server obtains the description information of the target task, it responds to the description information of the target task and sends a shooting request to the camera. Based on the shooting request, the camera shoots a first environment image when the robot is in the initial state, and sends the shot first environment image to the server.

[0216] In the embodiment of the present application, after the electronic device obtains the description information of the target task to be performed by the robot and the first environment image when the robot is in the initial state, the following step S202 is performed.

[0217] S202: Based on the description information of the target task, divide the target task into subtasks to obtain description information of N subtasks.

[0218] Wherein, N is a positive integer greater than 1.

[0219] In the embodiment of the present application, in order to enable the robot to effectively execute the long sequence task, the long sequence task is decomposed into a series of more manageable and concise subtasks. For each individual subtask, the control prediction model generates control information for the robot end effector to execute the subtask, thereby improving the robot's execution effect on the long sequence task.

[0220] Specifically, the electronic device divides the target task into a plurality of subtasks, for example, N subtasks, based on the description information of the target task obtained above, and then obtains description information of each of the N subtasks.

[0221] The embodiment of the present application does not limit the specific manner in which the electronic device divides the target task into subtasks based on the description information of the target task to obtain the description information of N subtasks.

[0222] In some embodiments, the electronic device divides the target task into N subtasks through the following steps S202-A and S202-B:

[0223] S202-A, obtaining a subtask division prompt word, where the subtask division prompt word is used to describe the subtask division control information;

[0224] S202-B, based on the description information of the target task and the subtask division prompt words, divide the target task into subtasks to obtain description information of N subtasks.

[0225] In this implementation, in order to accurately decompose the complex and long sequence target task into multiple short-term and manageable subtasks, the subtask division prompt words are also obtained. The subtask division prompt words are used to describe the subtask division control information, which usually starts and ends with grasping or releasing an object, indicating the complete interaction between the robot and the object. Then, based on the description information of the target task and the subtask division prompt words, the target task is divided into subtasks to obtain the description information of N subtasks. Through such task division, repeated or reverse trajectories in subtasks can be avoided, thereby minimizing potential ambiguity in the subsequent trajectory generation process.

[0226] In one example, the embodiment of the present application can divide the target task into N subtasks by using a trained task division model. The task division model can be a large model, such as a VLM model, which is trained to divide tasks. Specifically, Figure 4 As shown, the electronic device inputs the description information of the target task and the subtask division prompt words into the task division model. The task division model divides the target task into N subtasks based on the description information of the target task and the subtask division prompt words, and outputs the description information of each of the N subtasks.

[0227] In some embodiments, in order to further improve the accuracy of the division of subtasks, the first environment image is also considered in the task division process. At this time, the above S202-B may include: based on the first environment image, the description information of the target task and the subtask division prompt words, the target task is divided into subtasks to obtain the description information of N subtasks. In this implementation, when the electronic device divides the target task into subtasks, it can accurately divide the target task according to the description information of the target task, the first environment image and the subtask division prompt words, so that the subtasks based on the accurate division are defined as discrete stages of the robot operation task, and then the robot is controlled to accurately execute these subtasks, thereby achieving accurate execution of long sequence target tasks.

[0228] In one example, if the embodiment of the present application divides the target task into N subtasks through a trained task division model, then Figure 5 As shown, the electronic device inputs the description information of the target task, the subtask division prompt word and the first environment image into the task division model. The task division model M divides the target task T into N subtasks based on the description information of the target task T, the subtask division prompt word P and the first environment image I, and outputs the description information of each subtask in the N subtasks, that is, M(T,I,P)={S i} i＝1,…,N , where S i Represents the description information of the i-th subtask among N subtasks.

[0229] In this step, the electronic device divides the target task into N subtasks based on the above steps, and then executes the following step S203.

[0230] S203 . For the ith subtask among the N subtasks, based on the first environment image and the description information of the ith subtask, determine the three-dimensional voxel movement trajectory information when the robot performs the ith subtask.

[0231] Wherein, i is a positive integer less than or equal to N.

[0232] In the embodiment of the present application, after the electronic device divides the target task into N subtasks, it executes each subtask in sequence based on the execution order of the N subtasks. For ease of description, the i-th subtask among the N subtasks is taken as an example for description.

[0233] In an embodiment of the present application, for the ith subtask among the N subtasks, the electronic device generates a series of three-dimensional coordinates when the robot performs the ith subtask given a two-dimensional first environment image. Specifically, the electronic device determines the three-dimensional voxel movement trajectory information when the robot performs the ith subtask based on the first environment image and the description information of the ith subtask.

[0234] In some embodiments, the above S203 includes the following steps S203-A and S203-B:

[0235] S203-A, performing spatial grid division on the first environment image to obtain a first spatial grid image;

[0236] S203-B, based on the description information of the ith subtask, determine the three-dimensional voxel movement trajectory information when the robot performs the ith subtask in the first spatial grid image.

[0237] In a possible implementation, the electronic device projects the first environment image onto the top view to obtain the first top view environment image. At the same time, the depth value of each pixel in the first environment image is determined. In this way, the electronic device divides the first top view environment image into multiple grids on the top view plane based on the coordinate values ​​of the pixels in the first top view environment image, and divides the first top view environment image into multiple layers in the depth direction based on the depth value of each pixel in the first environment image, to obtain the first spatial grid image.

[0238] In some embodiments, when determining the three-dimensional voxel movement trajectory information when the robot performs the i-th subtask, the electronic device also considers the initial posture information of the robot when performing the i-th subtask.

[0239] In one example, the electronic device processes the description information of the i-th subtask, the initial posture information of the robot when performing the i-th subtask, and the first spatial grid image through a trajectory prediction model to obtain the three-dimensional voxel movement trajectory information when the robot performs the i-th subtask.

[0240] In the embodiment of the present application, after the electronic device determines the three-dimensional voxel movement trajectory information when the robot performs the i-th subtask based on the above method, it executes the following step S204.

[0241] S204. Process the three-dimensional voxel movement trajectory information of the ith subtask through the control prediction model to obtain the prediction control information of the ith subtask, and control the robot to perform the ith subtask based on the prediction control information of the ith subtask.

[0242] In the embodiment of the present application, the electronic device can determine the three-dimensional voxel movement trajectory information of each of the N subtasks according to the method of S203. Then, the three-dimensional voxel movement trajectory information of each of the N subtasks is processed by the control prediction model to obtain the prediction control information of each subtask.

[0243] In some embodiments, the above S204 includes the following steps S204-A to S204-C:

[0244] S204-A, determining the noise control information of the i-th subtask;

[0245] S204-B, taking the three-dimensional voxel movement trajectory information of the i-th subtask as the control condition, through the control prediction model, the noisy control information of the i-th subtask is denoised K times to obtain the predicted control information of the i-th subtask, where K is a positive integer.

[0246] In this implementation, the forward diffusion and reverse diffusion techniques are used to obtain the prediction control information of the ith subtask. The forward diffusion process is to determine the noise control information of the ith subtask. For example, the noise value obtained by random sampling is determined as the noise control information of the ith subtask.

[0247] In the embodiment of the present application, after the electronic device determines the noise control information of the ith subtask, it executes step S204-B, takes the three-dimensional voxel movement trajectory information of the ith subtask as the control condition, and performs K denoising processes on the noise control information of the ith subtask through the control prediction model to obtain the prediction control information of the ith subtask. This process can be understood as a reverse diffusion process.

[0248] In some embodiments, the above S204-B includes the following steps S204-B1 and S204-B2:

[0249] S204-B1, obtaining an initial environment image when the robot performs the i-th subtask, and initial posture information when the robot performs the i-th subtask;

[0250] S204-B2, taking the three-dimensional voxel movement trajectory information of the i-th subtask, the initial environment image when the robot performs the i-th subtask, and the initial posture information when the robot performs the i-th subtask as control conditions, the noisy control information of the i-th subtask is denoised K times through the control prediction model to obtain the predicted control information of the i-th subtask.

[0251] In this implementation, the control conditions for the control prediction model to denoise the noisy control information of the i-th subtask include, in addition to the three-dimensional voxel movement trajectory information of the i-th subtask, the initial environment image when the robot performs the i-th subtask and the initial posture information when the robot performs the i-th subtask.

[0252] In this implementation, if Fig.13 As shown, the electronic device inputs the three-dimensional voxel movement trajectory information of the ith subtask, the initial environment image when the robot performs the ith subtask, the initial posture information when the robot performs the ith subtask, and the noise control information of the ith subtask into the control prediction model, so that the control prediction model uses the three-dimensional voxel movement trajectory information of the ith subtask, the initial environment image when the robot performs the ith subtask, the initial posture information when the robot performs the ith subtask, and the noise control information of the ith subtask as control information, performs K denoising processes on the noise control information of the ith subtask, and obtains the predicted control information of the ith subtask.

[0253] In some embodiments, the control prediction model includes a denoising module. In this case, the above S204-B1 includes the following steps S204-B11 to S204-B14:

[0254] S204-B11, for the j-th denoising process in the K-th denoising processes, the three-dimensional voxel movement trajectory information of the i-th subtask, the initial environment image when the robot performs the i-th subtask, and the initial posture information when the robot performs the i-th subtask are used as control conditions, and the j-th denoising process is performed on the j-1 denoising control information of the i-th subtask through the denoising module to obtain the j-th noise prediction value, where j is a positive integer less than or equal to K, and if j is 1, the j-1 denoising control information is the noise control information of the i-th subtask;

[0255] S204-B12, obtaining j-th denoising control information of the ith subtask based on the j-th noise prediction value and the j-1-th denoising control information of the ith subtask;

[0256] S204-B13, taking the j-th denoising control information as the new j-1-th denoising control information, returning to repeat the above steps, and obtaining the K-th denoising control information of the i-th subtask;

[0257] S204-B14, determine the Kth denoising control information as the prediction control information of the i-th subtask.

[0258] In this implementation, the control prediction model includes a denoising module, which uses the three-dimensional voxel movement trajectory information of the i-th subtask, the initial environment image when the robot performs the i-th subtask, and the initial posture information when the robot performs the i-th subtask as control conditions, and performs multiple denoising processes on the noise-added control information of the i-th subtask. Among them, the denoising module is used to predict the noise value of each denoising process. In each denoising, the noise value currently predicted by the denoising module is subtracted from the previous denoising control information to generate new denoising control information and input it into the denoising module for further denoising. Repeat multiple times until the noise added to the control information of the i-th subtask is removed. In each denoising process, in addition to inputting the three-dimensional voxel movement trajectory information of the i-th subtask, the denoising module also inputs the initial environment image when the robot performs the i-th subtask, and the initial posture information when the robot performs the i-th subtask, so that the generated predicted control information of the i-th subtask is more in line with the actual situation, thereby improving the prediction accuracy of the control information. When the robot is controlled to perform the i-th subtask based on the accurately predicted control information, the execution effect of the task can be improved.

[0259] The following introduces the specific process of using a denoising module, taking the three-dimensional voxel movement trajectory information of the i-th subtask, the initial environment image when the robot performs the i-th subtask, and the initial posture information when the robot performs the i-th subtask as control conditions, performing multiple denoising processes on the noisy control information of the i-th subtask, and obtaining the predicted control information of the i-th subtask.

[0260] like Fig.14 As shown, the denoising process using the denoising module is an iterative process. For example, the denoising module uses the three-dimensional voxel movement trajectory information of the i-th subtask, the initial environment image when the robot performs the i-th subtask, and the initial posture information when the robot performs the i-th subtask as control conditions to perform the j-1 denoising control information of the i-th subtask for the j-th time to obtain the j-th noise prediction value. Then, based on the j-th noise prediction value and the j-th denoising control information of the i-th subtask, the j-th denoising control information of the i-th subtask is obtained. Next, the j-th denoising control information is used as the new j-1-th denoising control information. Through the denoising module, the three-dimensional voxel movement trajectory information of the i-th subtask, the initial environment image when the robot performs the i-th subtask, and the initial posture information when the robot performs the i-th subtask are used as control conditions to denoise the new j-th denoising control information to obtain a new j-th noise prediction value, and return to execute. After repeating the denoising operation K times, the K-th denoising control information can be obtained, and then the K-th denoising control information is determined as the prediction control information of the i-th subtask.

[0261] In some embodiments, the above Fig.16 As shown, the denoising module includes multiple network layers (for example, including M network layers). In this embodiment, the electronic device fuses the feature information of the output of each network layer in the denoising module with the three-dimensional voxel movement trajectory information of the i-th subtask, the initial environment image when the robot performs the i-th subtask, and the initial posture information when the robot performs the i-th subtask. It should be noted that before the fusion, the electronic device can extract features of the initial environment image when the robot performs the i-th subtask to obtain the image feature information of the initial environment image, and then fuse the feature information of the output of each network layer with the three-dimensional voxel movement trajectory information of the i-th subtask, the image feature information of the initial environment image when the robot performs the i-th subtask, and the initial posture information when the robot performs the i-th subtask.

[0262] In some embodiments, Fig.16As shown, the electronic device fuses the feature information output by some network layers of the denoising module with at least the three-dimensional voxel movement trajectory information of the i-th subtask, fuses the feature information output by some network layers of the denoising module with at least the initial environment image when the robot performs the i-th subtask, and fuses the feature information output by some network layers of the denoising module with at least the initial posture information when the robot performs the i-th subtask, to obtain the j-th noise prediction value.

[0263] Next, the above-mentioned step S204 - B11 is executed to obtain the j-th denoising control information of the ith subtask based on the j-th noise prediction value and the j-1-th denoising control information of the ith subtask.

[0264] For example, the difference between the j-1th denoising control information of the ith subtask and the jth noise prediction value is determined as the jth denoising control information of the ith subtask.

[0265] For another example, the jth noise prediction value of the electronic device is multiplied by the first parameter to obtain a first value, and then the j-1th denoising control information of the i-th subtask is subtracted from the first value to obtain a second value. The second value is multiplied by the second parameter to obtain the jth denoising control information of the i-th subtask.

[0266] The electronic device uses the j-th denoising control information of the i-th subtask determined above as the new j-1-th denoising control information, returns to execute the above S204-B11 steps, repeats the execution K times, obtains the K-th denoising control information of the i-th subtask, and then uses the K-th denoising control information as the prediction control information of the i-th subtask.

[0267] Furthermore, the technical effects of the robot control method proposed in the embodiment of the present application are introduced through specific experiments below.

[0268] In some embodiments, the method of the embodiment of the present application is extensively evaluated on RLBench (Robot Learning Benchmark). RLBench is based on the CoppelaSim (a dynamic robot simulator based on a physics engine) simulator, in which the Franka Panda (a lightweight and highly flexible robot) robot is used to perform various tasks. On RLBench, the control prediction model proposed in the embodiment of the present application and all benchmark models are trained to predict the joint positions of the next T steps and execute the first N steps of these T steps. Through experiments, it was found that T=12 and N=8 performed well in the experiment. The embodiment of the present application trained and evaluated the control prediction model proposed in the embodiment of the present application under two settings, which differed in the tasks and the number of cameras used:

[0269] Single-view setting: This setting includes a single camera view from the front and is applied to seven RLBench tasks.

[0270] Multi-view setting: This setting uses views from the front and the robot wrist. The wrist view moves dynamically during task execution and is applied to tasks with long task sequences and tasks with changes.

[0271] The embodiments of the present application respectively evaluate the capabilities of the trajectory conditional diffusion strategy (TDP) and the trajectory conditional diffusion strategy based on VLM guidance (VLM-TDP). TDP uses the real trajectory collected by demonstration as a condition, aiming to demonstrate the ability to detect a given trajectory. VLM-TDP uses the trajectory generated by VLM to demonstrate the overall performance. The experiments in the embodiments of the present application used the GPT-4o model. The strategy was trained for 500 cycles on RLBench tasks and long-term tasks, and was evaluated in 20 trials every 50 cycles. The embodiments of the present application report the results by taking the five averages of the highest success rates.

[0272] On the RLBench task, the method proposed in the above-mentioned embodiment of the present application was compared with three baseline models: the diffusion strategy (DP), the 3D diffusion strategy (DP3), and the language-guided object center diffusion strategy (Lang-o3dp). It is worth noting that the diffusion strategy and the method of the embodiment of the present application are 2D strategies with RGB images as input, while DP3 and Lang-o3dp are 3D strategies that use point clouds and RGB information. Although it seems unfair to compare 2D strategies with 3D strategies because 3D data inherently contains richer information, this comparison is still valuable. 2D data is generally easier to obtain and has the potential to outperform 3D methods in some scenarios, which highlights the importance of exploring the trade-offs and advantages of using 2D methods in specific environments. The model training and evaluation of the embodiment of the present application are as follows. Fig.19 For each of the seven tasks in RLBench, the present invention generates 40 demonstrations as training sets and 20 experiments as test sets. All methods follow the single-view setting.

[0273] Table 1: Success rates of different tasks in RLBench

[0274]

[0275]

[0276] Table 1 shows the experimental results of the method of the embodiment of the present application in seven tasks. TDP and VLM-TDP outperform the diffusion strategy in all tasks, indicating the advantage of introducing trajectory modality in the strategy. Although the method of the embodiment of the present application is a 2D strategy, its performance is comparable to the most advanced 3D strategy in most tasks, and even exceeds the 3D strategy in some tasks, with an average improvement of 3% in seven tasks.

[0277] In tasks such as opening a drawer, opening a bottle of red wine, and sweeping into a dustpan, the strategy of the embodiments of the present application is worse than the diffusion strategy, but not as good as the 3D strategy. In these tasks, the point cloud information is clearer and objects are easy to distinguish. This shows that point clouds are more effective than RGB information when processing distinguishable objects. However, in tasks such as watering, the leaves of plants are dense or obscured, and it is difficult for the 3D strategy to accurately evaluate the scene. In contrast, both the diffusion strategy and the method of the embodiments of the present application are better than the 3D strategy. These observations show that the method of the embodiments of the present application can increase the success rate to a higher level when dealing with these dense and chaotic scenes.

[0278] On the other hand, for tasks like placing a mobile phone and putting items into a drawer, the diffusion strategy and the 3D diffusion strategy have lower success rates because the mobile phone and the items are not easily distinguishable relative to the table or drawer (whether in RGB images or point clouds). In contrast, the trajectory guidance strategy and Lang-o3dp of the embodiment of the present application achieve a higher success rate by precisely aiming at the target object. In addition, the method of the embodiment of the present application performs well in the long-term task of "putting an item into a drawer", which consists of four subtasks (grabbing the handle, opening the drawer, grabbing the item and putting it into the drawer). The ability to decompose these tasks into subtasks enables the model of the embodiment of the present application to benefit from more structured execution, thereby performing well.

[0279] On the long sequence task, in order to evaluate the ability of the method of the embodiment of the present application to decompose complex problems into subtasks when solving long sequence tasks, the embodiment of the present application used a modified stacking block task in RLBench. The task goal is to pick up all the blocks in sequence and stack them in the center. Based on the definition of subtasks in the embodiment of the present application, a complete experiment usually includes two types of subtasks: picking and placing. Each subtask usually takes about 150 time steps, so stacking four blocks may exceed 1000 time steps in the entire experiment. The embodiment of the present application evaluated the model and diffusion strategy of the embodiment of the present application when stacking 1, 2, and 4 blocks. The embodiment of the present application manually divided the task into multiple subtasks and reported the results in Table 2:

[0280] Table 2-1: Success rate of long sequence tasks

[0281]

[0282] Table 2-2: Success rate of long sequence tasks

[0283]

[0284]

[0285] It can be observed that for both models, the placement subtask has a higher success rate than the picking subtask, indicating that placement is easier than picking. This task involves moving the gripper to the current position and placing the block in the center. In contrast, the picking subtask requires the model to first locate the block and then grab it. For the task of stacking multiple blocks, the policy also needs to decide the order of stacking. With trajectory conditioning, this decision burden is reduced, resulting in a 50% to 200% improvement in the performance of the execution policy, significantly improving the success rate of the entire trial.

[0286] When comparing the performance of tasks with different numbers of building blocks, both models showed a decrease in success rate when the selection complexity increased. However, the performance of the model in the embodiment of the present application decreased less. Although in theory the trajectory can guide the strategy to select the building block to be grabbed, the generated trajectory is sometimes unclear, especially when two building blocks are close together. This ambiguity comes from the limitation of the planning grid resolution. Increasing the grid resolution will increase the difficulty of VLM generating accurate trajectories. Therefore, the success rate of picking up a building block dropped from 0.88 (no ambiguity) to about 0.5 for more complex configurations.

[0287] Furthermore, the model of the embodiment of the present application shows stronger robustness than the diffusion strategy in all subtasks and complete experiments. In the simpler "placing" subtask, the robot moves to a fixed target position, the image input plays a smaller role, and the low-level data inside the robot is sufficient to plan actions. Therefore, despite the large noise (σ=0.64), the success rate of both models can be maintained at a reasonable level, but the model of the embodiment of the present application shows less performance degradation due to the additional trajectory conditions. In contrast, the "picking" subtask is more challenging because the strategy requires accurate object location information. Fig. 20 As shown, for the diffusion strategy, the image is the only source of information, and its performance drops sharply in a noisy environment, with the success rate dropping to 0% at σ = 0.32. However, the model of the embodiment of the present application guides the end effector closer to the object through trajectory guidance, so that the wrist camera can compensate for the noise at a shorter distance. Therefore, TDP still maintains 17.0% of the original performance at σ = 0.64, an absolute success rate of 0.15 in the "pick up" subtask, and a performance of 10.4% in the full task.

[0288] Next, the embodiment of the present application introduces more environmental changes. The embodiment of the present application selects four changes: background texture, object texture, smaller object and larger object. The result is as follows Fig.21 As shown. The strategy of the embodiment of the present application shows less performance degradation than the diffusion strategy under all changing conditions. It is worth noting that when the object size increases, the performance of both strategies in the "picking" subtask is improved, but the performance in the "placing" subtask decreases. Surprisingly, the overall performance of the model of the embodiment of the present application improves when the object size increases. The embodiment of the present application speculates that as the object size increases, the accuracy required for the robot to perform the task decreases, thereby reducing the difficulty of executing the strategy. Therefore, the trajectory conditions of the embodiment of the present application play a more significant role in improving performance.

[0289] The robot control method proposed in the embodiment of the present application obtains the description information of the target task to be performed by the robot, and the first environment image when the robot is in the initial state; based on the description information of the target task, the target task is divided into subtasks to obtain the description information of N subtasks; for the ith subtask among the N subtasks, based on the above-mentioned first environment image and the description information of the ith subtask, the three-dimensional voxel movement trajectory information when the robot performs the ith subtask is determined; and then the three-dimensional voxel movement trajectory information of the ith subtask is processed through the control prediction model to obtain the prediction control information of the ith subtask, and based on the prediction control information of the ith subtask, the robot is controlled to perform the ith subtask. It can be seen from this that the embodiment of the present application can improve the robustness of the robot operation and the execution success rate of long sequence tasks by dividing the target task into multiple subtasks for separate execution. In addition, for each subtask, the embodiment of the present application determines the three-dimensional voxel movement trajectory information when the robot performs the subtask, and then based on the three-dimensional voxel movement trajectory information, the control information when the robot performs the subtask can be accurately determined. In this way, the robot can accurately execute the subtask based on the accurately determined control information, thereby improving the control effect of the robot.

[0290] Combination of the above Figures 2 to 21 , describes in detail the method embodiment of the present application, and the following is combined with Fig. 22 , describe in detail the device embodiments of the present application.

[0291] Fig. 22 It is a schematic block diagram of a robot control device provided in one embodiment of the present application.

[0292] like Fig. 22 As shown, the robot control device 10 comprises:

[0293] An acquisition unit 11 is used to acquire description information of a target task to be performed by the robot and a first environment image when the robot is in an initial state;

[0294] A division unit 12 is used to divide the target task into subtasks based on the description information of the target task to obtain description information of N subtasks, where N is a positive integer greater than 1;

[0295] a trajectory determination unit 13, configured to determine, for an i-th subtask among the N subtasks, three-dimensional voxel movement trajectory information when the robot performs the i-th subtask based on the first environment image and the description information of the i-th subtask, where i is a positive integer less than or equal to N;

[0296] The prediction control unit 14 is used to process the three-dimensional voxel movement trajectory information of the i-th subtask by controlling the prediction model, obtain the prediction control information of the i-th subtask, and control the robot to perform the i-th subtask based on the prediction control information of the i-th subtask.

[0297] In some embodiments, the trajectory determination unit 13 is specifically used to perform spatial grid division on the first environment image to obtain a first spatial grid image; based on the description information of the i-th subtask, determine the three-dimensional voxel movement trajectory information of the robot when performing the i-th subtask in the first spatial grid image.

[0298] In some embodiments, the trajectory determination unit 13 is specifically used to project the first environment image onto a top-view image to obtain a first top-view environment image; determine the depth value of each pixel in the first environment image; divide the first top-view environment image into multiple grids on the top-view plane based on the coordinate values ​​of the pixel points in the first top-view environment image, and divide the first top-view environment image into multiple layers in the depth direction based on the depth value of each pixel point in the first environment image to obtain the first spatial grid image.

[0299] In some embodiments, the trajectory determination unit 13 is specifically used to obtain the initial posture information of the robot when performing the i-th subtask; based on the description information of the i-th subtask and the initial posture information of the robot when performing the i-th subtask, determine the three-dimensional voxel movement trajectory information of the robot when performing the i-th subtask in the first spatial grid image.

[0300] In some embodiments, the trajectory determination unit 13 is specifically used to process the description information of the i-th subtask, the initial posture information of the robot when performing the i-th subtask, and the first spatial grid image through a trajectory prediction model to obtain the three-dimensional voxel movement trajectory information of the robot when performing the i-th subtask.

[0301] In some embodiments, the three-dimensional voxel movement trajectory information of the i-th subtask includes marking information of the first grids occupied by the trajectory of the i-th subtask on the first overhead environment image, and the number of grids corresponding to each first grid in the depth direction.

[0302] In some embodiments, the division unit 12 is specifically used to obtain subtask division prompt words, and the subtask division prompt words are used to describe the division control information of the subtask; based on the description information of the target task and the subtask division prompt words, the target task is divided into subtasks to obtain the description information of the N subtasks.

[0303] In some embodiments, the division unit 12 is specifically configured to divide the target task into subtasks based on the first environment image, the description information of the target task and the subtask division prompt words to obtain the description information of the N subtasks.

[0304] In some embodiments, the prediction control unit 14 is used to determine the noise control information of the i-th subtask; taking the three-dimensional voxel movement trajectory information of the i-th subtask as the control condition, by controlling the prediction model, the noise control information of the i-th subtask is denoised K times to obtain the prediction control information of the i-th subtask, where K is a positive integer.

[0305] In some embodiments, the prediction control unit 14 is used to obtain the initial environment image when the robot performs the i-th subtask, and the initial posture information of the robot when performing the i-th subtask; using the three-dimensional voxel movement trajectory information of the i-th subtask, the initial environment image when the robot performs the i-th subtask, and the initial posture information when the robot performs the i-th subtask as control conditions, the noisy control information of the i-th subtask is denoised K times through the control prediction model to obtain the predicted control information of the i-th subtask.

[0306] In some embodiments, the control prediction model includes a denoising module, and the prediction control unit 14 is used for performing the j-th denoising process in the K denoising processes, taking the three-dimensional voxel movement trajectory information of the i-th subtask, the initial environment image when the robot performs the i-th subtask, and the initial posture information when the robot performs the i-th subtask as control conditions, through the denoising module, performing the j-th denoising process on the j-1 denoising control information of the i-th subtask to obtain the j-th noise prediction value, where j is less than or K. A positive integer, if the j is 1, the j-1th denoising control information is the noise addition control information of the ith subtask; based on the j-th noise prediction value and the j-1th denoising control information of the ith subtask, the j-th denoising control information of the ith subtask is obtained; the j-th denoising control information is used as the new j-1th denoising control information, and the above steps are repeated to obtain the K-th denoising control information of the ith subtask; the K-th denoising control information is determined as the prediction control information of the ith subtask.

[0307] In some embodiments, the prediction control unit 14 is used to determine the noise value obtained by random sampling as the noise addition control information of the i-th subtask.

[0308] It should be understood that the device embodiment and the method embodiment may correspond to each other, and similar descriptions may refer to the method embodiment. To avoid repetition, they will not be described here. Specifically, Fig. 22 The device shown can execute the above-mentioned robot control method embodiment, and the aforementioned and other operations and / or functions of each module in the device are respectively for realizing the above-mentioned method embodiment, which will not be repeated here for the sake of brevity.

[0309] Fig.23 It is a schematic block diagram of a model training device provided in one embodiment of the present application.

[0310] like Fig.23 As shown, the model training device 20 includes:

[0311] An acquisition unit 21 is used to acquire description information of a training task to be performed by the robot and a first environment image when the robot is in an initial state;

[0312] A division unit 22, configured to divide the training task into subtasks based on the description information of the training task to obtain description information of N subtasks, where N is a positive integer greater than 1;

[0313] a trajectory determination unit 23, configured to determine, for an i-th subtask among the N subtasks, three-dimensional voxel movement trajectory information of the robot when performing the i-th subtask based on the first environment image and the description information of the i-th subtask, where i is a positive integer less than or equal to N;

[0314] The prediction unit 24 is used to process the three-dimensional voxel movement trajectory information of the i-th subtask by controlling the prediction model to obtain the prediction control information of the i-th subtask;

[0315] The training unit 25 is used to determine the model loss of the control prediction model and train the control prediction model based on the model loss.

[0316] In some embodiments, the prediction unit 24 is specifically used to obtain the real control information of the i-th subtask; perform noise processing on the real control information to obtain the noisy control information of the i-th subtask; take the three-dimensional voxel movement trajectory information of the i-th subtask as the control condition, and perform K denoising processes on the noisy control information of the i-th subtask through the control prediction model to obtain the predicted control information of the i-th subtask, where K is a positive integer.

[0317] In some embodiments, the prediction unit 24 is specifically used to obtain the initial environment image when the robot performs the i-th subtask, and the initial posture information of the robot when performing the i-th subtask; taking the three-dimensional voxel movement trajectory information of the i-th subtask, the initial environment image when the robot performs the i-th subtask, and the initial posture information when the robot performs the i-th subtask as control conditions, the noisy control information of the i-th subtask is denoised K times through the control prediction model to obtain the predicted control information of the i-th subtask.

[0318] In some embodiments, the control prediction model includes a denoising module. In some embodiments, the prediction unit 24 is specifically used for performing the j-th denoising process in the K denoising processes, taking the three-dimensional voxel movement trajectory information of the i-th subtask, the initial environment image when the robot performs the i-th subtask, and the initial posture information when the robot performs the i-th subtask as control conditions, and performing the j-th denoising process on the j-1 denoising control information of the i-th subtask through the denoising module to obtain the j-th noise prediction value, where j is a small value. wherein the j-1 denoising control information is the noise addition control information of the ith subtask; based on the j-1 noise prediction value and the j-1 denoising control information of the ith subtask, the j-1 denoising control information of the ith subtask is obtained; the j-1 denoising control information is used as the new j-1 denoising control information, and the above steps are repeated to obtain the K-1 denoising control information of the ith subtask; and the K-1 denoising control information is determined as the prediction control information of the ith subtask.

[0319] In some embodiments, the training unit 25 is specifically used to determine the model loss of the control prediction model based on the noise prediction value in the K denoising processes and the noise added value when the real control information is subjected to K noise addition processes.

[0320] It should be understood that the device embodiment and the method embodiment may correspond to each other, and similar descriptions may refer to the method embodiment. To avoid repetition, they will not be described here. Specifically, Fig.23 The device shown can execute the above-mentioned model training method embodiment, and the aforementioned and other operations and / or functions of each module in the device are respectively for realizing the above-mentioned method embodiment, which will not be repeated here for the sake of brevity.

[0321] The above describes the device of the embodiment of the present application from the perspective of the functional module in conjunction with the accompanying drawings. It should be understood that the functional module can be implemented in hardware form, can be implemented by instructions in software form, and can also be implemented by a combination of hardware and software modules. Specifically, the steps of the method embodiment in the embodiment of the present application can be completed by the hardware integrated logic circuit and / or software form instructions in the processor, and the steps of the method disclosed in the embodiment of the present application can be directly embodied as a hardware decoding processor to perform, or a combination of hardware and software modules in the decoding processor to perform. Optionally, the software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, an electrically erasable programmable memory, a register, etc. The storage medium is located in a memory, and the processor reads the information in the memory, and completes the steps in the above method embodiment in conjunction with its hardware.

[0322] Fig.24 It is a schematic block diagram of an electronic device provided in an embodiment of the present application, and the electronic device may be the above-mentioned terminal device or server.

[0323] like Fig.24 As shown, the electronic device 40 may include:

[0324] The memory 41 and the processor 42, the memory 41 is used to store the computer program 43 and transmit the program code 43 to the processor 42. In other words, the processor 42 can call and run the computer program 43 from the memory 41 to implement the method in the embodiment of the present application.

[0325] For example, the processor 42 may be configured to execute the steps in the above method according to the instructions in the computer program 43 .

[0326] In some embodiments of the present application, the processor 42 may include but is not limited to:

[0327] General-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components, etc.

[0328] In some embodiments of the present application, the memory 41 includes but is not limited to:

[0329] Volatile memory and / or non-volatile memory. Among them, the non-volatile memory can be read-only memory (ROM), programmable ROM (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM) or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM bus random access memory (DR RAM).

[0330] In some embodiments of the present application, the computer program 43 may be divided into one or more modules, which are stored in the memory 41 and executed by the processor 42 to complete the method for recording pages provided by the present application. The one or more modules may be a series of computer program instruction segments capable of completing specific functions, and the instruction segments are used to describe the execution process of the computer program 43 in the electronic device.

[0331] like Fig.24 As shown, the electronic device 40 may further include:

[0332] The transceiver 44 may be connected to the processor 42 or the memory 41 .

[0333] The processor 42 may control the transceiver 44 to communicate with other devices, specifically, to send information or data to other devices, or to receive information or data sent by other devices. The transceiver 44 may include a transmitter and a receiver. The transceiver 44 may further include an antenna, and the number of antennas may be one or more.

[0334] It should be understood that the various components in the electronic device 40 are connected via a bus system, wherein the bus system includes not only a data bus but also a power bus, a control bus and a status signal bus.

[0335] According to one aspect of the present application, a computer storage medium is provided, on which a computer program is stored, and when the computer program is executed by a computer, the computer can perform the method of the above method embodiment. In other words, the present application embodiment also provides a computer program product containing instructions, and when the instructions are executed by a computer, the computer can perform the method of the above method embodiment.

[0336] According to another aspect of the present application, a computer program product or computer program is provided, the computer program product or computer program includes computer instructions, the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method of the above method embodiment.

[0337] In other words, when implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, a computer, a server, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (digital subscriber line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server, or data center. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or a data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a digital video disc (digital video disc, DVD)), or a semiconductor medium (e.g., a solid state drive (solid state disk, SSD)), etc.

[0338] Those of ordinary skill in the art will appreciate that the modules and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0339] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the module is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0340] The modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. For example, each functional module in each embodiment of the present application may be integrated into a processing module, or each module may exist physically separately, or two or more modules may be integrated into one module.

[0341] The above contents are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A robot control method, characterized in that: include: Acquire description information of a target task to be performed by the robot, and a first environment image when the robot is in an initial state; Based on the description information of the target task, divide the target task into subtasks to obtain description information of N subtasks, where N is a positive integer greater than 1; For an i-th subtask among the N subtasks, based on the first environment image and the description information of the i-th subtask, determine the three-dimensional voxel movement trajectory information when the robot performs the i-th subtask, where i is a positive integer less than or equal to N; The three-dimensional voxel movement trajectory information of the i-th subtask is processed by the control prediction model to obtain the prediction control information of the i-th subtask, and based on the prediction control information of the i-th subtask, the robot is controlled to perform the i-th subtask.

2. The method according to claim 1, characterized in that The determining, based on the first environment image and the description information of the i-th subtask, the three-dimensional voxel movement trajectory information when the robot performs the i-th subtask includes: Performing spatial grid division on the first environment image to obtain a first spatial grid image; Based on the description information of the i-th subtask, in the first spatial grid image, three-dimensional voxel movement trajectory information of the robot when performing the i-th subtask is determined.

3. The method according to claim 2, characterized in that The step of performing spatial grid division on the first environment image to obtain a first spatial grid image includes: Projecting the first environment image onto a top view to obtain a first top view environment image; Determine a depth value of each pixel in the first environment image; Based on the coordinate values ​​of the pixel points in the first overhead view environment image, the first overhead view environment image is divided into multiple grids on the overhead view plane, and based on the depth value of each pixel point in the first environment image, the first overhead view environment image is divided into multiple layers in the depth direction to obtain the first spatial grid image.

4. The method according to claim 3, characterized in that The step of determining, based on the description information of the i-th subtask, in the first spatial grid image, the three-dimensional voxel movement trajectory information when the robot performs the i-th subtask comprises: Obtaining initial position information of the robot when performing the i-th subtask; Based on the description information of the i-th subtask and the initial posture information of the robot when performing the i-th subtask, the three-dimensional voxel movement trajectory information of the robot when performing the i-th subtask is determined in the first spatial grid image.

5. The method according to claim 4, characterized in that The step of determining, in the first spatial grid image, the three-dimensional voxel movement trajectory information of the robot when performing the i-th subtask based on the description information of the i-th subtask and the initial posture information of the robot when performing the i-th subtask comprises: The description information of the i-th subtask, the initial posture information of the robot when performing the i-th subtask, and the first spatial grid image are processed through a trajectory prediction model to obtain the three-dimensional voxel movement trajectory information of the robot when performing the i-th subtask.

6. The method according to claim 5, characterized in that The three-dimensional voxel movement trajectory information during the i-th subtask includes marking information of the first grids occupied by the trajectory of the i-th subtask on the first overhead environment image, and the number of grids corresponding to each first grid in the depth direction.

7. The method according to any one of claims 1 to 6, characterized in that: The step of dividing the target task into subtasks based on the description information of the target task to obtain description information of N subtasks includes: Acquire a subtask division prompt word, where the subtask division prompt word is used to describe the subtask division control information; Based on the description information of the target task and the subtask division prompt words, the target task is divided into subtasks to obtain description information of the N subtasks.

8. The method according to claim 7, characterized in that The step of dividing the target task into subtasks based on the description information of the target task and the subtask division prompt words to obtain the description information of the N subtasks includes: Based on the first environment image, the description information of the target task and the subtask division prompt words, the target task is divided into subtasks to obtain the description information of the N subtasks.

9. The method according to any one of claims 1 to 6, characterized in that: The three-dimensional voxel movement trajectory information of the i-th subtask is processed by the control prediction model to obtain the prediction control information of the i-th subtask, including: Determining noise control information for the i-th subtask; Taking the three-dimensional voxel movement trajectory information of the i-th subtask as the control condition, the noisy control information of the i-th subtask is denoised K times through the control prediction model to obtain the predicted control information of the i-th subtask, where K is a positive integer.

10. The method according to claim 9, characterized in that The method of taking the three-dimensional voxel movement trajectory information of the i-th subtask as the control condition and performing K denoising processes on the noise control information of the i-th subtask by controlling the prediction model to obtain the prediction control information of the i-th subtask includes: Acquire an initial environment image when the robot performs the i-th subtask, and initial posture information when the robot performs the i-th subtask; Taking the three-dimensional voxel movement trajectory information of the i-th subtask, the initial environment image when the robot performs the i-th subtask, and the initial posture information when the robot performs the i-th subtask as control conditions, the noisy control information of the i-th subtask is denoised K times through the control prediction model to obtain the predicted control information of the i-th subtask.

11. The method according to claim 10, characterized in that The control prediction model includes a denoising module, which uses the three-dimensional voxel movement trajectory information of the i-th subtask, the initial environment image when the robot performs the i-th subtask, and the initial posture information when the robot performs the i-th subtask as control conditions, and performs K denoising processes on the noisy control information of the i-th subtask through the control prediction model to obtain the predicted control information of the i-th subtask, including: For the j-th denoising process in the K-th denoising processes, the three-dimensional voxel movement trajectory information of the i-th subtask, the initial environment image when the robot performs the i-th subtask, and the initial posture information when the robot performs the i-th subtask are used as control conditions, and the j-1-th denoising control information of the i-th subtask is subjected to the j-th denoising process through the denoising module to obtain the j-th noise prediction value, where j is a positive integer less than or equal to K, and if j is 1, the j-1-th denoising control information is the noise control information of the i-th subtask; Based on the j-th noise prediction value and the j-1-th denoising control information of the i-th subtask, obtaining the j-th denoising control information of the i-th subtask; The j-th denoising control information is used as the new j-1-th denoising control information, and the above steps are repeated to obtain the K-th denoising control information of the i-th subtask; The Kth denoising control information is determined as the prediction control information of the i-th subtask.

12. The method according to claim 9, characterized in that The determining the noise control information of the i-th subtask includes: The noise value obtained by random sampling is determined as the noise addition control information of the i-th subtask.

13. A model training method, characterized in that: include: Acquire description information of a training task to be performed by the robot, and a first environment image when the robot is in an initial state; Based on the description information of the training task, divide the training task into subtasks to obtain description information of N subtasks, where N is a positive integer greater than 1; For an i-th subtask among the N subtasks, based on the first environment image and the description information of the i-th subtask, determine the three-dimensional voxel movement trajectory information when the robot performs the i-th subtask, where i is a positive integer less than or equal to N; Processing the three-dimensional voxel movement trajectory information of the i-th subtask through the control prediction model to obtain prediction control information of the i-th subtask; A model loss of the control prediction model is determined, and the control prediction model is trained based on the model loss.

14. The method according to claim 13, characterized in that The three-dimensional voxel movement trajectory information of the i-th subtask is processed by the control prediction model to obtain the prediction control information of the i-th subtask, including: Obtaining real control information of the i-th subtask; Performing noise processing on the real control information to obtain the noisy control information of the i-th subtask; Taking the three-dimensional voxel movement trajectory information of the i-th subtask as the control condition, the noisy control information of the i-th subtask is denoised K times through the control prediction model to obtain the predicted control information of the i-th subtask, where K is a positive integer.

15. The method according to claim 14, characterized in that The method of taking the three-dimensional voxel movement trajectory information of the i-th subtask as the control condition and performing K denoising processes on the noise control information of the i-th subtask by controlling the prediction model to obtain the prediction control information of the i-th subtask includes: Acquire an initial environment image when the robot performs the i-th subtask, and initial posture information when the robot performs the i-th subtask; Taking the three-dimensional voxel movement trajectory information of the i-th subtask, the initial environment image when the robot performs the i-th subtask, and the initial posture information when the robot performs the i-th subtask as control conditions, the noisy control information of the i-th subtask is denoised K times through the control prediction model to obtain the predicted control information of the i-th subtask.

16. The method according to claim 15, characterized in that The control prediction model includes a denoising module, which uses the three-dimensional voxel movement trajectory information of the i-th subtask as a control condition, performs K denoising processes on the noisy control information of the i-th subtask through the control prediction model, and obtains the prediction control information of the i-th subtask, including: For the j-th denoising process in the K-th denoising processes, the three-dimensional voxel movement trajectory information of the i-th subtask, the initial environment image when the robot performs the i-th subtask, and the initial posture information when the robot performs the i-th subtask are used as control conditions, and the j-1-th denoising control information of the i-th subtask is subjected to the j-th denoising process through the denoising module to obtain the j-th noise prediction value, where j is a positive integer less than or equal to K, and if j is 1, the j-1-th denoising control information is the noise control information of the i-th subtask; Based on the j-th noise prediction value and the j-1-th denoising control information of the i-th subtask, obtaining the j-th denoising control information of the i-th subtask; The j-th denoising control information is used as the new j-1-th denoising control information, and the above steps are repeated to obtain the K-th denoising control information of the i-th subtask; The Kth denoising control information is determined as the prediction control information of the i-th subtask.

17. A robot control device, characterized in that: include: An acquisition unit, used to acquire description information of a target task to be performed by the robot, and a first environment image when the robot is in an initial state; A division unit, configured to divide the target task into subtasks based on the description information of the target task to obtain description information of N subtasks, where N is a positive integer greater than 1; a trajectory determination unit, configured to determine, for an i-th subtask among the N subtasks, three-dimensional voxel movement trajectory information of the robot when performing the i-th subtask based on the first environment image and the description information of the i-th subtask, where i is a positive integer less than or equal to N; A prediction control unit is used to process the three-dimensional voxel movement trajectory information of the i-th subtask by controlling the prediction model, obtain the prediction control information of the i-th subtask, and control the robot to perform the i-th subtask based on the prediction control information of the i-th subtask.

18. A model training device, characterized in that: include: An acquisition unit, used to acquire description information of a training task to be performed by the robot, and a first environment image when the robot is in an initial state; A division unit, configured to divide the training task into subtasks based on the description information of the training task to obtain description information of N subtasks, where N is a positive integer greater than 1; a trajectory determination unit, configured to determine, for an i-th subtask among the N subtasks, three-dimensional voxel movement trajectory information of the robot when performing the i-th subtask based on the first environment image and the description information of the i-th subtask, where i is a positive integer less than or equal to N; A prediction unit, configured to process the three-dimensional voxel movement trajectory information of the i-th subtask by controlling the prediction model to obtain prediction control information of the i-th subtask; A training unit is used to determine the model loss of the control prediction model and train the control prediction model based on the model loss.

19. An electronic device, characterized in that: including a processor and a memory; The memory is used to store computer programs; The processor is configured to execute the computer program to implement the method according to any one of claims 1 to 12 or 13 to 16.

20. A computer-readable storage medium, characterized in that: For storing computer programs; The computer program enables a computer to execute the method according to any one of claims 1 to 12 or 13 to 16 above.

Images