Robot motion generation methods and robot control systems

Abstract

This application provides a robot motion generation method and a robot control system. The method includes: acquiring multimodal observation data at the current moment; acquiring the modal weights of each modality and the initial noise trajectory; and iteratively exploring the initial noise trajectory in reverse chronological order based on the multimodal observation data, the modal weights of each modality, and the noise estimation model corresponding to each modality to generate a sequence of robot motion trajectories. This application can combine the noise estimation of pre-trained single-modal diffusion strategies during inference, efficiently fusing information from different modalities. This achieves the fusion of multimodal strategies while avoiding the complexity of multimodal data collection and joint training, and enhances the flexibility, real-time adaptability, robustness, and scalability of the motion generation process. Furthermore, by avoiding multimodal joint training, it significantly reduces training costs and computational resource requirements.

Description

Technical Field

[0001] This application relates to the field of robot control technology, and more specifically, to a method for generating robot motions and a control system for a robot. Background Technology

[0002] In robot policy learning, diffusion models have gradually become the mainstream method for generating robust policies due to their powerful ability to model multimodal action distributions.

[0003] Currently, diffusion models generally support the processing of single-modal data. If a diffusion model is required to support the processing of multimodal data, it is necessary to learn joint representations through end-to-end training to obtain a diffusion model that supports multimodal data.

[0004] However, when training a diffusion model supporting multimodal data using the above method, it is necessary to map multimodal data to a shared latent space. This results in the mixing of the distribution characteristics of each modality (such as the mean and variance of Gaussian noise), making them impossible to control individually. Furthermore, it is impossible to adjust the distribution combination weights during inference based on real-time requirements (such as modality confidence), neglecting the dynamic dependencies between different modalities at the distribution level. Simultaneously, this approach requires a large amount of aligned multimodal data, and the number of model parameters increases linearly with the number of modalities, leading to high training costs. Moreover, it relies on complex architectural design, making it difficult to adapt to incremental expansion with new modalities, resulting in low flexibility. Summary of the Invention

[0005] The purpose of this application is to address the shortcomings of the prior art by providing a robot motion generation method and a robot control system, thereby solving the problems of neglecting the dynamic dependence of different modalities at the distribution level, high training costs, and low flexibility in the prior art.

[0006] To achieve the above objectives, the technical solutions adopted in the embodiments of this application are as follows:

[0007] In a first aspect, one embodiment of this application provides a method for generating robot actions, the method comprising:

[0008] Acquire multimodal observation data at the current moment, wherein the multimodal observation data includes observation data under multiple modes;

[0009] The modal weights and initial noise trajectories of each modality are obtained, and the initial noise trajectories are used to characterize the initial motion trajectory before the robot is controlled.

[0010] Based on the multimodal observation data, the modal weights of each modality, and the noise estimation models corresponding to each modality, the initial noise trajectory is iteratively explored in reverse time order to generate the robot's motion trajectory sequence. The noise estimation models corresponding to each modality are obtained through pre-training. The motion trajectory sequence is used to indicate the robot's motion sequence within a preset time period after the current moment.

[0011] In one possible implementation, the step of iteratively exploring the initial noise trajectory in reverse chronological order based on the multimodal observation data, the modal weights of each modality, and the noise estimation model corresponding to each modality to generate the robot's motion trajectory sequence includes:

[0012] Based on the multimodal observation data, the modal weights of each modality, and the noise estimation model corresponding to each modality, the initial noise trajectory is iteratively explored in reverse time order to generate the robot's motion trajectory at each time step.

[0013] Based on the robot's motion trajectory at each time step, a sequence of the robot's motion trajectory is generated.

[0014] In one possible implementation, the step of iteratively exploring the initial noise trajectory in reverse chronological order based on the multimodal observation data, the modal weights of each modality, and the noise estimation model corresponding to each modality, to sequentially generate the robot's motion trajectory at each time step, includes:

[0015] Obtain the motion trajectory at the current time step. If the current time step is the first time step, the initial noise trajectory is used as the motion trajectory at the current time step. Otherwise, the motion trajectory at the current time step is determined based on the motion trajectory at the next time step.

[0016] Based on the multimodal observation data, the noise estimation model corresponding to each mode, and the action trajectory at the current time step, the noise information of each mode is determined;

[0017] Based on the mode weights of each mode and the noise information of each mode, the total noise at the current time step is determined;

[0018] Based on the motion trajectory at the current time step and the total noise at the current time step, the motion trajectory of the robot at the previous time step is generated.

[0019] In one possible implementation, determining the noise information of each mode based on the multimodal observation data, the noise estimation model corresponding to each mode, and the action trajectory at the current time step includes:

[0020] The observation data of the first mode and the action trajectory of the current time step are input into the noise estimation model corresponding to the first mode to predict the noise information of the first mode, wherein the first mode is any mode in the multi-mode.

[0021] In one possible implementation, determining the total noise at the current time step based on the mode weights of each mode and the noise information of each mode includes:

[0022] Calculate the product of the model weights and the corresponding noise information for each mode to obtain the product results for each mode;

[0023] Summing the product of each mode yields the total noise at the current time step.

[0024] In one possible implementation, generating the robot's motion trajectory at the previous time step based on the motion trajectory at the current time step and the total noise at the current time step includes:

[0025] Obtain the gradient step size coefficient, the denoising intensity coefficient, and the added Gaussian noise;

[0026] Calculate the first product of the gradient step size coefficient and the total noise at the current time step;

[0027] Calculate the difference between the motion trajectory at the current time step and the first product;

[0028] Calculate the second product of the difference and the denoising intensity coefficient;

[0029] The sum of the second product and the added Gaussian noise is calculated as the robot's trajectory in the previous time step at the current time step.

[0030] In one possible implementation, the process of determining the initial noise trajectory includes:

[0031] Obtain the trajectory dimension of the action trajectory sequence;

[0032] Based on the trajectory dimension, an initial noise trajectory is generated by independently sampling element by element from a standard Gaussian distribution.

[0033] In one possible implementation, the process of determining the mode weights of each mode includes:

[0034] Obtain the current task success rate and current confidence level for each modality;

[0035] The modality weights of each modality are calculated based on the current task success rate and the current confidence level of each modality.

[0036] In one possible implementation, the training process of the noise estimation model corresponding to each mode includes:

[0037] Obtain a first training dataset for the first modality, the first training dataset including real action sequences and observation data for the first modality;

[0038] Based on the preset noise scheduling coefficient, the real action sequence is iteratively processed at each time step to generate the action trajectory at each time step in sequence;

[0039] Based on the action trajectory at each time step, train the noise estimation model corresponding to the first mode.

[0040] Secondly, another embodiment of this application provides a robot control system, including: a processor, a storage medium, and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the robot control system is running, the processor communicates with the storage medium via the bus, and the processor executes the machine-readable instructions to perform the steps of any of the methods described in the first aspect above.

[0041] The beneficial effects of this application are as follows: By acquiring multimodal observation data, modal weights of each modality, and initial noise trajectories at the current moment, and iteratively exploring the initial noise trajectories in reverse chronological order based on the multimodal observation data, modal weights of each modality, and noise estimation models corresponding to each modality, a sequence of robot motion trajectories is generated. This allows for the combination of noise estimation from pre-trained single-modal diffusion strategies during inference, efficiently fusing information from different modalities. Thus, multimodal strategy fusion is achieved while avoiding the complexity of multimodal data collection and joint training. Furthermore, it supports dynamic adjustment of modal weights and multimodal expansion; when adding a new modality, only the corresponding noise estimation model needs to be trained for seamless integration without reconstructing the multimodal model, enhancing the flexibility, real-time adaptability, robustness, and scalability of the motion generation process. Moreover, by avoiding multimodal joint training, training costs and computational resource requirements are significantly reduced. Attached Figure Description

[0042] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0043] Figure 1 This is a schematic flowchart of a robot motion generation method provided in an embodiment of this application;

[0044] Figure 2 This is a schematic flowchart illustrating the process of generating a robot motion trajectory sequence in the robot motion generation method provided in this application embodiment.

[0045] Figure 3 This is a flowchart illustrating the process of sequentially generating the robot's motion trajectory at each time step in the robot motion generation method provided in this application embodiment;

[0046] Figure 4 This is a schematic flowchart illustrating the process of determining the total noise at the current time step in the robot motion generation method provided in this application embodiment.

[0047] Figure 5 This is a flowchart illustrating the process of generating the robot's motion trajectory at the previous time step in the robot motion generation method provided in this application embodiment;

[0048] Figure 6 A schematic flowchart illustrating the determination of an initial noise trajectory in the robot motion generation method provided in this application embodiment;

[0049] Figure 7 A schematic flowchart illustrating the determination of modal weights in the robot motion generation method provided in this application embodiment;

[0050] Figure 8 This is a schematic diagram of a process for training noise estimation models corresponding to each modality in the robot motion generation method provided in the embodiments of this application.

[0051] Figure 9 This is a schematic diagram of the control system of the robot provided in an embodiment of this application. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the accompanying drawings in this application are for illustrative and descriptive purposes only and are not intended to limit the scope of protection of this application. Furthermore, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in this application illustrate operations implemented according to some embodiments of this application. It should be understood that the operations in the flowcharts may not be implemented in sequence, and steps without logical contextual relationships may be reversed or implemented simultaneously. In addition, those skilled in the art, guided by the content of this application, may add one or more other operations to the flowcharts, or remove one or more operations from the flowcharts.

[0053] Furthermore, the described embodiments are merely some, not all, of the embodiments of this application. The components of the embodiments of this application described and illustrated herein can typically be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0054] It should be noted that the term "comprising" will be used in the embodiments of this application to indicate the presence of the features declared thereafter, but does not exclude the addition of other features.

[0055] Currently, diffusion models generally support the processing of single-modal data. If a diffusion model is required to support the processing of multimodal data, it is necessary to learn joint representations through end-to-end training to obtain a diffusion model that supports multimodal data.

[0056] However, when training a diffusion model supporting multimodal data using the above method, it is necessary to map multimodal data to a shared latent space. This results in the mixing of the distribution characteristics of each modality (such as the mean and variance of Gaussian noise), making them impossible to control individually. Furthermore, it is impossible to adjust the distribution combination weights during inference based on real-time requirements (such as modality confidence), neglecting the dynamic dependencies between different modalities at the distribution level. Simultaneously, this approach requires a large amount of aligned multimodal data, and the number of model parameters increases linearly with the number of modalities, leading to high training costs. Moreover, it relies on complex architectural design, making it difficult to adapt to incremental expansion with new modalities, resulting in low flexibility.

[0057] This application proposes a robot motion generation method to address the aforementioned problems. It acquires multimodal observation data, modal weights for each modality, and an initial noise trajectory at the current moment. Based on the multimodal observation data, modal weights, and noise estimation models corresponding to each modality, it iteratively explores the initial noise trajectory in reverse chronological order to generate a sequence of robot motion trajectories. This method can combine noise estimation from pre-trained single-modal diffusion strategies during inference, achieving multimodal strategy fusion. Furthermore, it supports dynamic weights and modal expansion, eliminating the need to reconstruct the multimodal model when adding new modalities, thus enhancing the flexibility, real-time adaptability, and scalability of the motion generation process. Additionally, it reduces training costs and improves the robustness of the motion generation process.

[0058] First, the relevant background of the robot motion generation method provided in the embodiments of this application will be described in detail.

[0059] It is understood that the robot motion generation method provided in this application embodiment can be deployed in a robot, for example, in a robot or a robot-like device.

[0060] For example, a humanoid robot is a type of robot that mimics the appearance, movement, or behavior patterns of a human. It typically possesses human-like structures such as a head, torso, arms, and legs, and can perform human-like functions such as running, dancing, jumping, swimming, fighting, grasping, and facial expression interaction. Furthermore, each arm is composed of multiple joint mechanisms, enabling it to perform different actions depending on the scenario.

[0061] Optionally, the robot may include a mechanical structure, a drive system, a sensing system, a control system, and a power system.

[0062] The mechanical structure includes at least one robotic arm, end effectors corresponding to each robotic arm, two legs, and two feet. The mechanical structure is used to perform specific actions.

[0063] The drive system includes servo motors, etc., which are used to control the rotation or linear motion of each joint in the robotic arm and provide precise power output.

[0064] Optionally, the sensing system may include sensor devices of different modalities such as torque sensors, vision cameras, inertial measurement units, and tactile sensors. The sensing system is used to detect environmental information in real time to obtain multimodal observation data in order to realize the control of the robot.

[0065] The control system includes a processor and a memory, etc. The memory stores machine-readable instructions that can be executed by the processor. When the robot's control system is running, the processor executes the machine-readable instructions to perform the steps of the robot motion generation method provided in this application embodiment, thereby controlling the operation of the robot.

[0066] The robot motion generation method provided in this application will be described in detail below with reference to several embodiments.

[0067] Figure 1 This is a flowchart illustrating a robot motion generation method provided in an embodiment of this application, with reference to... Figure 1 As shown, the executing entity of this method can be any electronic device with processing capabilities, such as the control system of the robot described above. The method includes:

[0068] S101. Obtain the multimodal observation data at the current moment.

[0069] It is understandable that during the robot motion generation process, the robot can predict the motion trajectory sequence over a future period of time by using the multimodal observation data at the current moment.

[0070] The current moment refers to the instant at which the robot begins to generate the action sequence, that is, the current time point when it acquires multimodal observation data and starts action generation.

[0071] For example, acquiring multimodal observation data at the current moment can be understood as acquiring the observation data of the first mode at the current moment, acquiring the observation data of the second mode at the moment corresponding to the current moment, and using the acquired observation data of the first mode and the observation data of the second mode as the multimodal observation data at the current moment.

[0072] Multimodal observation data includes observation data in multiple modes.

[0073] For example, the modality type of multimodal observation data may include one or more of the following: visual modalities, non-visual modalities, and advanced modalities.

[0074] The visual modalities include RGB images, depth maps / point clouds, infrared / thermal imaging, and event cameras. The non-visual modalities include tactile / force sensing, inertial measurement units (IMUs), sound / speech sensing, and proprioception. Specifically, proprioception refers to data from internal sensors such as joint angles and motor currents.

[0075] Among them, higher modalities include: language commands, radar / liDAR, and brain-computer interfaces (BCI).

[0076] S102. Obtain the modal weights of each mode and the initial noise trajectory.

[0077] Optionally, the modal weights of each modality are used to quantify the relative importance of different modalities to the final generated motion trajectory.

[0078] For example, the modal weights of each modality can be fixed modal weights obtained in advance, or dynamic modal weights of each modality calculated by a predefined modal weight calculation formula, or dynamic modal weights of each modality predicted by a pre-trained modal weight prediction model using multimodal observation data.

[0079] In one example, the modal weights of each mode can be any constant between 0 and 1.

[0080] In another example, the modal weights of each mode can be any constant between 0 and 1, and the sum of the modal weights of each mode can also be 1.

[0081] Optionally, the initial noise trajectory is used to characterize the initial motion trajectory before the robot is controlled. The initial noise trajectory provides the robot with a random starting point motion and is the starting point of the generated motion trajectory sequence, so that the initial noise trajectory can be converted into a physically feasible motion sequence through multimodal observation data in subsequent steps.

[0082] For example, the initial noise trajectory can be sampled from a standard Gaussian distribution, thereby enabling the initial noise trajectory to be obtained quickly, while improving the reversibility and convergence of the generated robot motion trajectory sequence.

[0083] For example, the initial noise trajectory can be sampled from a uniform distribution, making it simple and easy to implement and avoiding extreme values.

[0084] For example, the initial noise trajectory can be sampled from a mixture of Gaussian distributions, thereby enabling the capture of multi-peak action distributions.

[0085] For example, the initial noise trajectory can also be sampled from the robot's historical motion data, so that the obtained initial noise trajectory is closer to the real motion distribution, thus accelerating the denoising convergence.

[0086] For example, the initial noise trajectory can also be obtained through a pre-trained initial noise trajectory generation network. The multimodal observation data at the current time is input into the initial noise trajectory generation network to generate the initial noise trajectory, thereby accelerating the denoising convergence.

[0087] S103. Based on the multimodal observation data, the modal weights of each modality, and the noise estimation model corresponding to each modality, the initial noise trajectory is iteratively explored in reverse time order to generate the robot's motion trajectory sequence.

[0088] Optionally, after obtaining the model weights of each mode and the noise estimation model corresponding to each mode, the robot's motion trajectory sequence can be generated by iteratively exploring the initial noise trajectory in reverse time order based on the multimodal observation data, the mode weights of each mode, and the noise estimation model corresponding to each mode.

[0089] Among them, the noise estimation model corresponding to each mode is obtained through pre-training, and the motion trajectory sequence is used to indicate the robot's motion sequence within a preset time period after the current moment.

[0090] For example, a diffusion policy (DP) can be pre-trained for each modality, and the diffusion policy (DP) for each modality can be used as the noise estimation model for that modality. The noise estimation model for each modality is used to capture the semantic information of each modality and to model the geometric constraints of each modality.

[0091] For example, at each time step, noise prediction can be performed based on multimodal observation data, modal weights of each modality, and noise estimation models corresponding to each modality. The predicted noise can then be added sequentially to the initial noise trajectory in reverse order of the time steps to generate a sequence of robot motion trajectories.

[0092] For example, at each time step, noise prediction can be performed based on multimodal observation data, modal weights of each modality, noise estimation models corresponding to each modality, and initial noise trajectories to generate noise trajectories for each time step. Then, in reverse chronological order, noise prediction can be performed based on multimodal observation data, modal weights of each modality, and noise estimation models corresponding to each modality. The predicted noise can be added to the noise trajectories for each time step in sequence to generate a sequence of robot motion trajectories.

[0093] For example, taking a robot grasping task as an example, the multimodal observation data may include the RGB image, point cloud data and body perception data at the current moment. By executing S101-S103, a motion trajectory sequence can be generated. This motion trajectory sequence can represent the pose of the robotic arm and the pose of the end effector every 0.1 seconds within the next 2 seconds.

[0094] For example, after obtaining a motion trajectory sequence representing the pose of the robotic arm and the pose of the end effector every 0.1 seconds within the next 2 seconds, the robot can execute the motion trajectory sequence according to the motion trajectory sequence.

[0095] In this embodiment, by acquiring the current multimodal observation data, modal weights of each modality, and initial noise trajectories, and based on the multimodal observation data, modal weights of each modality, and the corresponding noise estimation models for each modality, the initial noise trajectories are iteratively explored in reverse chronological order to generate a sequence of robot motion trajectories. This allows for the combination of noise estimation from pre-trained single-modal diffusion strategies during inference, efficiently fusing information from different modalities. This achieves multimodal strategy fusion while avoiding the complexity of multimodal data collection and joint training. Furthermore, it supports dynamic adjustment of modal weights and multimodal expansion; when adding a new modality, only the corresponding noise estimation model needs to be trained for seamless integration, without the need to reconstruct the multimodal model. This enhances the flexibility, real-time adaptability, robustness, and scalability of the motion generation process. Moreover, by avoiding multimodal joint training, training costs and computational resource requirements are significantly reduced.

[0096] In one possible implementation, Figure 2 This is a flowchart illustrating the process of generating a robot motion trajectory sequence in the robot motion generation method provided in this application embodiment, with reference to... Figure 2As shown, in S103 above, based on multimodal observation data, modal weights of each modality, and noise estimation models corresponding to each modality, the initial noise trajectory is iteratively explored in reverse time order to generate the robot's motion trajectory sequence, including:

[0097] S201. Based on the multimodal observation data, the modal weights of each modality, and the noise estimation model corresponding to each modality, the initial noise trajectory is iteratively explored in reverse time order to generate the robot's motion trajectory at each time step.

[0098] Optionally, at each time step, noise prediction can be performed based on multimodal observation data, modal weights of each modality, and noise estimation models corresponding to each modality. The predicted noise can then be added sequentially to the initial noise trajectory in reverse order of time steps to generate the robot's motion trajectory at each time step.

[0099] Optionally, at each time step, noise prediction can be performed based on multimodal observation data, modal weights of each modality, noise estimation models corresponding to each modality, and initial noise trajectories to generate noise trajectories for each time step. Then, in reverse chronological order, noise prediction can be performed based on multimodal observation data, modal weights of each modality, and noise estimation models corresponding to each modality. The predicted noise can be added to the noise trajectories for each time step sequentially to generate the robot's motion trajectory at each time step.

[0100] The robot's motion trajectory at each time step refers to the robot's control commands at each time step, which can be used to control the robot's joint angles and end effector pose.

[0101] For example, continuing with the robot grasping task mentioned above, the multimodal observation data includes the RGB image, point cloud data and body perception data at the current moment. By executing S201, assuming that the duration of the generated motion trajectory sequence is 2 seconds and the control frequency is 10Hz, the robot's motion trajectory at 20 time steps can be generated.

[0102] S202. Generate the robot's motion trajectory sequence based on the robot's motion trajectory at each time step.

[0103] Optionally, after obtaining the robot's motion trajectory at each time step, the robot's motion trajectory at each time step can be sorted in reverse chronological order to generate a sequence of robot motion trajectories.

[0104] For example, continuing with the robot grasping task mentioned above, assuming the motion trajectory duration is 2 seconds and the control frequency is 10Hz, the robot's motion trajectory is generated in 20 time steps. By executing S202, a motion trajectory sequence including the motion trajectory in 20 time steps can be generated.

[0105] By using multimodal observation data, modal weights of each modality, and noise estimation models corresponding to each modality, the initial noise trajectory is iteratively explored in reverse time order, generating the robot's motion trajectory at each time step in sequence. Based on the robot's motion trajectory at each time step, a sequence of robot motion trajectories is generated. The motion trajectory can be gradually refined by spreading time steps, thereby achieving global optimization of the initial noise trajectory. Furthermore, the time step of motion generation can be decoupled from the time step of physical control, thus efficiently fusing information from different modalities.

[0106] In one possible implementation, Figure 3 This is a flowchart illustrating the process of sequentially generating the robot's motion trajectory at each time step in the robot motion generation method provided in this application embodiment, referring to... Figure 3 As shown, in S201 above, based on multimodal observation data, modal weights of each modality, and noise estimation models corresponding to each modality, the initial noise trajectory is iteratively explored in reverse time order to generate the robot's motion trajectory at each time step, including:

[0107] S301. Obtain the motion trajectory at the current time step.

[0108] Optionally, during the iterative exploration of the initial noise trajectory, the motion trajectory at each time step can be obtained. The number of time steps can be determined by combining the control frequency and the duration of the generated motion trajectory sequence.

[0109] If the current time step is the first time step, the initial noise trajectory is used as the action trajectory of the current time step; otherwise, the action trajectory of the current time step is determined based on the action trajectory of the next time step.

[0110] S302. Based on the multimodal observation data, the noise estimation model corresponding to each mode, and the action trajectory at the current time step, determine the noise information of each mode.

[0111] Optionally, at the current time step, based on the observation data of each mode in the multimodal observation data, the motion trajectory at the current time step, and the noise estimation model corresponding to each mode, the noise information of each mode is estimated to obtain the noise information of each mode.

[0112] The noise information of each mode can be understood as the quantitative correction information of the motion trajectory deviation implied in the observation data of the current mode. Specifically, when the initial noise trajectory is the initial noise trajectory sampled from the standard Gaussian distribution, the noise information of each mode can be represented in the form of a Gaussian distribution to indicate the direction and uncertainty of the correction.

[0113] S303. Determine the total noise at the current time step based on the mode weights of each mode and the noise information of each mode.

[0114] Optionally, after obtaining the noise information and modal weights of each mode, the total noise at the current time step can be determined by weighting and combining the noise information of each mode with the modal weights, thereby achieving robust action generation.

[0115] Optionally, after obtaining the noise information and modal weights of each modality, the noise information and modal weights of each modality can be combined through an attention mechanism to determine the total noise at the current time step, thereby achieving robust action generation.

[0116] Optionally, after obtaining the noise information and mode weights of each mode, the noise information and mode weights of each mode can be probabilistically fused using covariance to determine the total noise at the current time step, thereby achieving robust action generation.

[0117] The total noise at the current time step can indicate the optimization direction of the motion trajectory sequence jointly driven by multimodal observation data.

[0118] For example, when the initial noise trajectory is an initial noise trajectory sampled from a standard Gaussian distribution, and the noise information of each mode is represented by the direction and uncertainty of the correction in the form of a Gaussian distribution, by determining the total noise, the noise information of each mode can be integrated to generate the optimal action adjustment strategy. This can transform the "hard constraints" in traditional motion planning into probabilistic soft constraints, and naturally handle multimodal uncertainties through Gaussian distribution.

[0119] S304. Based on the motion trajectory at the current time step and the total noise at the current time step, generate the robot's motion trajectory at the previous time step.

[0120] Optionally, after obtaining the total noise at the current time step, the motion trajectory at the current time step can be optimized using the total noise at the current time step, thereby generating the robot's motion trajectory at the previous time step.

[0121] For example, the robot's motion trajectory at the previous time step can be obtained by subtracting the total noise at the current time step from the motion trajectory at the current time step.

[0122] By acquiring the motion trajectory at the current time step and determining the noise information of each mode using multimodal observation data, noise estimation models corresponding to each mode, and the motion trajectory at the current time step, the total noise at the current time step is determined using the modal weights and noise information of each mode. This allows for the generation of a noise correction signal that integrates the advantages of each mode. Thus, the robot's motion trajectory at the previous time step can be generated from the motion trajectory at the current time step and the total noise at the current time step. This ensures that the fused motion trajectory simultaneously meets high-level task requirements and low-level motion constraints. Furthermore, it can automatically reduce the impact of unreliable modal data through modal weight adjustment, significantly improving robustness in unstructured environments. Simultaneously, the time-step iterative trajectory generation method ensures spatiotemporal continuity at the motion sequence level. In addition, it eliminates the computational overhead of explicit constraint processing, significantly reducing the iterative cost of modal expansion and achieving a balance between computational efficiency and real-time performance.

[0123] In one possible implementation, step S302 above determines the noise information of each mode based on multimodal observation data, the noise estimation model corresponding to each mode, and the action trajectory at the current time step, including:

[0124] The observation data of the first mode and the motion trajectory at the current time step are input into the noise estimation model corresponding to the first mode to predict the noise information of the first mode.

[0125] The first mode is any mode in the multimodal model.

[0126] For example, the observation data of the first mode and the action trajectory of the current time step are input into the noise estimation model corresponding to the first mode. The encoder in the noise estimation model corresponding to the first mode extracts the observation features in the observation data of the first mode, and the time series encoder in the noise estimation model corresponding to the first mode extracts the trajectory features in the action trajectory of the current time step. The observation features and trajectory features are then concatenated or fused by cross-attention to generate fused features.

[0127] For example, based on the fused features, the estimated noise is calculated using the network entity in the noise estimation model corresponding to the first modality.

[0128] For example, the current time step is embedded through sinusoidal position coding or learnable embedding to adjust the intensity of the estimated noise, thereby obtaining noise information for the first mode.

[0129] For example, when the initial noise trajectory is a sampled initial noise trajectory from a standard Gaussian distribution, the noise information corresponding to each mode can be represented as follows: ,in, , For modality Corresponding noise information For modality The corresponding noise estimation model, For the current time step The trajectory of movement, For modality Observational data, It follows a Gaussian distribution. The mean of a Gaussian distribution is given. The expected value of the noise information represents the mode. Current action trajectory The optimal correction direction and magnitude, The covariance matrix of the noise information represents the mode. Uncertainty assessment of the correction amount.

[0130] By inputting the observation data of the first modality and the motion trajectory at the current time step into the noise estimation model corresponding to the first modality, the noise information of the first modality is predicted. This preserves the specificity of the noise information of each modality, avoids inter-modal interference, and allows each modality to focus on its strengths. Combining these modalities maximizes efficiency and improves task adaptability and the robustness of the motion generation process. Furthermore, predicting the noise information of the first modality from the motion trajectory at the current time step maintains the spatiotemporal consistency of the motion sequence. In addition, parallel computation and real-time inference of noise information from multiple modalities are possible. Moreover, it reduces data and computational requirements, facilitating deployment and expansion.

[0131] In one possible implementation, Figure 4 This is a flowchart illustrating the process of determining the total noise at the current time step in the robot motion generation method provided in this application embodiment, with reference to... Figure 4 As shown, in S303 above, the total noise at the current time step is determined based on the mode weights of each mode and the noise information of each mode, including:

[0132] S401. Calculate the product of the model weights and the corresponding noise information for each mode to obtain the product results for each mode.

[0133] Optionally, the model weights of each mode and the product of the noise information corresponding to each mode can be calculated separately to obtain the product results of each mode. The product results of each mode are used to indicate the contribution intensity and direction of each mode to the correction of the robot's motion trajectory in the current task scenario, and can represent the local correction parameters of each mode to the motion trajectory at the current time step.

[0134] For example, when the initial noise trajectory is a sampled initial noise trajectory from a standard Gaussian distribution, the noise information corresponding to each mode can be represented as follows: Then, the product of the modes can be expressed as: ,in, , For modality Modal weights, For modality The product result, Inheriting Gaussianity reflects the uncertainty of the correction amount. For modality Corresponding noise information For modality The corresponding noise estimation model, For the current time step The trajectory of movement, For modality Observational data, It follows a Gaussian distribution. The mean of a Gaussian distribution is given. The expected value of the noise information represents the mode. Current action trajectory The optimal correction direction and magnitude, The covariance matrix of the noise information represents the mode. Uncertainty assessment of the correction amount.

[0135] S402. Sum the product of each mode to obtain the total noise at the current time step.

[0136] Optionally, the sum of the product results of each mode can be calculated to obtain the total noise at the current time step.

[0137] For example, when the initial noise trajectory is an initial noise trajectory sampled from a standard Gaussian distribution, the noise information corresponding to each mode is... Modality The product can be expressed as Then the total noise at the current time step can be expressed as .in, For modality Modal weights, For modality The product result, Inheriting Gaussianity reflects the uncertainty of the correction amount. For modality Corresponding noise information For modality The corresponding noise estimation model, For the current time step The trajectory of movement, For modality Observational data, It follows a Gaussian distribution. The mean of a Gaussian distribution is given. The expected value of the noise information represents the mode. Current action trajectory The optimal correction direction and magnitude, The covariance matrix of the noise information represents the mode. Uncertainty assessment of the correction amount.

[0138] By calculating the product of the model weights and corresponding noise information for each modality, the product results of each modality are obtained. Then, the product results of all modalities are summed to obtain the total noise at the current time step. This allows for the quantification of the contribution strength and direction of each modality to action correction through a weighted expression, and the resolution of conflicts and propagation of uncertainties through linear combination. Furthermore, due to its lightweight computation, it also supports high-frequency policy updates.

[0139] In one possible implementation, Figure 5 This is a flowchart illustrating the process of generating the robot's motion trajectory at the previous time step in the robot motion generation method provided in this application embodiment. (Refer to...) Figure 5 As shown, in step S304 above, the robot's motion trajectory at the previous time step is generated based on the motion trajectory at the current time step and the total noise at the current time step, including:

[0140] S501, obtain the gradient step size coefficient, the denoising intensity coefficient, and the added Gaussian noise.

[0141] Optionally, the gradient step size coefficient can be obtained. Noise reduction intensity coefficient and additional Gaussian noise .

[0142] Wherein, gradient step size coefficient The update step size used to adjust the noise estimation affects the convergence speed.

[0143] Among them, the noise reduction intensity coefficient Used to control the proportion of total noise contributing to trajectory updates.

[0144] Among them, Gaussian noise is added. This is used to prevent deterministic updates from getting stuck in local optima and to enhance exploratory behavior.

[0145] For example, , , .

[0146] By using gradient step size coefficient and denoising intensity coefficient, the process of correcting the motion trajectory of the current time step through total noise can be controlled. By adding Gaussian noise, it can be ensured that the strategy retains a certain degree of randomness when generating the trajectory, thus preventing the robotic arm's movements from falling into a repetitive pattern.

[0147] S502. Calculate the first product of the gradient step size coefficient and the total noise at the current time step.

[0148] Optionally, the gradient step size coefficient is calculated. Total noise at the current time step First product .

[0149] S503. Calculate the difference between the motion trajectory at the current time step and the first product.

[0150] Optionally, calculate the current time step. movement trajectory with the first product The difference .

[0151] S504. Calculate the second product of the difference and the denoising intensity coefficient.

[0152] Optionally, calculate the difference. With denoising intensity coefficient The second product .

[0153] S505. Calculate the sum of the second product and the added Gaussian noise, which is the robot's trajectory in the previous time step at the current time step.

[0154] Optionally, calculate the second product. With added Gaussian noise and This refers to the robot's trajectory at the previous time step in the current time step.

[0155] The robot's motion trajectory at the previous time step is calculated by using gradient step size coefficient, denoising intensity coefficient, added Gaussian noise, motion trajectory at the current time step, and total noise at the current time step. This allows for the generation of robust and adaptive motion sequences under multimodal constraints.

[0156] In one possible implementation, Figure 6 This is a flowchart illustrating the determination of an initial noise trajectory in the robot motion generation method provided in this application embodiment, with reference to... Figure 6 As shown, the process of determining the initial noise trajectory includes:

[0157] S601. Obtain the trajectory dimension of the action trajectory sequence.

[0158] Optionally, the trajectory dimensions of the motion trajectory sequence to be generated can be obtained. The trajectory dimensions include the motion dimension and the number of time steps.

[0159] For example, suppose we need to generate a sequence of motion trajectories with 20 time steps, each motion being 7-dimensional (e.g., a 7-DOF robotic arm), then The motion dimension is 7, and the time step count is 20.

[0160] S602. Based on the trajectory dimension, sample elements independently from the standard Gaussian distribution to generate the initial noise trajectory.

[0161] Optionally, based on the trajectory dimension, for Each element The initial noise trajectory is obtained by independently sampling from a standard Gaussian distribution. .in, Indicates the first The first joint Each time step.

[0162] For example, taking a robot grasping task, the motion trajectory sequence is the motion trajectory sequence of a 7-joint robotic arm within 2 seconds, and the control frequency is 10Hz, as an example, the initial noise trajectory... It is a 140-dimensional vector, and each element of the vector is independently sampled from... ,For example: .

[0163] By using the trajectory dimension of the motion trajectory sequence and independently sampling element by element from the standard Gaussian distribution based on the trajectory dimension, an initial noise trajectory can be generated. This ensures that the obtained initial noise trajectory allows the strategy to generate diverse motion trajectory sequences, avoids getting trapped in local optima, and also provides a random starting point for robot motion generation.

[0164] In one possible implementation, Figure 7 This is a flowchart illustrating the determination of modal weights in the robot motion generation method provided in this application embodiment, with reference to... Figure 7 As shown, the process of determining the mode weights for each mode includes:

[0165] S701. Obtain the current task success rate and current confidence level for each modality.

[0166] Optionally, for each modality, the task success rate ri within a sliding window can be maintained using the sliding window method. The task success rate can be, for example, the success rate of the most recent 10 actions.

[0167] Optionally, for each modality, the current confidence level rj can be obtained based on the historical action success rate, the inverse variance of noise estimation, or the modality-specific reward.

[0168] S702. Calculate the modal weights of each modality based on the current task success rate and the current confidence level of each modality.

[0169] Optionally, the modality weights of each modality can be calculated based on the current task success rate and the current confidence level of each modality.

[0170] For example, for mode i, the mode weights of mode i can be calculated with reference to the following formula. :

[0171]

[0172] in, This represents the total number of modes.

[0173] By obtaining the current task success rate and current confidence level of each modality, and calculating the modality weight of each modality based on the current task success rate and current confidence level of each modality, it can respond to changes in the quality of observed data in real time and always prioritize trusting high-confidence modalities.

[0174] In one possible implementation, Figure 8 This is a flowchart illustrating the process of training noise estimation models for each modality in the robot motion generation method provided in this application embodiment. (Refer to...) Figure 8 As shown, the training process of the noise estimation model corresponding to each mode includes:

[0175] S801. Obtain the first training dataset in the first mode.

[0176] Optionally, a first training dataset for the first modality can be obtained. This first training dataset includes real action sequences and observation data for the first modality.

[0177] Alternatively, a diffusion model can be used as the initial noise estimation model, for example, a denoising diffusion probabilistic model (DDPM).

[0178] S802. Based on the preset noise scheduling coefficient, the real action sequence is iteratively processed at each time step to generate the action trajectory at each time step.

[0179] Optionally, Gaussian noise is gradually added to the real action sequence according to a preset noise scheduling coefficient αt, thereby destroying the data through Gaussian noise and constructing a diffusion chain.

[0180] For example, the first The movement trajectory at each time step It can be:

[0181]

[0182] in, , This is the noise dispatch coefficient.

[0183] S803. Based on the action trajectory at each time step, train the noise estimation model corresponding to the first mode.

[0184] Optionally, the motion trajectory at each time step can be used as a basis. Train the noise estimation model corresponding to the first mode.

[0185] For example, the training objective could be to minimize the noise estimation error.

[0186] For example, after training, each modality is saved. The noise estimation model ϵθi and the noise scheduling parameters {αt,σt} are given.

[0187] This application also provides a robot, such as... Figure 9 As shown, Figure 9 The schematic diagram of the robot control system provided in this application embodiment includes: a processor 901 and a memory 902, and optionally, a bus 903. The memory 902 stores machine-readable instructions executable by the processor 901. When the robot control system is running, the processor 901 and the memory 902 communicate via the bus 903. When the machine-readable instructions are executed by the processor 901, the steps of the robot motion generation method described above are performed.

[0188] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A robot action generation method characterized by comprising: The method comprises the following steps: acquiring multi-modal observation data at the current time, the multi-modal observation data comprising observation data in multiple modalities; acquiring modal weights of each modality and an initial noise trajectory, the initial noise trajectory being used to represent an initial action trajectory before action control of the robot; iteratively exploring the initial noise trajectory in a reverse order of time according to the multi-modal observation data, the modal weights of each modality and a noise estimation model corresponding to each modality, to generate an action trajectory sequence of the robot, the noise estimation model corresponding to each modality being obtained by pre-training, the action trajectory sequence being used to indicate an action sequence of the robot within a preset time period after the current time, and the modal weights of each modality being used to quantify the relative importance of different modalities to the finally generated action trajectory.

2. The robot action generation method of claim 1, wherein, The step of iteratively exploring the initial noise trajectory in a reverse order of time according to the multi-modal observation data, the modal weights of each modality and the noise estimation model corresponding to each modality, to generate the action trajectory sequence of the robot, comprises the following steps: iteratively exploring the initial noise trajectory in a reverse order of time according to the multi-modal observation data, the modal weights of each modality and the noise estimation model corresponding to each modality, to sequentially generate an action trajectory of the robot at each time step; generating the action trajectory sequence of the robot according to the action trajectory of the robot at each time step.

3. The robot action generation method of claim 2, wherein, The step of iteratively exploring the initial noise trajectory in a reverse order of time according to the multi-modal observation data, the modal weights of each modality and the noise estimation model corresponding to each modality, to sequentially generate an action trajectory of the robot at each time step, comprises the following steps: acquiring an action trajectory at a current time step, wherein if the current time step is the first time step, the initial noise trajectory is taken as the action trajectory at the current time step, otherwise, the action trajectory at the current time step is determined according to an action trajectory at a next time step of the current time step; determining noise information of each modality according to the multi-modal observation data, the noise estimation model corresponding to each modality and the action trajectory at the current time step; determining total noise at the current time step according to the modal weights of each modality and the noise information of each modality; generating an action trajectory of the robot at a previous time step of the current time step according to the action trajectory at the current time step and the total noise at the current time step.

4. The robot action generation method of claim 3, wherein, The step of determining noise information of each modality according to the multi-modal observation data, the noise estimation model corresponding to each modality and the action trajectory at the current time step, comprises the following steps: inputting observation data of a first modality and the action trajectory at the current time step into a noise estimation model corresponding to the first modality, to predict noise information of the first modality, wherein the first modality is any modality in the multiple modalities.

5. The robot action generation method of claim 3, wherein, The step of determining total noise at the current time step according to the modal weights of each modality and the noise information of each modality, comprises the following steps: calculating a product of the model weight of each modality and the corresponding noise information, to obtain a product result of each modality; summing the product results of the modalities, to obtain the total noise at the current time step.

6. The robot action generation method of claim 3, wherein, The generating, according to the action trajectory of the current time step and the total noise at the current time step, of the action trajectory of the robot at a previous time step of the current time step comprises: obtaining a gradient step length coefficient, a denoising intensity coefficient and additional Gaussian noise; calculating a first product of the gradient step length coefficient and the total noise at the current time step; calculating a difference between the action trajectory of the current time step and the first product; calculating a second product of the difference and the denoising intensity coefficient; calculating a sum of the second product and the additional Gaussian noise as the action trajectory of the robot at the previous time step of the current time step.

7. The robot action generation method of claim 1, wherein, The determination process of the initial noise trajectory comprises: obtaining a trajectory dimension of the action trajectory sequence; generating an initial noise trajectory by independently sampling elements from a standard Gaussian distribution according to the trajectory dimension.

8. The robot action generation method of claim 1, wherein, The determination process of the modal weight of each modality comprises: obtaining a current task success rate of each modality and a current confidence degree of each modality; calculating the modal weight of each modality according to the current task success rate of each modality and the current confidence degree of each modality. 9.The robot action generation method of claim 1, wherein, The training process of the noise estimation model corresponding to each modality comprises: obtaining a first training data set under a first modality, the first training data set comprising a real action sequence and observation data under the first modality; according to a preset noise scheduling coefficient, iteratively processing the real action sequence at each time step to sequentially generate an action trajectory at each time step; training the noise estimation model corresponding to the first modality according to the action trajectory at each time step.

10. A control system for a robot, characterized in that comprise: a processor and a memory, the memory storing machine readable instructions executable by the processor, when the control system of the robot is running, the processor executes the machine readable instructions to perform the steps of the robot action generation method according to any one of claims 1 to 9.

Images