A mechanical arm action sequence generation method and device, a model training method, and an electronic device

Abstract

The application provides a mechanical arm action sequence generation method and device, a model training method and an electronic device, and belongs to the technical field of robots. The method comprises the following steps: acquiring multi-modal perception data of a mechanical arm operation environment; extracting spatial geometric features of an operation object based on an RGB-D image, respectively; extracting body state features of the mechanical arm based on joint angle data; extracting interactive force features of an end effector based on six-dimensional force sensor data; performing fusion processing on the spatial geometric features, the body state features and the interactive force features to obtain a joint representation vector; inputting the joint representation vector as a condition to a pre-trained conditional diffusion model together with an initial noise-bearing action sequence randomly generated; and gradually inferring a final mechanical arm action sequence. The application improves the generalization capability of the mechanical arm in a complex scene by combining multi-modal perception data and a conditional diffusion model.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of robots, in particular to a robot arm action sequence generation method and device, a model training method and an electronic device. BACKGROUND

[0002] With the rapid development of artificial intelligence and robotics, robot arms are increasingly widely used in industrial assembly, logistics sorting, medical assistance and home service, etc. In a complex and dynamic real environment, it is very important to automatically generate safe, feasible, efficient and robust action sequences for robot arms.

[0003] Traditional action generation methods mainly include three categories: teaching and reproduction, planning algorithms and learning strategies.

[0004] Teaching and reproduction (such as behavior cloning) records human operation trajectories and directly reproduces them, which has a distribution offset problem, resulting in poor generalization ability in new scenarios, and difficulty in handling force control tasks. Although inverse reinforcement learning can infer the reward function from expert demonstrations, the training process is unstable, and the support for multi-modal input is limited, usually only supporting single visual or joint state input. Reinforcement learning methods have difficulty effectively processing high-dimensional visual input (such as RGB-D images), limiting their application in complex scenarios. SUMMARY

[0005] Therefore, the purpose of the present application is to provide a robot arm action sequence generation method, device, model training method and electronic device to improve the generalization ability of robot arms in complex scenarios.

[0006] In a first aspect, a robot arm action sequence generation method is provided, comprising:

[0007] Obtaining multi-modal perception data of a robot arm operating environment, the multi-modal perception data comprising: an RGB-D image of an operating object, current joint angle data of the robot arm, and six-dimensional force sensor data at an end effector;

[0008] Respectively extracting spatial geometric features of the operating object based on the RGB-D image, body state features of the robot arm based on the joint angle data, and interaction force features of the end effector based on the six-dimensional force sensor data;

[0009] Fusing the spatial geometric features, the body state features and the interaction force features to obtain a joint representation vector;

[0010] Inputting the joint representation vector as a condition to a pre-trained conditional diffusion model together with an initially randomly generated noisy action sequence; so as to execute a multi-step reverse denoising process through the conditional diffusion model to gradually infer a final robot arm action sequence.

[0011] Optionally, the spatial geometric features of the object to be extracted based on the RGB-D image include:

[0012] Convert the RGB-D images of the object being operated on, acquired at multiple consecutive time points, into a 3D point cloud;

[0013] The 3D point cloud data collected at each time point is downsampled and preprocessed to obtain a preprocessed temporal point cloud sequence;

[0014] Based on the PointNet++ network, hierarchical geometric feature extraction is performed on the preprocessed temporal point cloud sequence to obtain the spatial geometric features of the object being operated on.

[0015] Optionally, the 3D point cloud data acquired at each time step is downsampled and preprocessed to obtain a preprocessed temporal point cloud sequence, including:

[0016] The space containing the 3D point cloud acquired at each time moment is divided into a grid to obtain an initial voxel grid; the initial voxel grid consists of several voxel blocks.

[0017] The curvature of each voxel block is determined based on the features of the point cloud within each voxel block.

[0018] The size of each voxel is dynamically adjusted based on the curvature of each voxel to obtain the adjusted voxel mesh.

[0019] The point cloud data within the adjusted voxel grid is downsampled, and the centroids of each voxel block are extracted to form the 3D point cloud at the corresponding time. The sampled 3D point clouds at each time are then combined to form a temporal point cloud sequence.

[0020] Optionally, the spatial geometric features, ontological state features, and interaction force features are fused to obtain a joint representation vector, including:

[0021] Spatiotemporal synchronization processing is performed on spatial geometric features, ontological state features, and interactive force features;

[0022] Based on the multi-head cross-attention algorithm, the spatial geometric features, ontological state features and interactive force features after spatiotemporal synchronization are fused to obtain a spatiotemporally aligned joint representation vector.

[0023] Optionally, a multi-step reverse denoising process is performed using a conditional diffusion model to progressively infer the final robotic arm motion sequence, including:

[0024] For the initial noisy action sequence, denoising iterations are performed step by step according to a preset number of diffusion steps. The denoising process at each step of the iteration is as follows:

[0025] In each denoising step, the noise of the noisy action sequence input in the current step is predicted;

[0026] The noisy action sequence of the current step is denoised using the predicted noise to obtain the denoised action sequence of the current step.

[0027] After each denoising step, the denoised action sequence is subjected to dynamic constraint projection, and the trajectory formed by the denoised action sequence after dynamic constraint projection is smoothed and fine-tuned.

[0028] The motion sequence after trajectory smoothing and fine-tuning is used as the input for the next step. The above denoising process is repeated until all reverse denoising steps are completed, and the final robotic arm motion sequence is output.

[0029] Optionally, performing dynamically constrained projection on the denoised action sequence includes:

[0030] Based on the preset dynamic constraint range, the parameters in the denoised motion sequence are mapped to the dynamic constraint range, which includes the joint angle limit range, speed limit range, acceleration limit range, and torque limit range of the end effector of the robotic arm; the parameters in the denoised motion sequence include the joint position, speed, acceleration of the robotic arm, and torque of the end effector.

[0031] Secondly, a training method for a robotic arm motion sequence generation model is provided. The robotic arm motion sequence generation model adopts a conditional diffusion model, and the method includes:

[0032] Acquire historical action sequence samples, which include real action sequences and their corresponding multimodal perception data;

[0033] A forward diffusion process is performed on the real action sequence, and progressively increasing Gaussian noise is added to generate a noisy action sequence.

[0034] Using the joint representation vector of multimodal sensing data as a condition, reverse inference is performed on noisy action sequences to predict the noise in the noisy action sequences;

[0035] Based on the loss values ​​of predicted noise and added real noise, optimize the parameters of the conditional diffusion model;

[0036] Repeat the above process until the conditional diffusion model reaches the preset iteration termination condition, and the trained conditional diffusion model is obtained.

[0037] Optionally, the method further includes:

[0038] In the reverse reasoning process, a dynamic constraint loss function is introduced. The dynamic constraint loss includes joint angle limiting loss, end contact force safety loss and zero moment point stability loss.

[0039] The total training loss of the conditional diffusion model is determined by a weighted sum of noise loss and dynamic constraint loss, and the parameters of the conditional diffusion model are optimized based on the total training loss.

[0040] Thirdly, a robotic arm motion sequence generation device is provided, the device comprising:

[0041] The acquisition unit is used to acquire multimodal perception data of the robotic arm's operating environment. The multimodal perception data includes: RGB-D images of the object being operated on, current joint angle data of the robotic arm, and six-dimensional force sensor data at the end effector.

[0042] The extraction unit is used to extract the spatial geometric features of the object being manipulated based on RGB-D images, extract the body state features of the robotic arm based on joint angle data, and extract the interaction force features of the end effector based on six-dimensional force sensor data.

[0043] The fusion unit is used to fuse spatial geometric features, ontological state features, and interaction force features to obtain a joint representation vector;

[0044] The generation unit is used to input the joint representation vector as a conditional input, along with a randomly generated initial noisy action sequence, into a pre-trained conditional diffusion model; so that the conditional diffusion model can perform a multi-step reverse denoising process to gradually infer the final robotic arm action sequence.

[0045] Fourthly, an electronic device is provided, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0046] Memory, used to store computer programs;

[0047] A processor, when executing a program stored in memory, implements either the first aspect or the second aspect of the method.

[0048] This invention provides a method, apparatus, model training method, and electronic device for generating robotic arm motion sequences. It acquires multimodal perception data of the robotic arm's operating environment, extracts spatial geometric features of the manipulated object based on RGB-D images, extracts the robotic arm's body state features based on joint angle data, and extracts the interaction force features of the end effector based on six-dimensional force sensor data. The spatial geometric features, body state features, and interaction force features are fused to obtain a joint representation vector. This joint representation vector is used as a conditional input, along with a randomly generated initial noisy motion sequence, and input into a pre-trained conditional diffusion model. The conditional diffusion model then performs a multi-step reverse denoising process to progressively infer the final robotic arm motion sequence. By combining multimodal perception data and the conditional diffusion model, this invention uses the joint representation vector, representing global features, as a diffusion condition at each step for global correction, avoiding error accumulation and making the generated robotic arm motion sequence more accurate and reliable. This improves the generalization ability of the robotic arm in complex scenarios.

[0049] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0050] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0051] Figure 1 A flowchart of a robotic arm motion sequence generation method provided by an embodiment of the present invention is shown;

[0052] Figure 2 A flowchart of a training method for a robotic arm motion sequence generation model provided in an embodiment of the present invention is shown;

[0053] Figure 3 This diagram illustrates the structure of a robotic arm motion sequence generation device provided in an embodiment of the present invention.

[0054] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present invention is shown. Detailed Implementation

[0055] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0056] Traditional action generation methods mainly fall into three categories: demonstration and reproduction, planning algorithms, and learning strategies. Demonstration and reproduction (such as behavior cloning) directly reproduce human action trajectories by recording them, but this suffers from distribution bias, resulting in poor generalization ability in new scenarios and difficulty in handling force control tasks. While inverse reinforcement learning can deduce reward functions from expert demonstrations, the training process is unstable, and its support for multimodal inputs is limited, typically only supporting single visual or joint state inputs. Reinforcement learning methods struggle to effectively handle high-dimensional visual inputs (such as RGB-D images), limiting their application in complex scenes.

[0057] Based on this, embodiments of the present invention provide a method and apparatus for generating robotic arm motion sequences, which are described below through embodiments.

[0058] This invention provides a method for generating robotic arm motion sequences, applicable to scenarios such as service robots and industrial grasping systems that require performing complex tasks in unstructured environments. This method integrates visual, proprioceptive, and force information, utilizing a conditional diffusion model to generate safe, accurate, and physically constrained robotic arm motion sequences. Figure 1 As shown, it includes the following steps:

[0059] Step S101: Acquire multimodal perception data of the robotic arm's operating environment.

[0060] The multimodal perception data includes: RGB-D images of the manipulated object, current joint angle data of the robotic arm, and six-dimensional force sensor data at the end effector.

[0061] RGB-D images refer to image data that simultaneously contains color information (Red, Green, Blue) and depth information. In one example, an RGB-D camera, such as the Intel RealSense D455, Microsoft Kinect v2, or Orbbec Astra, can be used for acquisition. The RGB-D camera is mounted on a fixed bracket near the end effector of the robotic arm or above the worktable to ensure complete coverage of the operating area. Preferably, it is mounted on the robotic arm, allowing the camera to move with the arm and thus achieve dynamic perspective perception.

[0062] The current joint angle data of the robotic arm refers to the real-time angle or displacement values ​​of each rotational or translational joint. In one example, this can be acquired using a high-precision encoder (such as a photoelectric encoder or magnetic encoder) built into the robotic arm. Each joint integrates an encoder to directly measure the joint angle. The data is transmitted to the main control unit via EtherCAT or CAN bus, with a sampling frequency of 100~500 Hz.

[0063] Six-dimensional force sensor data at the end effector refers to the three-dimensional force and three-dimensional torque acting on the end effector, used to sense its contact state with the environment. A six-dimensional force / torque sensor (such as the ATI Mini45 or RobotiqForce / Torque Sensor) can be used. In one example, this six-dimensional force / torque sensor can be installed between the end flange of the robotic arm and the end effector, enabling precise measurement of the interaction forces between the end effector and the object.

[0064] Step S102: Extract the spatial geometric features of the object to be operated based on the RGB-D image, extract the body state features of the robotic arm based on the joint angle data, and extract the interaction force features of the end effector based on the six-dimensional force sensor data.

[0065] After obtaining the multimodal data, the data features of each modality are extracted.

[0066] In one feasible implementation, after reading the joint angle data, a trained TCN-BiLSTM network can be used to extract temporal features, thereby obtaining the ontology state features. .

[0067] In another implementation, after reading the six-dimensional force sensor data, zero-mean normalization can be performed to remove static interference from the sensor and the environment. Then, the preprocessed data is converted into a six-dimensional force vector, and the six-dimensional force vectors from multiple consecutive time steps are used as an input sequence and input into a time-series modeling model composed of a one-dimensional convolutional network and a bidirectional long short-term memory network to obtain the force constraint embedding vector. ,in, Indicates time step The six-dimensional force vector, This represents the time step count. The force constraint is then embedded into the vector. As a feature of interaction force.

[0068] Regarding spatial geometric features The extraction of [data] will be described in the following examples and will not be repeated here.

[0069] Step S103: The spatial geometric features, ontological state features, and interaction force features are fused to obtain a joint representation vector.

[0070] Step S104: The joint representation vector is used as a conditional input and input together with the randomly generated initial noisy action sequence into the pre-trained conditional diffusion model; so that the conditional diffusion model can perform a multi-step reverse denoising process to gradually infer the final robotic arm action sequence.

[0071] In this step, the structure of the conditional diffusion model is explained. First, the backbone of the conditional diffusion model is a U-Net architecture with an encoder-decoder structure and skip connections. The diffusion steps K are encoded as position vectors and used as one of the inputs to the model.

[0072] This invention, by fusing multimodal data, constructs a three-dimensional perception system integrating the three-dimensional geometry, ontological state, and interactive forces of the operating environment. This system can perceive not only object geometry and the robotic arm configuration but also the contact state with the object. This provides richer and more accurate data support for the subsequent generation of the robotic arm's motion sequences. Furthermore, it employs a conditional diffusion model to generate the motion sequences. Compared to traditional regression-based methods, where each step depends on the previous step, leading to error accumulation, this invention uses a conditional diffusion model to denoise all actions in the sequence simultaneously, without relying on the previous action. Each step uses a joint representation vector representing global features as the diffusion condition; that is, each denoising step is a global correction of the current noisy sequence, avoiding error accumulation and making the generated robotic arm motion sequences more accurate and reliable.

[0073] Based on the above embodiments, the spatial geometric features of the object to be extracted from the RGB-D image include:

[0074] Step S102A: Convert the RGB-D images of the object being operated on, acquired at multiple consecutive time points, into a three-dimensional point cloud.

[0075] In this step, images containing the object being operated on can be continuously acquired at a certain frame rate.

[0076] For each frame of RGB-D image, through the camera's intrinsic parameter matrix:

[0077] ;

[0078] Using the two-dimensional coordinates (u,v) and depth value D(u,v) of each pixel, each pixel can be mapped from the image plane to a normalized coordinate system in the camera coordinate system: ,in and For the camera's focal length, The pixel coordinates of the primary point. Using depth values ​​from the depth map.

[0079] To reconstruct the three-dimensional spatial coordinates of each pixel:

[0080]

[0081] Obtain the three-dimensional spatial coordinates of the point. .

[0082] Step S102B: Perform downsampling preprocessing on the 3D point cloud data collected at each time step to obtain a preprocessed temporal point cloud sequence.

[0083] For each frame of 3D point cloud, downsampling is performed to reduce data redundancy and improve subsequent processing efficiency.

[0084] While downsampling reduces data redundancy, it may also lead to the loss of some key feature points. Therefore, this embodiment further optimizes the point cloud downsampling preprocessing step and proposes a dynamic voxelization method based on local curvature adaptive adjustment of voxel size to retain key geometric details while reducing data redundancy.

[0085] Specifically, the steps include the following:

[0086] Step S102B1: Divide the space of the 3D point cloud acquired at each time moment into a grid to obtain the initial voxel grid.

[0087] The initial voxel grid consists of several voxel blocks.

[0088] In this step, a rapid memorization mesh is first created based on the same partitioning size. All the partitioned voxel blocks collectively cover the area of ​​the point cloud, forming a spatial index structure that facilitates subsequent rapid querying and processing.

[0089] Step S102B2: Determine the curvature of each voxel block based on the features of the point cloud within each voxel block.

[0090] In this step, the voxel blocks being targeted are non-empty voxel blocks, meaning that each voxel block must contain at least one point.

[0091] In one feasible implementation, the steps for calculating curvature include:

[0092] For each non-empty voxel block, extract the set of coordinates of all points inside it. Calculate the covariance matrix of this point set. ;

[0093] in The center of mass of the point within the voxel; For the first The coordinates of the points; The number of three-dimensional points in the point set.

[0094] Eigenvalue decomposition of the covariance matrix yields three eigenvalues. ;

[0095] in, Represents the variance in the direction of minimum variance; Indicates the variance in the intermediate direction; This represents the variance in the direction of maximum variance.

[0096] The local curvature of this voxel block is defined as:

[0097] ;

[0098] Ci≈0: This indicates that the point distribution is close to a plane (a flat area, such as a desktop).

[0099] Ci≈1: indicates that the distribution of points is highly irregular (high curvature areas, such as edges, corners, handles).

[0100] Step S102B3: Dynamically adjust the size of each voxel based on the curvature of each voxel to obtain the adjusted voxel mesh.

[0101] In this step, the strategy is adjusted, for example, by reducing the size of voxels with high curvature and increasing the size of voxels with low curvature.

[0102] Step S102B4: Downsample the point cloud data in the adjusted voxel grid, extract the centroid points in each voxel block to form the three-dimensional point cloud at the corresponding time, and combine the sampled three-dimensional point clouds at each time to form a temporal point cloud sequence.

[0103] In this step, for voxel blocks with high curvature, the number of samples is reduced to retain as much point cloud data as possible to preserve local details. For voxel blocks with low curvature, sampling is significantly reduced to decrease data redundancy.

[0104] The embodiments of the present invention significantly reduce the amount of data while preserving key geometric features, effectively balancing the accuracy and efficiency of subsequent feature extraction.

[0105] Step S102C: Based on the PointNet++ network, perform hierarchical geometric feature extraction on the preprocessed temporal point cloud sequence to obtain the spatial geometric features of the object being operated on.

[0106] PointNet++ is a deep learning model for processing point cloud data. It is an improvement and extension of the original PointNet architecture. Its principle lies in extracting multi-scale features from point clouds in a layered manner, such as the shape, boundary, and structural information of the objects being manipulated.

[0107] By using PointNet++, multi-level and localized feature extraction of point cloud data is achieved, improving the accuracy and robustness of spatial geometric representation.

[0108] Existing technologies often use simple splicing or weighted fusion methods to process multimodal data, without considering the inconsistencies of different modalities in time and space, which leads to distortion of fused features and affects the accuracy of subsequent action generation.

[0109] Therefore, based on the above embodiments, the spatial geometric features, ontological state features, and interaction force features are fused to obtain a joint representation vector including:

[0110] Step S103A: Perform spatiotemporal synchronization processing on spatial geometric features, ontological state features, and interactive force features.

[0111] In this step, the time is first synchronized. When the three modal data are acquired by the sensor, their respective timestamps are generated simultaneously. Using a time reference, the three timestamps are time-corrected and synchronized.

[0112] After time synchronization, spatial coordinates are aligned, and all features are uniformly converted to the robot arm's base coordinate system.

[0113] Step S103B: Based on the multi-head cross-attention algorithm, the spatial geometric features, ontology state features and interaction force features after spatiotemporal synchronization are fused to obtain a spatiotemporally aligned joint representation vector.

[0114] In this step, spatial geometric features are... Ontological state characteristics Force constraint embedding vector These are mapped to query, key, and value matrices, respectively:

[0115]

[0116] Then, the input is fed into the cross-attention fusion module to calculate the attention weights, construct cross-modal correlations, and obtain the fused joint representation vector. :

[0117] .

[0118] This invention employs a multi-head cross-attention mechanism for fusion, allowing one modality to actively "query" information from other modalities. For example, spatial geometric features can focus on which joint angles are related to the current object's position, or whether force perception has reached a point of contact. This achieves cross-modal semantic alignment, enhancing the discriminative power of the fused features. It accurately reflects the collaborative relationship between vision, motion, and force perception.

[0119] Based on the above embodiments, a multi-step reverse denoising process is performed using a conditional diffusion model to gradually infer the final robotic arm motion sequence, including the following steps:

[0120] For the initial noisy action sequence, denoising iterations are performed step by step according to a preset number of diffusion steps. The denoising process at each step of the iteration is as follows:

[0121] Step A: In each denoising step, predict the noise in the noisy action sequence input to the current step.

[0122] In this step, the current noisy action sequence is... Joint representation vector and current diffusion step index The input is fed into a pre-trained conditional diffusion U-Net structure to obtain the predicted noise:

[0123] ;

[0124] In the first step, the noisy action sequence is a randomly generated initial noisy action sequence, and the noisy action sequence input in each subsequent step is the action sequence after the previous step of denoising.

[0125] Step B: Use the predicted noise to denoise the noisy action sequence of the current step to obtain the denoised action sequence of the current step.

[0126] In this step, the denoised action sequence is calculated based on the inverse denoising formula of the conditional diffusion model. The inverse denoising formula is as follows:

[0127] ;

[0128] in, and For positive noise scheduling; Indicates the noise in the prediction; Indicates the first Candidate action sequences for denoising at each time step.

[0129] Step C: After each denoising step, perform dynamic constraint projection on the denoised action sequence, and perform trajectory smoothing and fine-tuning on the trajectory formed by the denoised action sequence after dynamic constraint projection.

[0130] In one feasible implementation, performing dynamically constrained projection on the denoised action sequence includes:

[0131] Based on the preset dynamic constraint range, the parameters in the denoised motion sequence are mapped to the dynamic constraint range, which includes the joint angle limit range, speed limit range, acceleration limit range, and torque limit range of the end effector of the robotic arm; the parameters in the denoised motion sequence include the joint position, speed, acceleration of the robotic arm, and torque of the end effector.

[0132] In one example, dynamic projection can be performed using the projection formula:

[0133]

[0134] in, Indicates the motion parameters after projection; These are the motion parameters in the denoised motion sequence for each step; the motion parameters are one of the following: joint position, velocity, acceleration, and torque of the end effector. The range of limits for the corresponding action parameters; feasible region It is a feasible set of limits for the joint angle, speed, acceleration, and torque of the end effector of the robotic arm.

[0135] In another example, a lightweight smoothing algorithm can be used to smooth the action vectors in the action sequence. This algorithm could be, for example, exponential moving average, low-pass filtering, or spline interpolation.

[0136] In one specific implementation, based on the Jacobi kinematic equations:

[0137] ;

[0138] The trajectory of the projected motion sequence is fine-tuned. By calculating the pose error of the end effector and combining the limiting constraints of joint acceleration and joint torque, the trajectory adjustment is minimized, so that the motion sequence can reduce energy consumption and impact while maintaining accessibility, and the linear velocity and angular velocity of the robot arm end effector are obtained after fine-tuning.

[0139] in, The linear and angular velocities of the end effector in Cartesian space. This is the joint velocity vector.

[0140] If the directly generated motion sequence may violate the physical constraints of the robotic arm, such as joint over-limit, excessive speed leading to motor overload, or excessive contact force damaging the object, it will affect the feasibility and safety of the motion. Therefore, after each denoising step, a dynamic constraint projection is introduced to correct unreasonable configurations in real time and map unreasonable motions into the feasible region, ensuring that each candidate motion is within the safe range and avoiding the final output being unexecutable.

[0141] In addition, since high-frequency noise may be introduced during the noise reduction process, resulting in an uneven trajectory, vibration, or unstable control, this embodiment of the invention also introduces trajectory smoothing fine-tuning to eliminate high-frequency jitter, making the movement of the robotic arm more stable, reducing noise, and extending its lifespan.

[0142] Step D: Use the motion sequence after trajectory smoothing and fine-tuning as the input for the next step, repeat the above denoising process until all reverse denoising steps are completed, and output the final robotic arm motion sequence.

[0143] In another embodiment, a training method for a robotic arm motion sequence generation model is provided. The robotic arm motion sequence generation model employs a conditional diffusion model, such as... Figure 2 As shown, the method includes the following steps:

[0144] Step S201: Obtain historical action sequence samples, which include real action sequences and their corresponding multimodal perception data.

[0145] In this step, a large amount of historical data is collected on the robotic arm performing various operational tasks (such as grasping, placing, assembling, pushing and pulling) in real or simulated environments.

[0146] The multimodal sensing data also includes RGB-D images of the manipulated object, current joint angle data of the robotic arm, and six-dimensional force sensor data at the end effector. The multimodal sensing data is fused to obtain a joint representation vector; the fusion process is the same as in the above embodiments and will not be repeated here.

[0147] Step S202: Perform a forward diffusion process on the real action sequence, adding progressively increasing Gaussian noise to generate a noisy action sequence.

[0148] For the initial real action sequence Gaussian noise is added step by step over time to obtain a series of perturbed noisy action sequences. The forward diffusion formula is as follows:

[0149]

[0150] in, For the first Action vectors in the action sequence of a step; For the first Noise added step by step.

[0151] Step S203: Using the joint representation vector of the multimodal sensing data as a condition, perform reverse inference on the noisy action sequence to predict the noise of the noisy action sequence.

[0152] In this step, the noisy action sequence is... Real noise and joint representation vector The input to the decoding part of the U-Net architecture is used for noise prediction, and the final output is the result of adding noise. Predicted value .

[0153] Step S204: Optimize the parameters of the conditional diffusion model based on the loss values ​​of the predicted noise and the added real noise.

[0154] In this step, the noise loss is expressed as follows:

[0155] ;

[0156] in, This represents the Gaussian noise injected during sampling; This represents the noise in the prediction.

[0157] Step S205: Repeat the above process until the conditional diffusion model reaches the preset iteration termination condition, and obtain the trained conditional diffusion model.

[0158] The preset iteration termination condition is, for example, reaching the required number of iterations or satisfying the convergence condition.

[0159] Based on the above embodiments, the method further includes:

[0160] Step S206: Introduce a dynamic constraint loss function during the reverse reasoning process. The dynamic constraint loss includes joint angle limiting loss, end contact force safety loss, and zero moment point stability loss.

[0161] In each forward propagation round, after the model predicts the noise and recovers the initial denoising action, instead of directly calculating the noise loss, it further evaluates the physical feasibility of the denoising results and constructs three dynamic constraint losses:

[0162] Joint angle limiting loss:

[0163] ;

[0164] in, This represents the joint angle in frame t. This indicates the maximum permissible angle of the joint.

[0165] End contact force safety loss:

[0166] ;

[0167] in, This represents the end contact force of the t-th frame; Indicates the safety threshold for end force;

[0168] Zero-moment point stability loss:

[0169]

[0170] in, Represents the coordinates of the zero torque point; Indicates a supporting polygon; This represents the shortest distance from a point to the boundary of the supporting polygon.

[0171] Step S207: Determine the total training loss of the conditional diffusion model based on the weighted sum of noise loss and dynamic constraint loss, and optimize the parameters of the conditional diffusion model based on the total training loss.

[0172] In this step, the total loss is:

[0173]

[0174] The meanings of each parameter are the same as in the above formula, and will not be repeated here.

[0175] This embodiment further proposes a technique for introducing a physical dynamics constraint loss function in the reverse reasoning process. By incorporating the physical dynamics of the robotic arm as a regularization term into the training objective, the actions generated by the model not only conform to the data distribution but also naturally meet the requirements of safety and executability.

[0176] Based on the same inventive concept, a robotic arm motion sequence generation device is provided, such as... Figure 3 As shown, the device includes:

[0177] The acquisition unit 301 is used to acquire multimodal perception data of the robotic arm's operating environment. The multimodal perception data includes: RGB-D image of the object being operated on, current joint angle data of the robotic arm, and six-dimensional force sensor data at the end effector.

[0178] The extraction unit 302 is used to extract the spatial geometric features of the operation object based on RGB-D images, extract the body state features of the robotic arm based on joint angle data, and extract the interaction force features of the end effector based on six-dimensional force sensor data.

[0179] The fusion unit 303 is used to fuse spatial geometric features, ontological state features and interaction force features to obtain a joint representation vector;

[0180] The generation unit 304 is used to input the joint representation vector as a conditional input, together with the randomly generated initial noisy action sequence, into the pre-trained conditional diffusion model; so that the conditional diffusion model can perform a multi-step reverse denoising process to gradually infer the final robotic arm action sequence.

[0181] Based on the same technical concept, embodiments of the present invention also provide an electronic device, such as... Figure 4 As shown, it includes a processor 401, a communication interface 402, a memory 403, and a communication bus 404, wherein the processor 401, the communication interface 402, and the memory 403 communicate with each other through the communication bus 404.

[0182] Memory 403 is used to store computer programs;

[0183] The processor 401, when executing the program stored in the memory 403, implements the steps of the method for generating robotic arm motion sequences and the training method for the robotic arm motion sequence generation model.

[0184] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0185] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0186] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0187] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0188] The device for generating robotic arm motion sequences provided in this embodiment of the invention can be specific hardware on the device or software or firmware installed on the device. The implementation principle and technical effects of the device provided in this embodiment of the invention are the same as those in the foregoing method embodiments. For the sake of brevity, any parts not mentioned in the device embodiments can be referred to the corresponding content in the foregoing method embodiments. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can all be referred to the corresponding processes in the above method embodiments, and will not be repeated here.

[0189] In the embodiments provided by this invention, it should be understood that the disclosed apparatus and method can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0190] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0191] In addition, the functional units in the embodiments provided by the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0192] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, essentially, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0193] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. In addition, the terms "first", "second", "third", etc. are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0194] Finally, it should be noted that the above-described embodiments are merely specific implementations of the present invention, used to illustrate the technical solutions of the present invention, and not to limit it. The scope of protection of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments within the technical scope disclosed in the present invention, or make equivalent substitutions for some of the technical features; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention. All should be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for generating a robotic arm motion sequence, characterized in that, include: Acquire multimodal perception data of the robotic arm's operating environment. The multimodal perception data includes: RGB-D images of the object being operated on, current joint angle data of the robotic arm, and six-dimensional force sensor data at the end effector. The spatial geometric features of the manipulated object are extracted based on the RGB-D image, the body state features of the robotic arm are extracted based on the joint angle data, and the interaction force features of the end effector are extracted based on the six-dimensional force sensor data. The spatial geometric features, ontological state features, and interactive force features are fused to obtain a joint representation vector; The joint representation vector is used as a conditional input, and together with the randomly generated initial noisy action sequence, it is input into a pre-trained conditional diffusion model; so that the conditional diffusion model can perform a multi-step reverse denoising process to gradually infer the final robotic arm action sequence.

2. The method according to claim 1, characterized in that, The step of extracting the spatial geometric features of the object based on the RGB-D image includes: Convert the RGB-D images of the object being operated on, acquired at multiple consecutive time points, into a 3D point cloud; The 3D point cloud data collected at each time point is downsampled and preprocessed to obtain a preprocessed temporal point cloud sequence; Based on the PointNet++ network, hierarchical geometric feature extraction is performed on the preprocessed temporal point cloud sequence to obtain the spatial geometric features of the operation object.

3. The method according to claim 2, characterized in that, The downsampling preprocessing of the 3D point cloud data acquired at each time moment to obtain the preprocessed temporal point cloud sequence includes: The space containing the 3D point cloud acquired at each time moment is divided into a grid to obtain an initial voxel grid; the initial voxel grid is composed of several voxel blocks. The curvature of each voxel block is determined based on the features of the point cloud within each voxel block; The size of each voxel block is dynamically adjusted based on the curvature of each voxel block to obtain the adjusted voxel mesh. The point cloud data within the adjusted voxel grid is downsampled, and the centroids of each voxel block are extracted to form a three-dimensional point cloud at the corresponding time. The sampled three-dimensional point clouds at each time are then combined to form a temporal point cloud sequence.

4. The method according to claim 1, characterized in that, The spatial geometric features, ontological state features, and interaction force features are fused to obtain a joint representation vector, which includes: The spatial geometric features, ontological state features, and interactive force features are subjected to spatiotemporal synchronization processing. Based on the multi-head cross-attention algorithm, the spatial geometric features, ontological state features, and interaction force features after spatiotemporal synchronization are fused to obtain a spatiotemporally aligned joint representation vector.

5. The method according to claim 1, characterized in that, A multi-step inverse denoising process is performed using a conditional diffusion model to progressively infer the final robotic arm motion sequence, including: For the initial noisy action sequence, a denoising iteration is performed step by step according to a preset number of diffusion steps. The denoising process at each step of the iteration is as follows: In each denoising step, the noise of the noisy action sequence input in the current step is predicted; The noisy action sequence of the current step is denoised using the predicted noise to obtain the denoised action sequence of the current step. After each denoising step, the denoising action sequence is subjected to dynamic constraint projection, and the trajectory formed by the denoising action sequence after dynamic constraint projection is subjected to trajectory smoothing and fine adjustment. The motion sequence after trajectory smoothing and fine-tuning is used as the input for the next step. The above denoising process is repeated until all reverse denoising steps are completed, and the final robotic arm motion sequence is output.

6. The method according to claim 5, characterized in that, Performing dynamically constrained projection on the denoised action sequence includes: Based on a preset dynamic constraint range, the parameters in the denoised action sequence are mapped to the dynamic constraint range, wherein the dynamic constraint range includes the joint angle limit range, speed limit range, acceleration limit range, and torque limit range of the end effector of the robotic arm; the parameters in the denoised action sequence include the joint position, speed, acceleration of the robotic arm, and torque of the end effector.

7. A training method for a robotic arm motion sequence generation model, characterized in that, The robotic arm motion sequence generation model employs a conditional diffusion model, and the method includes: Acquire historical motion sequence samples, which include real motion sequences and their corresponding multimodal perception data; the multimodal perception data includes: RGB-D images of the manipulated object, current joint angle data of the robotic arm, and six-dimensional force sensor data at the end effector; A forward diffusion process is performed on the real action sequence, and progressively increasing Gaussian noise is added to generate a noisy action sequence. Using the joint representation vector of the multimodal sensing data as a condition, reverse inference is performed on the noisy action sequence to predict the noise of the noisy action sequence; Based on the loss values ​​of the predicted noise and the added real noise, the parameters of the conditional diffusion model are optimized. Repeat the above process until the conditional diffusion model reaches the preset iteration termination condition, and the trained conditional diffusion model is obtained.

8. The method according to claim 7, characterized in that, The method further includes: In the reverse reasoning process, a dynamic constraint loss function is introduced, which includes joint angle limiting loss, end contact force safety loss and zero moment point stability loss. The total training loss of the conditional diffusion model is determined based on the weighted sum of the noise loss and the dynamic constraint loss, and the parameters of the conditional diffusion model are optimized based on the total training loss.

9. A robotic arm motion sequence generation device, characterized in that, The device includes: The acquisition unit is used to acquire multimodal perception data of the robotic arm's operating environment. The multimodal perception data includes: an RGB-D image of the object being operated on, the current joint angle data of the robotic arm, and six-dimensional force sensor data at the end effector. The extraction unit is used to extract the spatial geometric features of the operation object based on the RGB-D image, extract the body state features of the robotic arm based on the joint angle data, and extract the interaction force features of the end effector based on the six-dimensional force sensor data. The fusion unit is used to fuse the spatial geometric features, ontological state features, and interaction force features to obtain a joint representation vector; The generation unit is used to input the joint representation vector as a conditional input, together with the randomly generated initial noisy action sequence, into a pre-trained conditional diffusion model; so that the conditional diffusion model can perform a multi-step reverse denoising process to gradually infer the final robotic arm action sequence.

10. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; The memory is used to store computer programs; When the processor executes the program stored in the memory, it implements the method described in any one of claims 1-8.

Images