Robot control method, device and system based on diffusion model and medium

Abstract

The invention relates to the technical field of data processing, in particular to a robot control method, device and system based on a diffusion model and a medium, and the method comprises the steps: collecting the position information of the tail end of an index finger and the state information of a flexible linear object in a human hand teaching process, and constructing a training sample; training the training sample by using a diffusion model, and learning mapping from a high noise state to a clear target state through a noise prediction network in the training diffusion model to obtain a trained diffusion model; the training process of the diffusion model comprises forward noise addition and reverse noise reduction; the state of the flexible linear object and the tail end position of the mechanical arm are collected, a current observation environment is constructed, the current observation environment is input into the trained diffusion model, a target action sequence is generated, and the mechanical arm is controlled to execute control actions based on the target action sequence; according to the invention, the accuracy and real-time performance of the robot to control the flexible linear object can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and in particular to a robot control method, device, system and medium based on a diffusion model. BACKGROUND

[0002] Flexible linear objects (such as cables, ropes, catheters, etc.) are widely used in industrial manufacturing, medical auxiliary surgery, household service robots, and many other fields. Compared with rigid objects, flexible linear objects have infinite degrees of freedom, nonlinear deformation, strong coupling, and other characteristics, which bring significant challenges to robot control. How to enable robots to accurately perceive and control flexible linear objects has become one of the frontiers of robot perception and control research in recent years.

[0003] In order to realize the operation of flexible objects, the methods in the related art mostly rely on physical modeling and simulation, such as using finite element models, particle spring models or Kirchhoff models to simulate the deformation of objects. However, such methods rely on accurate modeling parameters, have large computational complexity, and are difficult to apply in real-time to actual task scenarios.

[0004] On the other hand, vision-based flexible object perception methods have received attention in recent years. However, most of these methods still rely on large-scale labeled data or manually designed intermediate representations, and lack the ability to model complex deformations.

[0005] In terms of trajectory generation, the methods in the related art mostly use imitation learning or reinforcement learning strategies for policy modeling. Using imitation learning combined with graph neural networks for operation policy generation, although there is some improvement in accuracy, still faces the problems of insufficient generalization ability and high computational complexity.

[0006] In addition, in terms of teaching methods, robot learning usually relies on dedicated teaching devices or manual annotation data, which not only increases the cost of data collection, but also limits the scalability and universality of the task. SUMMARY

[0007] Therefore, the embodiments of the present application aim to provide a robot control method, device, system and medium based on a diffusion model to solve one or more technical problems in the prior art and provide at least one beneficial option or create conditions.

[0008] In one aspect, the embodiments of the present application provide a robot control method based on a diffusion model, which comprises the following steps: Collecting the position information of the tip of the index finger and the state information of the flexible linear object during the human hand teaching process to construct training samples; The diffusion model is trained by using a diffusion model to train the training sample, a noise prediction network in the diffusion model is learned to map from a high noise state to a clear target state, and a trained diffusion model is obtained; the training process of the diffusion model includes forward noise adding and reverse noise removing; The state of the flexible linear object and the position of the end of the mechanical arm are collected, the current observation environment is constructed, the current observation environment is input into the trained diffusion model, a target action sequence is generated, and the mechanical arm is controlled to perform the operation action based on the target action sequence.

[0009] Optionally, the index position information of the index finger in the hand demonstration process and the state information of the flexible linear object are collected to construct a training sample, including: Video images in the hand demonstration process are acquired, the flexible linear object in the video images is converted into an ordered point sequence, and the index position information of the index finger in the hand demonstration process is synchronously acquired; The ordered point sequence of each frame of the flexible linear object, the index position and the state node of the target are spliced to serve as an observation environment; The observation environment of the current frame, the observation environment of the previous frame and the state node of the target are taken as inputs, and an action sequence formed by the index end trajectory at the corresponding moment is taken as a training label to form a training sample.

[0010] Optionally, the flexible linear object in the video images is converted into an ordered point sequence, and the index position information of the index finger in the hand demonstration process is synchronously acquired, including: The flexible linear object in the video images is modeled based on a vision algorithm, and an ordered point sequence representing the curved shape of the flexible linear object is extracted; the ordered point sequence includes a plurality of state nodes uniformly distributed along the center line of the flexible linear object; The hand tracking module is used to identify the hand skeleton key points in the video images to obtain the index position information of the index finger in the hand demonstration process.

[0011] Optionally, the flexible linear object in the video images is modeled based on a vision algorithm, and an ordered point sequence representing the curved shape of the flexible linear object is extracted, including: The center line of the flexible linear object is acquired by using a skeleton extraction algorithm, a plurality of sampling points are uniformly selected on the center line according to a preset interval, and the sampling points are taken as candidate nodes of the ordered point sequence; Each candidate node is regarded as a centroid of a Gaussian distribution by using a Gaussian mixture model, and the position of the candidate node is iteratively adjusted by using an expectation maximization algorithm, so that the candidate node fits the actual shape of the flexible linear object; Each candidate node is smoothed to obtain an ordered point sequence representing the curved shape of the flexible linear object.

[0012] Optionally, the training of the training sample by using the diffusion model includes: learning a mapping from a high-noise state to a clear target state by training a noise prediction network in the diffusion model, to obtain a trained diffusion model, including: In the forward noise adding stage, Gaussian noise is gradually added to the action sequence according to a preset noise scheduling strategy, to generate a noise action sequence under different noise levels. In the reverse noise removing stage, the noise action sequence and the observed environment are taken as inputs, a loss value is determined based on a predicted noise output by the noise prediction network, and the parameters of the diffusion model are updated to reduce the loss value to a loss threshold, to obtain the trained diffusion model.

[0013] Optionally, the training of the training sample by using the diffusion model includes: learning a mapping from a high-noise state to a clear target state by training a noise prediction network in the diffusion model, to obtain a trained diffusion model, including: The noise action sequence is randomly sampled, one time step is selected from all noise adding time steps each time, and the noise action sequence, the original action sequence and the true noise corresponding to the time step are extracted; The noise action sequence, the embedding vector of the time step and the observed environment are input into a noise prediction network based on a one-dimensional U-Net structure, to output a predicted noise. The mean square error loss between the predicted noise and the true noise is calculated, and the parameters of the noise prediction network are updated by a back propagation algorithm to reduce the mean square error loss value to a loss threshold, to obtain a trained noise prediction network.

[0014] Optionally, the acquisition of the state of the flexible linear object and the position of the end of the robot arm, the construction of the current observed environment, the input of the current observed environment into the trained diffusion model, the generation of a target action sequence, and the control of the robot arm to perform a manipulation action based on the target action sequence include: The current state image of the flexible linear object is acquired in real time, and is fused with the real-time position coordinates of the end of the robot arm after being converted into an ordered point sequence, to construct a current observed environment. The current observed environment and the target state node are input into the trained diffusion model, and a target action sequence of the end of the robot arm is generated through a multi-step reverse noise removing process, the target action sequence including position and attitude information of an end effector in a Cartesian space. The robot control system drives joint motors to perform corresponding manipulation actions according to the generated target action sequence, to realize deformation adjustment of the flexible linear object to a target state.

[0015] In another aspect, an embodiment of the present application provides a robot manipulation device based on a diffusion model, including: The data acquisition module is configured to acquire the index finger tip position information in the human hand demonstration process and the state information of the flexible linear object, and construct a training sample. The model training module is configured to train the training sample by using a diffusion model, learn a mapping from a high-noise state to a clear target state by training a noise prediction network in the diffusion model, and obtain a trained diffusion model. The motion control module is configured to acquire the state of the flexible linear object and the position of the end of the robot arm, construct a current observation environment, input the current observation environment into the trained diffusion model, generate a target action sequence, and control the robot arm to perform a control action based on the target action sequence.

[0016] In another aspect, an embodiment of the present application provides a robot control system based on a diffusion model, comprising: at least one processor; at least one memory configured to store at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the above method.

[0017] In another aspect, an embodiment of the present application provides a computer readable storage medium, which stores a processor executable program, and the processor executable program is used to execute the above method when executed by a processor.

[0018] Embodiments of the present application have the following beneficial effects: The present application constructs a training sample by acquiring the index finger tip position information in the human hand demonstration process and the state information of the flexible linear object, trains the training sample by using a diffusion model, makes the model learn a mapping from a high-noise state to a clear target state, and then generates a target action sequence according to a current observation environment to control a robot arm to perform a control action in actual control, effectively solves the problems of traditional methods, such as dependence on accurate physical modeling parameters, large amount of calculation, insufficient generalization ability, high demonstration cost, etc., improves the accuracy and real-time performance of the robot in controlling the flexible linear object, reduces the data acquisition cost, and enhances the expansibility and universality of the task. BRIEF DESCRIPTION OF DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed in the embodiments. Obviously, the drawings described below are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0020] Figure 1is a step flow schematic diagram of a robot control method based on a diffusion model provided by an embodiment of the application; Figure 2 is a framework diagram of a robot control method based on a diffusion model provided by an embodiment of the application; Figure 3 is a schematic diagram of a diffusion model generating an action sequence in an embodiment of the application; Figure 4 is a schematic diagram of a one-dimensional U-Net action sequence noise prediction network in an embodiment of the application; Figure 5 is a structural block diagram of a robot control device based on a diffusion model provided by an embodiment of the application; Figure 6 is a structural block diagram of a robot control system based on a diffusion model provided by an embodiment of the application. DETAILED DESCRIPTION

[0021] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application is further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and not intended to limit the present application.

[0022] It should be noted that although the module division is made in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a manner different from the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification and claims and the above-described drawings are used to distinguish similar objects and do not necessarily describe a specific order or sequence.

[0023] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in the specification herein is for describing the embodiments of the present application only and is not intended to limit the present application.

[0024] In addition, the described features, structures or characteristics can be combined in any suitable manner in one or more embodiments. In the following description, many specific details are provided to give a full understanding of the embodiments of the present application. However, one skilled in the art will realize that the technical solutions of the present application can be practiced without one or more of the specific details, or other methods, components, devices, steps, etc. can be used. In other cases, well-known methods, devices, implementations or operations are not shown or described in detail to avoid obscuring the aspects of the present application.

[0025] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically separate entities. That is, these functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0026] The flowcharts shown in the accompanying drawings are for illustrative purposes only and do not necessarily include all contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may be decomposed, while others may be combined or partially combined. Therefore, the actual execution order may vary depending on the actual situation.

[0027] Related technologies include deep neural network-based image-to-trajectory mapping methods, which extract the state of flexible objects from images and then predict the trajectory of the robot's end point. Video prediction models are used to infer the flexible object's future state and generate control strategies. While effective, these methods suffer from unstable predictions, slow convergence, and limitations in dataset quality.

[0028] The following are the main problems in the related technologies for robot manipulation of flexible linear objects: Limited ways of expressing shapes: Related technologies usually use images, point clouds or meshes to represent the shapes of flexible objects. They lack effective modeling of the essential geometric features of the shapes, making it difficult to use them as control targets to accurately guide robots to complete manipulation tasks.

[0029] The teaching method is highly dependent: The teaching method of related technologies needs to rely on dedicated hardware or cumbersome labeling process, which not only increases the cost of data collection and processing, but also limits the scalability and practicality of the system.

[0030] The present invention aims to provide a robot manipulation method, device, system, and medium based on a diffusion model. This method utilizes a visual approach to reduce the dimension of a flexible object's shape into shape constraints in the form of an ordered array of points, which can better express its target form. The method uses videos of human hand manipulation as teaching data, equating the fingertips to the robot's tips, significantly lowering the data acquisition threshold and improving training efficiency. The method is suitable for precise manipulation and shape adjustment of flexible linear objects such as cables, wires, ropes, and hoses. This method combines visual perception with a diffusion model for trajectory generation, exhibiting excellent versatility and robustness. It can be widely applied to flexible object manipulation tasks such as intelligent manufacturing, service robotics, and medical-assisted surgery, improving automation and operational precision.

[0031] like Figure 1 and Figure 2 As shown, Figure 1 An embodiment of the present invention provides a robot manipulation method based on a diffusion model, the method comprising the following steps: S100, collect the index finger end position information in the human hand demonstration process and the state information of the flexible linear object, and construct a training sample; S200, train the training sample by using a diffusion model, learn the mapping from a high noise state to a clear target state through a noise prediction network in the diffusion model, and obtain a trained diffusion model; the training process of the diffusion model includes forward noise adding and reverse noise removing; S300, collect the state of the flexible linear object and the end position of the mechanical arm, construct a current observation environment, input the current observation environment into the trained diffusion model, generate a target action sequence, and control the mechanical arm to execute a control action based on the target action sequence.

[0032] The application provides a robot control method, device and system based on a diffusion model and a medium. In a demonstration process, a state representation of a flexible linear object and a position of a finger end are detected by using an existing vision scheme. When a human hand demonstration video is collected, three-dimensional coordinate information is acquired by using a binocular vision system or a depth camera. Two-dimensional image coordinates are converted into three-dimensional space coordinates through camera calibration parameters, so that the space position data of the index finger end position and the flexible linear object state have a unified coordinate system, and the accuracy of model training and reasoning is improved.

[0033] Specifically, the state of the flexible linear object is abstracted as a group of ordered nodes uniformly distributed along the center line of the flexible linear object and the position of the index finger end. The above two groups of information are combined and regarded as the observation of the model. In the training process, the observation is used as the input of the model. The state of the flexible linear object finally placed is used as the training constraint. The change of the position of the index finger end is regarded as the action of the model, and is used as the training label in the training process. Then, the diffusion model is trained to have the action prediction ability. In the reasoning stage, the real-time flexible linear object state and the end position of the mechanical arm are acquired by using a camera and a robot demonstrator to obtain the observation environment. The state of the target shape is input into the trained diffusion model together. The diffusion model generates the action sequence of the end of the mechanical arm. The robot executes the motion sequence, so that the shape control of the deformable flexible linear object is realized.

[0034] In some embodiments, the collection of the index finger end position information in the human hand demonstration process and the state information of the flexible linear object, and the construction of the training sample include: S110, acquiring a video image in a human hand demonstration process, converting a flexible linear object in the video image into an ordered point sequence, and synchronously acquiring index finger end position information in the human hand demonstration process; S120, splicing the ordered point sequence of each frame of the flexible linear object, the index finger end position and the state node of the target as an observation environment; S130, taking the observation environment of the current frame, the observation environment of the previous frame and the state node of the target as inputs, taking the action sequence formed by the index finger end trajectory at the corresponding moment as a training label, and forming a training sample.

[0035] In this embodiment, by acquiring video images in the hand demonstration process, the contour features of the flexible linear object are extracted through image recognition technology, and are converted into an ordered point sequence uniformly distributed along the center line; at the same time, the real-time position coordinates of the index finger end in the hand demonstration process are captured by using a target detection algorithm. The ordered point sequence of each frame, the index finger end position coordinates and the preset target state node are spliced to form the observation environment data structure of the model. The observation environment data of continuous multiple frames are arranged in time sequence, and combined with the action sequence formed by the index finger end moving trajectory in the corresponding time period to form a training sample set, which is used for diffusion model parameter learning.

[0036] This embodiment effectively reduces the complexity of demonstration data acquisition by converting visual information in the hand demonstration process into structured training data, while retaining the key association features between flexible linear object deformation and operation action, thereby providing high-quality input samples for subsequent diffusion model training.

[0037] The implementation process of the scheme proposed in the present application is shown in Figure 2 The specific implementation steps of each link are as follows: I. Demonstration and data acquisition At present, most diffusion model strategies take the video of the working environment as input and output the end-to-end motion control scheme of the robot. However, due to the high dimension and large amount of data of video information, traditional models often have difficulty in obtaining good training results under limited video data. Therefore, in the present application, the input of the diffusion model uses low-dimensional point sequence information after visual processing. Specifically, a visual tracking algorithm for deformable flexible linear objects is used to represent the shape change of the object through an ordered point sequence uniformly distributed on the axis of the object.

[0038] In addition, the traditional data acquisition method through robot arm demonstration has the limitations of long time consumption, high labor intensity and high cost. The hand-based demonstration data can significantly reduce the time and cost of data acquisition. For the video-based end-to-end diffusion model strategy, although the difference between the hand and the robot arm needs to be solved, the input method based on state representation in the present application only focuses on the point position interacting with the cable, so the finger end position can be directly captured through the visual scheme and input into the diffusion model for training.

[0039] This method enables the model to learn the motion patterns of the interaction points. During the inference phase, only the camera and the robot's hand-eye calibration, combined with the inverse kinematics of the robot, are needed to establish the mapping relationship between the motion point position and the robot's end position, enabling precise control of the deformable flexible linear object shape by the robot.

[0040] By reducing the dimensionality of the model input and simplifying the collection process of the demonstration data, the complexity of data collection and training is reduced, while the robustness and practicality of the model in actual operation are improved. This innovative method can effectively reduce the dependence on complex manual demonstration, reduce development costs, and adapt to a wider range of application scenarios.

[0041] In some embodiments, the flexible linear object in the video image is converted into an ordered point sequence, and the index finger end position information in the hand demonstration process is synchronously obtained, including: Based on the visual algorithm, the state of the flexible linear object in the video image is modeled, and an ordered point sequence representing the curve shape of the flexible linear object is extracted; the ordered point sequence includes a plurality of state nodes uniformly distributed along the center line of the flexible linear object; The hand tracking module is used to identify the hand skeleton key points in the video image to obtain the index finger end position information in the hand demonstration process.

[0042] In this embodiment, the state of the flexible linear object is modeled by a visual algorithm. Specifically, an image segmentation technology based on deep learning can be used to accurately extract the contour area of the flexible linear object from the video image, and then a curve fitting algorithm is used to process the contour to obtain the center line. Subsequently, a plurality of points are uniformly selected on the center line according to a preset sampling interval to form an ordered point sequence, and the coordinate information of each point collectively constitutes a quantitative description of the current shape of the flexible linear object. For the hand tracking module, an existing open-source hand pose estimation model can be integrated, which can detect the hand in the video image in real time and output the three-dimensional coordinates of a plurality of skeleton key points including the index finger end. In terms of synchronous acquisition, the frame number of the video image is aligned with the timestamp output by the hand tracking module to ensure that the ordered point sequence of each frame of the flexible linear object corresponds to the index finger end position information at the same time, laying a precise data foundation for subsequent construction of training samples.

[0043] Specifically, to achieve an abstract representation of the state of the flexible linear object, the present application uses a pure visual algorithm for real-time tracking of deformable linear objects in occlusion scenarios, the core goal of which is to abstract the state of the flexible linear object into an ordered point sequence uniformly distributed along the center line wherein represents the number of nodes, represents the pixel coordinates of the nodes. The idea is to treat the state of the cable as a set of nodes as the centroids of a number of Gaussian distributions in a Gaussian Mixture Model (GMM), i.e. the algorithm assumes that the points in the point cloud coming from the flexible linear object are randomly sampled from these Gaussian distributions, i.e. where denotes the n-th point in the point cloud at time t, denotes the coordinate of the n-th state node at time t, is the variance of the Gaussian distribution, denotes the Gaussian distribution. The final positions of the nodes are then determined using an Expectation-Maximization (EM) algorithm that iteratively optimizes. These nodes strictly follow the topology of the flexible linear object, being evenly distributed along its centerline with a fixed interval, and completely and continuously outline the geometry of the object. The connections between neighboring nodes form piecewise linear curves that accurately reflect the deformation states of the flexible linear object in space, such as bending and stretching. The coordinates of each node not only identify its specific position in the picture, but also convey the overall pose information of the object through the relative position relationship between nodes, such as the change in the direction of the node sequence, which can reflect the bending direction of the object.

[0044] In some embodiments, the visual algorithm based on the state modeling of the flexible linear object in the video image extracts an ordered point sequence representing the curved shape of the flexible linear object, including: obtaining the centerline of the flexible linear object through a skeleton extraction algorithm, and uniformly selecting a plurality of sampling points on the centerline according to a predetermined interval, the sampling points being candidate nodes of the ordered point sequence; treating each candidate node as a centroid of a Gaussian distribution using a Gaussian Mixture Model, and iteratively adjusting the positions of the candidate nodes through an Expectation-Maximization algorithm to make the candidate nodes fit the actual shape of the flexible linear object; smoothing each candidate node to obtain an ordered point sequence representing the curved shape of the flexible linear object.

[0045] The embodiment separates the main structure of the flexible linear object from the pre-processed video image through the skeleton extraction algorithm, removes redundant pixels through the thinning operation, and retains the single-pixel-width center line contour. Then, according to the length and shape complexity of the object, the sampling interval is set, and the candidate nodes are selected equidistantly on the center line, ensuring that the node distribution can completely cover the object shape and avoid data redundancy caused by high density. Subsequently, the candidate nodes are used as the initial centroid of the Gaussian mixture model, and the parameters of each Gaussian distribution are continuously optimized through the expectation maximization algorithm, so that the point cloud distribution generated by the model is highly matched with the actual contour characteristics of the object in the image, thereby accurately adjusting the node position. Finally, the sliding average filter is used to smooth the node coordinates, eliminate the position fluctuations caused by noise interference, and obtain a continuous and stable ordered point sequence, which provides accurate shape representation data for subsequent model training.

[0046] First, the input video image is pre-processed, including image denoising, contrast enhancement and other operations, to improve the accuracy of subsequent processing. Then, the edge detection algorithm is used to extract the contour information of the flexible linear object, and the approximate area of the object in the image is preliminarily determined. Then, the instance segmentation model based on deep learning is used to separate the flexible linear object from the complex background, and the binary image containing only the target object is obtained. Then, the skeleton extraction algorithm is used to obtain the center line of the flexible linear object, which can accurately reflect the overall shape of the object. On this basis, multiple sampling points are uniformly selected on the center line according to the preset interval parameter, and these sampling points will be used as the initial candidate nodes of the ordered point sequence. In order to further optimize the position of the nodes, the Gaussian mixture model (GMM) is introduced, and each candidate node is regarded as the centroid of a Gaussian distribution, and the expectation maximization (EM) algorithm is used to iteratively adjust the position of the nodes, so that the nodes can better fit the actual shape of the flexible linear object. At the same time, combined with the idea of dynamic programming, the node sequence is smoothed to ensure that the distance between adjacent nodes remains consistent and to avoid local mutations, and finally an ordered point sequence that can accurately represent the curve shape of the flexible linear object is obtained.

[0047] Then, the Media Pipe framework is used to identify the key points of the operator's hand to obtain the position information of the finger tip. Media Pipe is an open-source cross-platform computer vision framework developed by Google, widely used in real-time pose estimation, face detection, hand tracking and other tasks, with good computing efficiency and real-time performance. In this invention, the built-in Hand Tracking module of Media Pipe is used to process the video image captured by the camera, which can output the coordinate data of 21 three-dimensional hand key points in real time, including the position of the index finger tip.

[0048] During the data collection process, the system continuously captures hand motion images, and uses the Media Pipe model to identify and locate the key points of the hand in each frame of image, and extracts the coordinates of the tip of the index finger. This position is regarded as the expression of the human operator's control intention, and is used as a representative point of the "robot's end effector" in subsequent data labeling and model training, participating in the mapping modeling of the entire operation action.

[0049] In some embodiments, the training of the training sample by the diffusion model includes: In the forward noise adding stage, Gaussian noise is gradually added to the action sequence according to a preset noise scheduling strategy, to generate noise action sequences at different noise levels; In the reverse noise removing stage, the noise action sequence and the observed environment are input, a loss value is determined based on the predicted noise output by the noise prediction network, and the parameters of the diffusion model are updated to reduce the loss value to a loss threshold, to obtain the trained diffusion model.

[0050] Specifically, the noise prediction network adopts a U-Net architecture, the input of which includes the noise action sequence at the current time, the observed environment data (such as the ordered point sequence of the flexible linear object, the position of the tip of the index finger, and the target state node), and the time step embedding vector of the current noise level, and the output is the predicted value of the original noise. In the training process, the network parameters are continuously optimized by minimizing the mean square error loss function between the predicted noise and the real added noise. After multiple rounds of iterative training, when the loss function converges and the action prediction accuracy of the model on the validation set reaches a preset threshold, the training is stopped, and a diffusion model with stable action prediction capability is obtained. The model can generate an end-of-arm action sequence close to the optimal one through the reverse noise removing process under the condition of given initial observed environment and target shape, to achieve precise shape control of the flexible linear object.

[0051] In some embodiments, the training of the training sample by the diffusion model includes: Randomly sampling the noise action sequence, selecting one time step from all noise-added time steps each time, and extracting the noise action sequence, the original action sequence, and the real noise corresponding to the time step; Inputting the noise action sequence, the embedding vector of the time step, and the observed environment into the noise prediction network based on the one-dimensional U-Net structure, and outputting the predicted noise; The mean square error loss between the predicted noise and the real noise is calculated, and the parameters of the noise prediction network are updated through the back propagation algorithm so that the mean square error loss value is reduced to the loss threshold, and the trained noise prediction network is obtained.

[0052] This embodiment effectively integrates the state information of flexible linear objects with the position of the robotic arm's end position to construct an observation environment. Combined with the powerful noise prediction and denoising capabilities of the diffusion model, this approach achieves end-to-end modeling, from complex environment perception to precise motion sequence generation. This diffusion model-based training method not only fully learns the relationship between motion characteristics and environmental constraints during human instruction, but also enhances the model's adaptability to the randomness and uncertainty of motion sequences through a two-way process of forward denoising and backward denoising. This lays a solid foundation for the robotic arm's real-time and precise manipulation of flexible linear objects in the subsequent reasoning phase.

[0053] 2. Model Training The diffusion model used in the present invention is a type of generative model that aims to recover data from noise through a step-by-step denoising process.

[0054] like Figure 3 As shown in Figure 3, the diffusion model is divided into forward denoising and reverse denoising processes, which correspond to the training and inference processes of the model respectively.

[0055] Specifically, the forward noise addition process is to pass the original data (action sequence) through multiple time steps (like ) gradually transforms into pure noise, for each time step , the process of data noise addition can be expressed as formula (1).

[0056] (1); in, yes Action sequence of time steps; yes Action sequence of time steps; It is control The hyperparameter of the time step noise level belongs to the pre-set noise schedule (variance schedule), represents Gaussian distribution; represents the identity matrix; express The noise coefficient of the time step is used to scale the action sequence of the previous time step during the noise addition process to control the intensity of the noise addition; yes Covariance matrix of the time-step noise-adding process.

[0057] Since the forward noise adding of data can be considered as a Markov process, the relationship between the initial action sequence and the noise action sequence after a specified noise step can be derived by mathematical deduction. If , , denotes the noise coefficient at the jth time step, then we have: (2); wherein denotes the initial action sequence, denotes the noise coefficient at the tth time step, denotes the product of noise coefficients from the 1st time step to the tth time step, denotes the mean term after scaling the initial action sequence, which gradually decreases with the increase of time step t, reflecting the decreasing influence weight of the original action sequence in the noise adding process; denotes the proportion of the original action sequence replaced by noise after t steps of noise adding, and the greater the value, the stronger the influence of noise on data; denotes the covariance matrix of the tth time step noise action sequence, reflecting the dispersion degree of the noise-added data.

[0058] In the actual training process, the noise-added noise action sequence is randomly sampled using the sampling formula (3), and each time a time step t is selected from all noise adding time steps, and the noise action sequence corresponding to the time step, the original action sequence and the true noise are extracted; the noise action sequence, the embedding vector of the time step t and the observed environment are input into the noise prediction network, and the network outputs the predicted value of the noise .

[0059] (3); wherein denotes the true noise, ; Then, a neural network is trained to predict the noise, and the mean square error loss between the predicted noise and the true noise is calculated, and the parameters of the noise prediction network are updated by the back propagation algorithm to minimize the loss function value. Through the iterative optimization of a large number of training samples, the noise prediction network gradually learns the noise removal rule of the action sequence under different noise levels, thereby constructing a diffusion model that can accurately predict noise and generate clear action sequences.

[0060] The training goal is to minimize the following formula: (4); wherein, represents a loss function for measuring the difference between the predicted noise of the neural network output and the real noise, represents the learnable parameters of the neural network, is the observed environment and the target pose of the flexible linear object, is the neural network for outputting the predicted noise ; represents the expectation of all possible time steps t, initial action sequences and real noise . By minimizing this loss function, the noise prediction network can accurately learn the mapping relationship from the high noise state to the clear target state.

[0061] As Figure 4 shown, in the training process, the noise prediction network of the present application adopts a one-dimensional U-Net neural network. U-Net is a convolutional neural network structure for image segmentation tasks, which has an encoder-decoder structure. One-dimensional U-Net is suitable for processing sequence data. The noise prediction network learns the mapping from the high noise state to the clear target state by step-by-step denoising of the perturbation of the input data.

[0062] In the training sample construction process, the present application adopts the following strategies: Use the existing vision system to abstractly represent the state of the flexible linear object node, and synchronously obtain the index tip position information in the human hand demonstration process; splice the state node of the flexible linear object of each frame, the index tip position and the target state node as the observed environment ; Take the continuous 16 frames of index tip trajectory as the original action sequence, as the training label for diffusion modeling; In each training sample, the input is the state node of the target and the observed environment of two frames (the current frame and the previous frame), and the training label is the action sequence formed by the index trajectory in this period of time.

[0063] In the reverse denoising inference process, starting from the standard Gaussian noise , at each time step , the neural network is used to output the predicted noise, and denoising sampling is performed; (5); wherein, represents variance scheduling, represents sampling randomness, ; continuously iterate to finally obtain the noise-free action sequence at time to make the robot execute.

[0064] In some embodiments, the flexible linear object state is collected and the position of the robot arm end is acquired, the current observation environment is constructed, the current observation environment is input into the trained diffusion model, the target action sequence is generated, and the robot arm is controlled to perform the manipulation action based on the target action sequence, comprising: The current state image of the flexible linear object is collected in real time, and after being converted into an ordered point sequence, it is fused with the real-time position coordinates of the robot arm end to construct the current observation environment; The current observation environment and the target state node are input into the trained diffusion model, and the target action sequence of the robot arm end is generated through a multi-step reverse denoising process, which includes the position and attitude information of the end effector in the Cartesian space; The robot control system drives the joint motor to perform the corresponding manipulation action according to the generated target action sequence, so as to realize the deformation adjustment of the flexible linear object to the target state.

[0065] In this embodiment, the state information of the flexible linear object is effectively fused with the position of the robot arm end to construct the observation environment, and the powerful noise prediction and denoising ability of the diffusion model is combined to realize the end-to-end modeling from complex environment perception to precise action sequence generation.

[0066] III. Inference phase In the inference phase, the present application realizes the real-time collection and input construction of the flexible linear object state and the position of the robot arm end through the depth camera and the robot system. Specifically, the system includes a depth camera installed on a work platform and a robot end position output. The vision sensor can continuously collect image information of the flexible linear object, and based on the previously mentioned vision algorithm, the current state of the deformable flexible linear object is modeled to extract an ordered point sequence representing the curve shape of the object; at the same time, the position of the robot arm end effector in the robot system is read and mapped into the image through the hand-eye calibration matrix.

[0067] Subsequently, the system combines the flexible object state point sequence collected by the current frame with the position coordinates of the end effector to form a model input observation environment quantity, which is input into the trained diffusion model. The model uses an inverse diffusion process to gradually denoise from random noise to generate a series of action sequences, which describe the desired trajectory of the robot arm end in the future time steps in the form of ordered points.

[0068] The robot control system executes the corresponding action point by point according to the sequence, drives the end effector to move along the specified trajectory, and thus realizes the step-by-step manipulation of the flexible linear object. Since the generation process of the action sequence integrates the behavior strategy learned from human hand demonstration, the robot can reproduce the shape adjustment behavior consistent with human operation while maintaining the target shape constraint.

[0069] Through the above manner, the flexible linear object is efficiently, autonomously and accurately controlled, and the method can be widely applied to cable arrangement, medical catheter laying, soft material assembly and other robot operation scenes.

[0070] Compared with the flexible linear object control technology of the related art, the robot control method based on the diffusion model has the following advantages: First, the existing method depends on high-dimensional visual information (such as a video frame sequence) for end-to-end modeling, and the model training process requires a large amount of data and high label accuracy, and often shows insufficient generalization ability when facing complex deformation and various operation tasks. The application reduces the input dimension and improves the model training efficiency and generalization by abstracting the state of the flexible linear object into a structured ordered node sequence and using it as the condition input of the model while keeping the complete expression of the geometric features of the object shape.

[0071] Second, the traditional teaching method generally needs complex sensor deployment or robot playback mechanism, while the application innovatively uses a hand teaching video to automatically extract the finger tip trajectory using a lightweight vision module such as Media Pipe, instead of real robot trajectory acquisition, greatly reducing the physical cost and time cost of data acquisition, making the control learning of flexible objects more efficient, practical and easy to promote. Combined with the advantage of the conditional diffusion model in modeling complex target distribution, the application can accurately control the shape of the target object and has good robustness and real-time performance.

[0072] Referring to Figure 5 The embodiment of the application provides a robot control device based on a diffusion model, which comprises: A data acquisition module is configured to acquire the position information of the index finger tip and the state information of the flexible linear object during the hand teaching process, and to construct training samples. A model training module is configured to train the training samples using a diffusion model, learn the mapping from a high-noise state to a clear target state through a noise prediction network in the diffusion model, and obtain a trained diffusion model; the training process of the diffusion model comprises forward noise addition and reverse noise removal. A motion control module is configured to acquire the state of the flexible linear object and the position of the end of the robot arm, construct a current observation environment, input the current observation environment into the trained diffusion model, generate a target action sequence, and control the robot arm to perform a control action based on the target action sequence.

[0073] It can be seen that the contents in the above method embodiment are applicable to the device embodiment, the device embodiment specifically realizes the same functions as the above method embodiment, and achieves the same beneficial effects as the above method embodiment.

[0074] With reference to Figure 6 The embodiment of the present application provides a robot control system based on a diffusion model, which comprises: at least one processor; at least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the above method.

[0075] It can be seen that the contents in the above method embodiments are all applicable to the present system embodiment, the present system embodiment specifically implements the same functions as the above method embodiments, and achieves the same beneficial effects as the above method embodiments.

[0076] In addition, the embodiment of the present application further discloses a computer program product or a computer program, which is stored in a computer readable storage medium. The processor of the computer device can read the computer program from the computer readable storage medium, and the processor executes the computer program, so that the computer device executes the above method. Similarly, the contents in the above method embodiments are all applicable to the present storage medium embodiment, the present storage medium embodiment specifically implements the same functions as the above method embodiments, and achieves the same beneficial effects as the above method embodiments.

[0077] The device embodiments described above are only schematic, wherein the units described as separate components can or can not be physically separate, that is, can be located in one place, or can be distributed on multiple network units. According to actual needs, part or all of the modules can be selected to achieve the purpose of the present embodiment scheme.

[0078] Those skilled in the art can understand that all or some of the steps in the above disclosed method, the function modules / units in the system and the device can be implemented as software, firmware, hardware or appropriate combination thereof.

[0079] The terms "first," "second," "third," "fourth," and the like (if any) in the specification of the present application and the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a particular order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in orders other than those illustrated or described herein. In addition, the terms "including" and "having," and any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such process, method, product, or apparatus.

[0080] It should be understood that in this application, "at least one (item)" means one or more, and "more" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships can exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or plural.

[0081] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0082] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0083] In addition, each of the functional units in the embodiments of the present application can be integrated in one processing unit, or each unit can exist alone physically, or two or more units can be integrated in one unit. The integrated unit can be implemented in the form of hardware, or in the form of a software functional unit.

[0084] When the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on such an understanding, the technical solutions of the present application essentially, or the part that contributes to the prior art, or all or a part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes multiple instructions used to cause a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in the embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, and various other media that can store programs.

[0085] The preferred embodiments of the embodiments of the present application are described above with reference to the accompanying drawings, and are not intended to limit the scope of the embodiments of the present application. Any modification, equivalent replacement, and improvement made by those skilled in the art without departing from the scope and essence of the embodiments of the present application shall be within the scope of the embodiments of the present application.

Claims

1. A method for robot manipulation based on a diffusion model, characterized in that, The method comprises the following steps: Collecting index finger tip position information in a human hand demonstration process and state information of a flexible linear object, and constructing a training sample; Training the training sample by using a diffusion model, learning a mapping from a high-noise state to a clear target state through a noise prediction network in the diffusion model, and obtaining a trained diffusion model; the training process of the diffusion model comprises forward noise addition and reverse noise removal; Collecting the state of the flexible linear object and the position of the robot arm tip, constructing a current observation environment, inputting the current observation environment into the trained diffusion model, generating a target action sequence, and controlling the robot arm to perform a control action based on the target action sequence.

2. The method of claim 1, wherein, The method comprises the following steps: Obtaining video images in a human hand demonstration process, converting the flexible linear object in the video images into an ordered point sequence, and synchronously obtaining index finger tip position information in the human hand demonstration process; Splicing the ordered point sequence of each frame of the flexible linear object, the index finger tip position, and the state node of the target as an observation environment; Taking the observation environment of the current frame, the observation environment of the previous frame, and the state node of the target as input, and taking the action sequence formed by the index finger tip trajectory at the corresponding time as a training label to form a training sample.

3. The method of claim 2, wherein, The method comprises the following steps: Modeling the state of the flexible linear object in the video images based on a visual algorithm, and extracting an ordered point sequence representing the curved shape of the flexible linear object; the ordered point sequence comprises a plurality of state nodes uniformly distributed along the center line of the flexible linear object; Using a hand tracking module to identify the hand skeleton key points in the video images to obtain the index finger tip position information in the human hand demonstration process.

4. The method of claim 3, wherein, The method comprises the following steps: Obtaining the center line of the flexible linear object through a skeleton extraction algorithm, uniformly selecting a plurality of sampling points on the center line according to a preset interval, and taking the sampling points as candidate nodes of the ordered point sequence; Using a Gaussian mixture model to regard each candidate node as a centroid of a Gaussian distribution, and iteratively adjusting the positions of the candidate nodes by using an expectation maximization algorithm, so that the candidate nodes fit the actual shape of the flexible linear object; Smoothing each candidate node to obtain an ordered point sequence representing the curved shape of the flexible linear object.

5. The method of claim 1, wherein, The method comprises the following steps: In the forward noise addition stage, gradually add Gaussian noise to the action sequence according to a preset noise scheduling strategy to generate noise action sequences at different noise levels; In the reverse denoising stage, the noisy action sequence and the observed environment are used as input, and the loss value is determined based on the predicted noise output by the noise prediction network. The parameters of the diffusion model are updated to reduce the loss value to the loss threshold, and a trained diffusion model is obtained.

6. The method of claim 5, wherein, The method uses the noise action sequence and the observation environment as input, determines the loss value based on the predicted noise output by the noise prediction network, and updates the parameters of the diffusion model so that the loss value is reduced to the loss threshold to obtain a trained diffusion model, including: Randomly sample the noise action sequence, select one time step from all the noise time steps each time, and extract the noise action sequence, original action sequence and real noise corresponding to the time step; The noise action sequence, the embedding vector of the time step and the observation environment are input into the noise prediction network based on the one-dimensional U-Net structure, and the predicted noise is output; The mean square error loss between the predicted noise and the actual noise is calculated, and the parameters of the noise prediction network are updated through the back propagation algorithm so that the mean square error loss value is reduced to the loss threshold, and the trained noise prediction network is obtained.

7. The method of claim 1, wherein, The method collects the state of the flexible linear object and the position of the end of the robotic arm, constructs the current observation environment, inputs the current observation environment into the trained diffusion model, generates a target action sequence, and controls the robotic arm to perform manipulation actions based on the target action sequence, including: The current state image of the flexible linear object is collected in real time, converted into an ordered point sequence, and then integrated with the real-time position coordinates of the end of the robotic arm to construct the current observation environment; The current observation environment and target state node are input into the trained diffusion model, and a multi-step inverse denoising process is used to generate the target action sequence of the end effector. This target action sequence contains the position and posture information of the end effector in Cartesian space. The robotic arm control system drives the joint motors to perform corresponding manipulation actions according to the generated target action sequence, realizing the deformation adjustment of the flexible linear object to the target state.

8. A robot handling device based on a diffusion model, characterized in that include: The data acquisition module is used to collect the position information of the index finger end during the teaching process and the status information of the flexible linear object to construct training samples; A model training module is used to train training samples using a diffusion model, and learn the mapping from a high-noise state to a clear target state by training the noise prediction network in the diffusion model to obtain a trained diffusion model; the training process of the diffusion model includes forward denoising and reverse denoising; The motion control module is used to collect the state of the flexible linear object and the position of the end of the robotic arm, construct the current observation environment, input the current observation environment into the trained diffusion model, generate the target action sequence, and control the robotic arm to perform manipulation actions based on the target action sequence.

9. A robot manipulation system based on a diffusion model, characterized in that, include: at least one processor; at least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the method according to any one of claims 1 to 7.

10. A computer readable storage medium having stored therein a program which is executable by a processor, characterized in that, The processor-executable program is configured to perform the method according to any one of claims 1 to 7 when executed by the processor.

Images