Optimization method for object recognition and obstacle avoidance grabbing of double-arm robot

Abstract

The invention discloses an optimization method for object recognition and obstacle avoidance grabbing of a double-arm robot, and the method comprises the steps: collecting an image of an object to be grabbed, carrying out the restoration based on a diffusion network and an image generation network, generating a point cloud, and constructing a target object point cloud model library; generating a point cloud for a to-be-grabbed object image collected in real time, performing point cloud segmentation and point cloud filtering to obtain point cloud information after filtering optimization, and performing point cloud registration with a target object point cloud model library; adopting a grabbing pose detection algorithm based on point cloud local concavity and convexity constraint and point cloud pose constraint to obtain an optimal grabbing pose; planning a motion path of the two arms of the robot based on a path planning algorithm of two-arm decoupling planning, two-way search and a target point guiding force field; and multi-range movement of the robot in the factory is planned based on a reward and punishment mechanism. The training effect of the blurred image is improved, point cloud filtering is adopted, pose constraints are improved, and the mechanical arm is helped to be effectively matched with the target object.

Description

Technical Field

[0001] The invention belongs to the technical field of robot object grasping, and in particular relates to an optimization method for object recognition and obstacle avoidance grasping of a dual-arm robot. Background Art

[0002] The manufacturing industry is facing the challenge of transformation and upgrading. Improving production efficiency and reducing labor costs have become the goals pursued by enterprises. With the rapid development of intelligent manufacturing and automation technology, dual-arm robots can effectively improve the automation level of production lines with their flexibility and collaborative operation capabilities, and play an increasingly important role in industries such as industry, logistics, and medical care. In many application scenarios, robot grasping operations are the key link in realizing automated processes. Under the command of external instructions, driving dual-arm robots to move autonomously to grasp different objects is now a common robotic arm operation method. However, when dealing with different objects with complex shapes and variable postures, existing robot grasping often has problems such as low grasping success rate, poor operation efficiency, and easy collision of motion trajectories.

[0003] In recent years, many scholars have proposed corresponding solutions to the above problems. Wan Qin published an algorithm for estimating the posture of workpieces with different textures at different points in space in the journal "Control Theory and Applications", but the algorithm can only be applied to specific industrial workpieces and cannot be generalized to other grasping objects; Jin Kaixuan proposed to use point cloud to capture the surface imaging image of the grasped object and transmit the surface imaging image to the upper computer grasping control unit, but its grasping posture is relatively single, which makes it difficult to change the grasping posture to effectively grasp irregular parts in complex environments. It is suitable for industrial robotic arms with fixed grasping postures, but not suitable for dual-arm robots used in daily life; Liu Guanghui proposed a dual-arm motion algorithm that uses robotic arm kinematic models and collision detection rules as operating constraints in the journal "Computer Integrated Systems", but the algorithm lacks node guidance in the middle of the search process, and the algorithm may have local optimal solution properties, resulting in low algorithm planning efficiency. Summary of the invention

[0004] The main purpose of the present invention is to overcome the shortcomings and deficiencies of the prior art, provide an optimization method for object recognition and obstacle avoidance grasping of a dual-arm robot, reduce the texture blur of blurred images in the grasping data set, improve the situation where the grasping network mistakes the edge of an object as the contour of a graspable object, thereby improving the generalization ability of the robot arm to grasp different objects, improve the success rate of the dual-arm robot in processing object grasping tasks, and enhance the safety and flexibility of the dual-arm robot's spatial motion planning.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions: One aspect of the present invention provides an optimization method for object recognition and obstacle avoidance grasping of a dual-arm robot, comprising the following steps: Collect the image of the object to be grasped and repair it based on the diffusion network and image generation network, generate point cloud for the repaired image, and build a point cloud model library of the target object; Generate point cloud for the real-time captured image of the object to be grasped, perform point cloud segmentation and point cloud filtering, and obtain point cloud information after filtering optimization; Performing point cloud registration on the filtered and optimized point cloud information and the target object point cloud model library to determine whether it is an object to be grasped; After identifying the object to be grasped, a grasping posture detection algorithm based on local concavity and convexity constraints of point cloud and point cloud posture constraints is used to obtain the optimal grasping posture; The motion path of the robot's dual arms is planned based on dual-arm decoupling planning, bidirectional search, and the path planning algorithm of the target point gravitational field; Plan the robot's movement in the working environment based on the reward and punishment mechanism.

[0006] As a preferred technical solution, the restoration based on the diffusion network and the image generation network is specifically as follows: Through the generator 𝐺, from the initial distribution p ( z ) is mapped to the high-dimensional data space to obtain the generator sample x g = G ( z ), z~p ( z ), the distribution is expressed as ; A real dataset obtained from collecting images of objects to be grasped p ( x ) to extract the real data sample 𝑥, x~p ( x ); Diffusion process: at each step, the generator samples x g Add Gaussian noise to the real data sample 𝑥, and get the noisy image sample after 𝑇 steps, specifically: ; in, q ( y | x ) is the mixed distribution in the 𝑇th iteration; 𝑦 is the added random interference noise sample; the size of the step number 𝑇 is affected by β t constraint: ; Mixture components obtained by diffusion q ( y | x ,t ) is a Gaussian distribution whose mean is proportional to 𝑥 and whose variance depends on the noise level at the current step 𝑡, expressed as follows: ; in, a t =1- β t , , I is the threshold parameter of the set distribution; The mixed distribution obtained by the diffusion process is used to train a discriminator 𝐷 that can distinguish between real and generated noise samples, and the image judged as true by the discriminator 𝐷 is output as a repaired image.

[0007] As a preferred technical solution, the mixed distribution obtained by the diffusion process is used to train a discriminator 𝐷 that can distinguish between real and generated noise samples, specifically: The objective function is defined as: ; in, f and ϑ Respectively represent the adjustable parameters of the neural network in the discriminator 𝐷 and the generator 𝐺, D φ is the probability that the sample output is false.

[0008] As a preferred technical solution, the point cloud is generated by using random sampling consistency segmentation, specifically: Given a parametric model in space, the point cloud in the target space follows the following formula: axe + by + Czech = d ; in,( x , y , z ) represents the spatial Cartesian coordinates of the target object point cloud, ( a , b , c ) represents the space normal vector of the target plane, d Represents the distance from the origin of the reference coordinate system to the plane; Randomly select three points in the space as sampling points and substitute them into the parameterized model as follows: a 1 x + b 1 y + c 1 z= d 1 ; Randomly select another point in the space as the standard point and substitute it into the parameterized model as follows: a 1 x 2 + b 1 y 2 + c 1 z 2 = d 2 ; If the threshold distance r =| d 1 - d 2 | is less than 0.2, the corresponding point is judged as an internal point, that is, a point that satisfies the target model, otherwise it is an external point; After dividing the four points in the above space into inner points and outer points, record the number of inner points, then reselect three other random sampling points and execute the steps of judging inner points and outer points, set the number of iterations, and when the number of iterations is reached, take the model with the largest number of inner points as the optimal fit to the actual target point cloud model, and remove the redundant point clouds.

[0009] As a preferred technical solution, the point cloud filtering adopts Euclidean clustering denoising filtering, specifically: Construct a KD-Tree in the point cloud coordinate system and divide the remaining points in the space after random one-time segmentation. n Points are mounted in it; Initialize the point cloud and mark all points in the space as unvisited, ready for clustering; Traverse the point cloud, for each unvisited point, use KD-Tree to find its neighboring points and calculate the Euclidean distance between it and the neighboring points; for any two points in space c i and c j , use the following formula to calculate its Euclidean distance: ; In the above formula, d ( c i - c j )represent c i and c j The Euclidean distance of cix represent c i point x Axis coordinates, c iy represent c i point y Axis coordinates, c iz represent c i point z Axis coordinates, c j Similarly; Perform clustering and classification, traverse the entire KD-Tree to obtain n The Euclidean distance of neighboring points d ( c i - c j ), and compare it with the set threshold r Compare and mark the points whose Euclidean distance is less than the threshold as the same type of point cloud; After clustering is completed, the final optimized point cloud of the target to be grasped is obtained.

[0010] As a preferred technical solution, the grasping posture detection algorithm based on point cloud local concavity constraint and point cloud posture constraint is specifically: Taking the target point cloud as input, randomly sample multiple angles of the target point cloud to obtain multiple sets of candidate grasping poses; The force closure index, grasping spinor space, local concavity and convexity constraint of point cloud and point cloud pose constraint are used as the constraint conditions of grasping pose to perform pose screening. The force closure index is specifically: requiring the resultant force and resultant moment acting on the object to be zero, which is used to measure whether the grasping is stable; The grasping spinor space is a collection of force spinors of all contact points. Specifically, the force spinors of the contact points are: the force and torque applied by the contact points to the object when the object is grasped. The force and torque are combined into a force spinor. When the grasping spinor space can completely limit the degrees of freedom of movement of the object to be grasped, the movement of the object is constrained. The local concavity and convexity constraint of the point cloud uses the principal component analysis method to calculate the curvature of the point cloud, specifically: The captured point cloud obtained by point cloud registration M Any point within m i , in its neighborhood m i and its neighbors m ij Perform an analysis of covariance: ; ; C i is a 3*3 covariance matrix, l is the number of points in the neighborhood, is the center of the domain point cloud, and They are C i The eigenvalues ​​and eigenvectors of Assume that the eigenvalues ​​satisfy ,but for m i The normal vector for m i The neighborhood curvature x i ( m i )have: ; in, i =1,2,…, L , L Represents the number of sampling points; field curvature x i ( m i ) is used to measure the point cloud M The degree of curvature, x i The smaller it is, the flatter the neighborhood is. x i The larger the value, the greater the curvature and fluctuation of the neighborhood; For the surface point cloud grasped by the gripper, the local concavity constraint is as follows: F ≤ x i ( m i )≤ G ; in, F and G are constants, F Represents the lower limit of the curvature of the captured surface point cloud, G Represents the upper limit of the curvature of the grasped surface point cloud; The point cloud pose constraints are specifically: After point cloud registration, the target's position in three-dimensional space is obtained. The target position is represented by vector , , and origin V 0Indicates that the origin of the gripper coordinate system is a 0 , the three orthogonal vectors are used respectively , , express; The grab sampling condition equation is: ; in, m is a constant, t To set the parameters that determine the size of the capture range, a 1 , a 2 , a 3 are the three orthogonal vectors of the gripper , , The value of v 1 , v 2 , v 3 are the target pose vectors , , The value of The screened candidate grasping postures are input into the grasping evaluation network to score all grasping postures; Output the grasp pose with the highest score.

[0011] As a preferred technical solution, the dual-arm decoupling planning is specifically as follows: Set the initial posture and target posture of the master arm and slave arm; Perform path planning for the master arm or slave arm; Determine whether a collision occurs: If a collision occurs, the slave arm path is abandoned, and the master arm trajectory information is added to the slave arm status validity judgment, and the slave arm path planning is re-performed; If no collision occurs, the path planning of the master arm and the slave arm is completed.

[0012] As a preferred technical solution, the bidirectional search and the target point gravitational field are specifically as follows: Perform a bidirectional search and select the direction of tree update. Specifically, place the starting point and the end point in two different diffusion sets at the same time, and perform bidirectional transmission in each direction. Before each round of diffusion, determine the number of nodes in each tree and expand the tree with the smaller number of nodes until the two trees intersect. Select the initial state as the starting node; Randomly sample in the motion space to obtain random nodes q rand; Finding random nodes q rand The closest node q near ; Calculate initial orientation , where || ⋅ || is the norm in the configuration space, set to L1 or L2 norm; Under the influence of the gravitational field, a new growth direction is obtained d' , and determine the new node q' new , specifically: define the target point for any node q The gravitational function ,in k is the gravitational coefficient, q goal is the target point; the gravitational function is applied to q near Get new growth direction , and the new node ,in q new When the gravity function is not added q near Towards q rand The direction is based on the step length ∆ q Expanded nodes; judge q' new Is it an obstacle point: If it is an obstacle point, return to find a random node q rand The closest node q near If it is not an obstacle point, q' new Add to the node collection; connect q' new and q near , determine whether the connection collides: if a collision occurs, delete q' new And return to find the random node q rand The closest node q near If no collision occurs, determine whether the initial point is connected to the target point; If no node connection appears, return to randomly sampling in the motion space to obtain a random node q rand If a node connection occurs, a connection path is generated and path planning ends.

[0013] As a preferred technical solution, the movement of the robot in the working environment is planned based on the reward and punishment mechanism, specifically: In the action space, the robot can choose a limited and discrete speed, ranging from the interval [0, V max ] are uniformly selected, the heading angle Ω is uniformly distributed in [0,2π], and the robot running speed in space is decomposed along the Cartesian coordinate system into [ V x , V y ], P g , P , are the current spatial positions of the robot, target point, and pedestrian, respectively. Their spatial positions are decomposed into the X and Y axes in the Cartesian coordinate system. The running reward function is set as: S = S pass + S cross + S over ; ; in, S pass Avoid running reward functions that exceed the phenomenon. S cross To avoid the running reward function of the crossing phenomenon, S over To avoid running reward functions that cause pass-by, is the distance between the robot and the target, and the distance between the robot and the pedestrian is , the relative rotation angle between the two robots when they meet is , [ V x , V y ]、[ V x ' , V y ' ] are the current velocities of the two mobile dual-arm robots in space in Cartesian coordinates, and the angle |Θ-Ω| is limited to [-π, π].

[0014] Another aspect of the present invention further provides an optimization system for object recognition and obstacle avoidance grasping of a dual-arm robot, which is applied to the above-mentioned optimization method for object recognition and obstacle avoidance grasping of a dual-arm robot, including a diffusion network image restoration module, a point cloud segmentation and filtering module, a registration recognition module, a grasping posture detection module, a dual-arm motion path planning module and a robot movement planning module; The diffusion network image restoration module is used to collect the image of the object to be captured and restore it based on the diffusion network and the image generation network, generate a point cloud for the restored image, and build a point cloud model library of the target object; The point cloud segmentation and filtering module is used to generate a point cloud for the real-time collected image of the object to be captured and perform point cloud segmentation and point cloud filtering to obtain point cloud information after filtering optimization; The registration and recognition module is used to perform point cloud registration on the point cloud information after filtering and optimization with the point cloud model library of the target object to determine whether it is an object to be grasped; The grasping posture detection module is used to obtain the optimal grasping posture by adopting a grasping posture detection algorithm based on local concavity and convexity constraints of point clouds and point cloud posture constraints; The dual-arm motion path planning module is used to plan the motion path of the robot's dual arms according to dual-arm decoupling planning, bidirectional search, and a path planning algorithm of a target point gravitational field; The robot movement planning module is used to plan the movement of the robot in the working environment according to the reward and punishment mechanism.

[0015] Compared with the prior art, the present invention has the following advantages and beneficial effects: (1) The present invention adopts a diffusion image generation network, which improves the image training efficiency, helps to improve the restoration performance of the generative adversarial network, improves the situation where the grasping network mistakes the edge of the object as the contour of the graspable object, improves the training effect of blurred images, and helps with subsequent point cloud matching, so as to improve the generalization ability of the robot arm to grasp different objects.

[0016] (2) The present invention uses point cloud filtering and improved posture constraints to help the robotic arm effectively match the target object and grasp the object with a reasonable grasping posture. In complex occluded environments, it helps it effectively identify the target grasped object and perform grasping operations, thereby improving the success rate of the dual-arm robot in handling object grasping tasks.

[0017] (3) The present invention adopts a target point-guided spatial motion planning algorithm and a dual-arm coordinated motion planning criterion to improve the efficiency of the dual-arm robot in spatial motion planning, enabling it to effectively plan paths in a complex environment in a shorter time and avoid self-collision of the dual arms or collision of the arm ends with obstacles in the spatial environment.

[0018] (4) The present invention adopts an operation penalty function to ensure that the robot can avoid collisions with pedestrians or other mobile robots when operating in complex environments, enhance the reliability of the robot in different scenarios, and improve the safety and flexibility of the dual-arm robot's spatial movement. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is a flow chart of an optimization method for object recognition and obstacle avoidance grasping of a dual-arm robot according to an embodiment of the present invention; Figure 2 It is a schematic diagram of the structure of the traditional image generation model; Figure 3 is a schematic diagram of the structure of a diffusion image generation network according to an embodiment of the present invention; Figure 4 is a tree diagram of an improved robot arm motion planning algorithm according to an embodiment of the present invention; Figure 5 is a schematic diagram of the motion planning process of a robotic arm according to an embodiment of the present invention; Figure 6 is a flow chart of the dual-arm coordinated motion planning criteria of an embodiment of the present invention; Figure 7 is a robot operation safety criterion diagram according to an embodiment of the present invention; Figure 8 It is a structural schematic diagram of an optimization system for object recognition and obstacle avoidance grasping of a dual-arm robot according to an embodiment of the present invention. DETAILED DESCRIPTION

[0020] In order to enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present application.

[0021] Example: like Figure 1 As shown, this embodiment provides an optimization method for object recognition and obstacle avoidance grasping of a dual-arm robot, comprising the following steps: S1. Collect the image of the object to be grasped and repair it based on the diffusion network and image generation network, generate point cloud for the repaired image, and build a point cloud model library of the target object.

[0022] A reliable grasping data set is helpful to help the robot arm to successfully identify the object to be grasped in the application environment. The data set of conventional objects in life has been made relatively complete, but with the increase of robot application environments, the grasping of irregular objects is also increasing, such as the grasping demand for composite workpieces in industrial applications. However, edge texture phenomenon is prone to appear in the image of irregular objects, that is, the boundary between the object and the background is not obvious and clear; the reason for its occurrence is mostly due to the uneven shooting lighting problem caused by the geometric shape of the irregular object or the uneven surface of the workpiece, which will interfere with the correct detection of the edge of the object by the robot arm grasping algorithm during grasping. Therefore, this embodiment proposes a method for image enhancement of the data set using a diffusion network, by introducing a diffusion model into an image generator, the image generated by the generator and the real image undergo a forward diffusion process, and a repaired image with Gaussian noise and a real image are obtained, which are used as the input of the image discrimination module, and the image judged as true is obtained by cyclic iteration as the final output repaired image.

[0023] 1.1. Traditional image generation model; Traditional image generation models, such as the GAN model, aim to learn the data distribution of the target dataset by constructing an image generator and an image discriminator in two neural network modules and establishing a minimum-maximum game between the two modules. p ( x ). The generator 𝐺 will be drawn from a simple prior distribution such as the standard normal or uniform distribution p ( z ) is a random noise vector sampled from z as input and attempts to generate pseudo samples of the original data G ( z ). The input of the discriminator 𝐷 is from p ( x ) The real data sample 𝑥 extracted or the pseudo sample generated by 𝐺 G ( z ), and try to correctly classify them as real samples or fake samples. The goal of 𝐺 is to deceive 𝐷 to make it wrongly judged, while the goal of 𝐷 is to accurately distinguish G ( z ) and 𝑥. The structure of the generated adversarial network is as follows Figure 2 shown.

[0024] 1.2. Diffusion Image Generation Network; First, through the traditional generator network 𝐺, from the initial distribution p ( z ) is mapped to a high-dimensional data space to generate samples close to the real x g . Generator Sample x g =G ( z ), z~p ( z ) is expressed as To make the generator more robust, noise is injected into the generated samples through a diffusion process that adds Gaussian noise at each step. x g The diffusion process can be regarded as a Markov chain, starting from the initial sample 𝑥, through T After 𝑇 steps, it gradually becomes a noisy image sample. The size of the number of steps 𝑇 is affected by β t constraint: .

[0025] In this embodiment, the random interference noise sample 𝑦 added in any step of the diffusion process is defined as a mixed distribution q ( y | x ). q ( y | x , t ) is a Gaussian distribution with a mean proportional to 𝑥 and a variance depending on the noise level at the current step 𝑡. For real samples x~p ( x ) and generate samples x g ~p ( x g ) uses the same diffusion process and mixed distribution. After adding the diffusion process, the mixed distribution of the real image and the generated image are expressed as: (1); in, q ( y | x ) is the mixed distribution in the 𝑇th iteration, and the mixed component obtained by diffusion q ( y | x , t ) is expressed as follows, where a t =1- β t , , I It is the threshold parameter of the distribution and can be adjusted to a reasonable range manually.

[0026] (2); Sampling 𝑦 from the mixed distribution can obtain real samples and noisy versions of generated samples with different degrees of noise. The more steps taken in the diffusion process, the more noise is added to 𝑦, and the less information is retained from 𝑥. Then use this mixed distribution obtained by the diffusion process to train a discriminator 𝐷 that can distinguish between real and generated noise samples, and iterate the images judged as fake by the discriminator 𝐷 to repair them cyclically, and output the images judged as real as the repaired images. The diffusion image generation network framework is as follows Figure 3 shown.

[0027] The objective function of the diffusion generative adversarial network is defined as follows, where f and ϑ Represent the adjustable parameters of the neural network in the discriminator and generator, respectively. D φ is the probability that the sample output is false.

[0028] (3); The training goal of the objective function is to maximize the discriminator's ability to distinguish true from false, that is, in any diffusion step 𝑡, a high probability is assigned to the real image after adding noise, and a low probability is assigned to the generated image after adding noise. The goal of the generator is to generate samples that deceive the discriminator to the greatest extent in any diffusion step 𝑡.

[0029] 𝑝(𝑥) is the distribution of real pictures; q ( y | x , t ) is the conditional distribution of the given original data 𝑥 and the noise sample 𝑦 with a diffusion step size of 𝑡. According to Gaussian redistribution, y Simplified to , σ is the scaling factor, is a Gaussian noise distribution. Similarly, because the diffusion process generates samples , you can y g Rewrite .

[0030] By introducing the diffusion model into the image generation network, the model can focus on the authenticity judgment of the image itself, while ignoring its edge texture and structural information, thereby improving the restoration performance of the generative adversarial network; secondly, this method can distinguish the contours of objects, making it convenient to generate a relatively complete RGB image for each complex workpiece information, which can be used to design a point cloud model library for subsequent spatial object capture.

[0031] S2. Generate a point cloud for the real-time collected image of the object to be grasped and perform point cloud segmentation and point cloud filtering to obtain point cloud information after filtering optimization.

[0032] At present, grasping detection based on 2D images has been widely used in visual grasping of robotic arms, but a single 2D image cannot fully reflect the three-dimensional spatial information of the target to be grasped, while grasping detection using 3D point clouds can directly obtain the three-dimensional spatial information of the target object. Therefore, after obtaining the spatial information of the grasping target, how to select a reasonable grasping posture is the difficulty of current research. Therefore, in this patent, it is proposed to first segment the object data collected by the depth camera, and design a point cloud database of the target object based on the adversarial image generation network mentioned above. When the robotic arm grasps, the corresponding object is segmented in the image, and the point cloud is filtered to obtain the point cloud information after filtering and optimization, and the point cloud is aligned with the database to determine whether it is the object to be grasped, and then the improved grasping posture detection algorithm is used to obtain the gripper spatial posture and stable grasping posture that are more in line with the force analysis. The overall flow chart is as follows Figure 1 shown.

[0033] 2.1. Point cloud segmentation and registration method.

[0034] 2.1.1. Random sampling and one-time segmentation.

[0035] The random sampling consensus algorithm is a method that uses an iterative approach to estimate model parameters from a set of discrete data. First, a parameterized model in space is designed. The point cloud in the target space follows the following model formula: axe + by + Czech = d (4); In the above formula ( x , y , z ) represents the spatial Cartesian coordinates of the target object point cloud, ( a , b , c ) represents the space normal vector of the target plane, d Represents the distance from the origin of the reference coordinate system to the plane. The random sampling consistency segmentation algorithm assumes that the point cloud distribution in space includes internal points and external points. The internal points are points that meet the target model, and the external points are opposite. First, three points in the space are randomly selected (at least three points are required to determine the plane), and these three points are substituted into the parameterized model as sampling points. The equation can be obtained: a 1 x + b 1 y + c 1 z = d 1 (5); Randomly select another point in the space as the standard point and substitute it into the model to get a 1 x 2 + b 1 y 2 + c 1 z 2 = d 2 , through the distance threshold r To determine whether the point is an interior point: r =| d 1 - d 2 | (6); If the distance threshold r If it is less than 0.2, the point is judged as an inner point. After dividing the four points in the above space into inner points and outer points, the number of inner points is recorded, and then another three random sampling points are reselected and the steps of judging inner points and outer points are executed. The number of iterations is set. After the number of iterations is reached, the model with the largest number of inner points is considered to be the optimal fit to the actual target point cloud model, and the redundant point clouds are removed.

[0036] 2.1.2. Euclidean clustering denoising filter.

[0037] Using random one-time segmentation can effectively segment the target object, but there are still some discrete point clouds. In order to further clean up these noises and extract the point cloud of the target object, the Euclidean clustering denoising filter is used to classify points with similar distances as the same category. It can separate the point cloud of the target object from other noise in the space.

[0038] First, a KD-Tree is constructed in the point cloud coordinate system and the remaining points in the space after random one-time segmentation are n Points are mounted in it, which can efficiently manage points in space and speed up the search process of neighboring points. Then, the point cloud is initialized and all points in space are marked as unvisited, ready for clustering. Next, the point cloud is traversed, and for each unvisited point, the KD-Tree is used to find its neighboring points and calculate the Euclidean distance between it and the neighboring points. For any two points in space c i and c j , use the following formula to calculate its Euclidean distance: (7); In the above formula, d ( ci - c j )represent c i and c j The Euclidean distance of c ix represent c i point x Axis coordinates, c iy represent c i point y Axis coordinates, c iz represent c i point z Axis coordinates, c j Afterwards, clustering and classification are performed, and the entire KD-Tree is traversed to obtain n The Euclidean distance of neighboring points d ( c i - c j ), and compare it with the threshold r Comparison, Threshold r The value can be manually adjusted and set to mark points with a Euclidean distance less than the threshold as the same type of point cloud. Finally, after clustering, the final optimized point cloud of the target to be grasped is obtained.

[0039] S3, performing point cloud registration on the filtered and optimized point cloud information and the target object point cloud model library to determine whether it is an object to be grasped.

[0040] S4. After identifying the object to be grasped, a grasping posture detection algorithm based on the local concavity and convexity constraints of the point cloud and the point cloud posture constraints is used to obtain the optimal grasping posture.

[0041] Traditional posture detection algorithms often use deep learning to detect grasping postures, so that they can predict the optimal grasping posture from a set of 3D point cloud data. This algorithm uses the target point cloud as input. For irregular objects, there may be grasping errors when using a single perspective. Therefore, multiple groups of candidate grasping postures are obtained by random sampling at multiple angles of the target point cloud. Then, force closure indicators and grasping spinor space are used as constraints for grasping postures to screen postures. The screened candidate grasping postures are input into the grasping evaluation network, which can score all grasping postures and finally output the optimal grasping posture.

[0042] The force closure index is specifically: if an object will not fall out of the robot's hand after it is grasped, no matter what direction the external force is applied to it, then this grasp is called force closure. Force closure requires that the resultant force and torque acting on the object are zero. When grasping an object, the force closure index is used to measure whether the grasp is stable, that is, whether the force applied to the object by the fingers or the grasping tool can completely constrain the movement of the object. For example, in a simple plane grasping scenario, use two fingers to clamp an object. If the force applied by the fingers can balance the gravity of the object and any possible external force interference, so that the object has no tendency to translate or rotate in the plane, force closure is achieved.

[0043] The grasping spinor space is specifically: a space composed of force spinors at all possible contact points. When an object is grasped, each contact point will exert a force and a torque on the object, and these forces and torques can be combined into a force spinor. The force spinors of all contact points are combined to form a grasping spinor space. Taking the above-mentioned scenario of two fingers clamping an object as an example, each contact point between the finger and the object will generate a force spinor, and the space formed by these two force spinors is the grasping spinor space. As long as the grasping spinor space can completely limit the freedom of movement of the object to be grasped, the movement of the object can be constrained.

[0044] The grasp scoring network usually uses only two indicators to judge the stability of candidate grasps, without using the local surface information and global posture information of the point cloud, resulting in a large number of low-quality grasps, affecting the execution efficiency and accuracy of the algorithm. Therefore, this patent adds point cloud local concavity constraints and point cloud posture constraints on the basis of traditional methods to improve the grasp generation constraints.

[0045] The local concavity and convexity constraint of the point cloud can be realized by analyzing the curvature of the point cloud in the local coordinate system of the candidate grasp. For calculating the curvature of the point cloud, the principal component analysis method can be used. M Any point within m i , in its neighborhood m i and its neighbors m ij Perform an analysis of covariance: (8); (9); C i is a 3*3 covariance matrix, l is the number of points in the neighborhood, is the center of the domain point cloud, and They are C i The eigenvalues ​​and eigenvectors of . Assume that the eigenvalues ​​satisfy ,but for m i The normal vector for m i The neighborhood curvature x i ( m i )have: (10); in, i =1,2,…, L , L Represents the number of sampling points. The field curvature can measure the point cloud M The degree of curvature, x i The smaller it is, the flatter the neighborhood is. x i The larger the value, the greater the curvature of the neighborhood. For the surface point cloud grasped by the gripper, this embodiment samples the following constraints as its local concavity constraints: F ≤ x i ( m i )≤ G (11); in, F and G are constants, F Represents the lower limit of the curvature of the captured surface point cloud, G Represents the upper limit of the curvature of the grasped surface point cloud. These two values ​​can be adjusted according to the actual grasping effect.

[0046] In addition to using the local point cloud of the target surface to be grasped, the overall pose of the point cloud can also be used to screen candidate grasps, that is, point cloud pose constraints. After point cloud registration, the pose of the target in three-dimensional space is obtained, and the target pose can be expressed by vector , , and origin V 0 Indicates that the origin of the gripper coordinate system is a 0 , the three orthogonal vectors are used respectively , , express.

[0047] Taking the two-finger gripper at the end of the robot arm as an example, its position with the target object should satisfy the following constraints: first, the two-finger gripper should be distributed on both sides of the main axis. Second, the two-finger gripper should be parallel to the surface of the target to be grasped, that is, perpendicular to the surface normal of the target. Finally, the two-finger gripper should be as close to the center of mass of the object as possible when grasping the object for stability. Assuming that the mass is uniform, the center point of the target can be used instead of the center of mass.

[0048] Taking the above three constraints as the conditions for point cloud pose constraints, the grab sampling condition equation can be generated as follows: (12); m is a constant, a 1 , a 2 , a 3 are the three orthogonal vectors of the gripper , , The value of v 1 , v 2 , v 3 are the target pose vectors , , The value of t The parameters that determine the size of the capture range can be adjusted according to actual conditions. t When it is small, the grasp will be sampled in the area closer to the center of mass, the grasp is more stable but the grasping range is smaller; t When it is larger, the grasp will sample areas farther away from the center of mass, and the grasp range will be larger.

[0049] S5. Plan the motion path of the robot's dual arms based on dual-arm decoupling planning, bidirectional search, and a path planning algorithm based on the target point gravitational field.

[0050] 5.1. Traditional robotic arm motion planning algorithm.

[0051] For complex systems with multiple degrees of freedom such as robotic arms, if path planning is only performed in the workspace, it can only ensure that the end effector of the robotic arm avoids obstacles, but it cannot ensure that the robotic arm itself can also avoid obstacles. Therefore, scholars in the past introduced the concept of C space (configuration space) in the robotic arm motion space, that is, the space related to the robot configuration. All robots can be regarded as point robots in C space, thus converting the path planning problem of robots with complex configurations in the workspace into the path planning problem of a point in C space.

[0052] The RRT algorithm is a common path planning algorithm at present. Its search process is as follows: (1) Given a starting point q start , target point q goal and algorithm step length ∆ q , generate a random point in C space q rand1 .

[0053] (2) q rand1 The vector direction formed by the parent node is the direction of the first expansion of the random tree. q start As the starting point, ∆ q Expand the first child node by the step size q new1 .

[0054] (3) For nodes q new1 Check the validity of the state. If it is determined to be a valid point, the node q new1 Add to the node set and edge set of the random tree; if it fails, discard it, reselect the random point and repeat the above process.

[0055] (4) Generate another random point in space q rand2 .

[0056] (5) From the previously generated nodes, select q rand2 The nearest point, such as q new1 ,exist arrive q new2 in the direction of q new1 As the starting point, ∆ q Expand the second child node for the step size q new2 .

[0057] (6) Repeat the above steps until the distance between the generated new node and the target point is less than ∆ q The search ends when .

[0058] (7) From q goal Start to trace back upwards, searching according to the corresponding relationship between parent nodes and child nodes until the parent node is found. q startThen, arrange the found nodes in order to get a collision-free path from the starting point to the target point.

[0059] Traditional algorithms continuously expand the tree structure by randomly sampling in the sampling space. However, since the nodes guiding the growth of the random tree are generated uniformly and randomly in the space and it is a one-way search, the sampling points will be spread throughout the entire sampling space. The random tree spends a considerable amount of time growing in a direction away from the target point. In high-dimensional spaces or in environments with many obstacles, the search efficiency will be greatly reduced.

[0060] 5.2. Improve the robot arm motion planning algorithm.

[0061] In response to the RRT search problem in 5.1, this embodiment proposes a dual-arm global bidirectional planning algorithm combined with a target point gravitational force field. The specific algorithm flow is introduced below.

[0062] like Figure 4 As shown in the figure, bidirectional search means that during the search process, the starting point and the end point are placed in two different diffusion sets at the same time, and bidirectional transmission is performed in each direction. Before each round of diffusion, the number of nodes in each tree is determined, and the tree with a smaller number of nodes is expanded until the two trees intersect, that is, a complete trajectory curve is generated.

[0063] like Figure 5 As shown, the diffusion of the objective function is explained with the starting point as the tree starting direction. In the diffusion book of the starting point, q start is the initial node, q goal is the target node, q rand is a random node generated by the random tree in the configuration space, and its parent node is q near , when no gravitational function is added, q near Will be towards q rand The direction is based on the step length ∆ q Expand a new node q new , its growth direction is: (13); Where || ⋅ || is the norm in the configuration space, which can be set to L1 or L2 norm. q goal A gravitational field is set at , and the gravitational coefficient is k , its value is customizable, q goal exist q near The gravitational potential energy atU and q goal exist q near The gravitational force is: (14); (15); Then we get the target point for any node q The gravitational function of (16); Apply the gravitational function to q near On, so that q near The point will be affected by the gravitational force from the target point, and will be biased to a certain extent. q goal , and thus obtain new growth directions d' and the new node q' new : (17); (18); because F ( q ), the random tree will tend to grow towards the target direction. If you want to change the gravity of the target point, you only need to change the gravity coefficient. k When k When is larger, the target node has a greater attraction on the random tree, guiding the random tree to grow more towards the target point; k When it is smaller, the growth of the random tree is more divergent and tends to expand to the surrounding environment, which is conducive to avoiding obstacles and searching for accessible paths. The size of the coefficient is adjusted k To ensure that the algorithm q goal No oscillation occurs near the end point. By introducing the gravitational function field and bidirectional search strategy based on the RRT algorithm, the generation of random trees is accelerated. The end point is the starting direction of the tree. Similarly, under the action of gravity q' near Generate orientation q start The intermediate tree node q'' new , as the search tree is updated they approach each other until they are connected.

[0064] like Figure 6As shown in the figure, in order to avoid the dimensional disaster caused by centralized planning, the coordinated motion of the two arms is planned in a decoupled manner. The planning process is shown in the figure below. Now it is stipulated that one of the two arms is the main arm and the other arm is the slave arm. Here, the left arm is the main arm and the right arm is the slave arm. First, the improved RRT algorithm is used to plan the path of the main arm. When planning the path, the main arm does not consider the state of the slave arm, but only considers the static obstacles in the environment. After the main arm plans a feasible path, it starts to plan the path of the slave arm. When the slave arm first plans the path, it also only considers the obstacles in the environment, and can obtain a collision-free path for the slave arm. At this time, the path points of the two paths are given time information, and collision detection is performed for the path points at the same time. If there is no collision point between the two robotic arms after the detection, the motion planning process ends; if a collision is detected at some points in the middle, the path of the slave arm will be discarded, and the trajectory information of the main arm will be added to the collision library of the slave arm, and the path of the slave arm will be replanned until the slave arm plans a path that does not collide with the main arm and obstacles in the environment.

[0065] In order to prove the effectiveness of the above guidance algorithm, a simulation environment of a dual-arm robot was constructed in the software Pybullet. Ten obstacles were randomly generated as environment one, and thirty obstacles were randomly generated as environment two. A spatial Cartesian coordinate point was given as the target point. The obstacle avoidance ability of the dual arms moving to the target point was tested. The controller of the same algorithm was run ten times continuously, and the average time and error cost were taken as evaluation indicators. The constructed environment and test result data are shown below: ; Table 1. Experimental data of environment 1.

[0066] ; Table 2. Experimental data of environment 2.

[0067] In the above results, RRT (attractive) represents the gravity-optimized RRT algorithm, while RRT (attractive-dual) represents the gravity-optimized RRT algorithm combined with the dual-arm coordination method for optimization. From the above results, it can be seen that the proposed improved algorithm RRT (attractive-dual) has the lowest time consumption and error in a variety of complex environments, and as the number of obstacles in the complex environment increases, the optimization of the algorithm is gradually improved compared with the traditional RRT algorithm. For example, in environment 1, the average motion time of RRT and RRT (attractive) is the same, but in environment 2 where obstacles increase, the time required for RRT (attractive) is significantly shorter than that of the traditional RRT algorithm.

[0068] S6. Plan the robot's multi-range movement in the factory based on a reward and punishment mechanism.

[0069] In industrial applications, dual-arm robots are often equipped with a mobile chassis for multi-range movement in factories. In order to avoid collisions, dual-arm robots need to follow simple movement rules. Although the rules are intuitive, they are difficult to quantify accurately. For example, when a robot passes a pedestrian, when and at what angle it turns is highly random, making the problem more complicated to handle. Therefore, a reward and punishment mechanism is used to induce the robot to follow the factory's operating habits, which facilitates the regulation of robot operation safety.

[0070] In the action space, the robot can choose a limited and discrete speed. The speed ranges from [0, V max ] are uniformly selected, the heading angle Ω is uniformly distributed in [0,2π], and the robot running speed in space can be decomposed along the Cartesian coordinate system as [ V x , V y ], P g , P , are the current spatial positions of the robot, target point, and pedestrian respectively. Their spatial positions can be decomposed into the X and Y axes in the Cartesian coordinate system. The running reward function is set as: S = S pass + S cross + S over (19); (20); In the above formula, S pass Avoid running reward functions that exceed the phenomenon. S cross To avoid the running reward function of the crossing phenomenon, S over To avoid running reward functions that cause pass-by, is the distance between the robot and the target, and the distance between the robot and the pedestrian is , the relative rotation angle between the two robots when they meet is , [ V x , V y ]、[ V x ' , V y' ] are the current velocities of the two mobile dual-arm robots in space in Cartesian coordinates, and the angle |Θ-Ω| is limited to [-π, π]. The penalty set is shown in the figure below: Figure 7 As shown in the figure, the robot and pedestrian are divided into three behaviors according to their relative positions and speeds, as well as their target directions. If the robot appears in the shaded area, it will be penalized, corresponding to the three situations of overtaking, passing by, and crossing.

[0071] like Figure 8 As shown, in another embodiment of the present application, a dual-arm robot object recognition and obstacle avoidance grasping optimization system is provided, the system includes a diffusion network image restoration module, a point cloud segmentation and filtering module, a registration recognition module, a grasping posture detection module, a dual-arm motion path planning module and a robot movement planning module; The diffusion network image restoration module is used to collect the image of the object to be captured and restore it based on the diffusion network and the image generation network, generate a point cloud for the restored image, and build a point cloud model library of the target object; The point cloud segmentation and filtering module is used to generate a point cloud for the real-time collected image of the object to be captured and perform point cloud segmentation and point cloud filtering to obtain point cloud information after filtering optimization; The registration and recognition module is used to perform point cloud registration on the point cloud information after filtering and optimization with the point cloud model library of the target object to determine whether it is an object to be grasped; The grasping posture detection module is used to obtain the optimal grasping posture by adopting a grasping posture detection algorithm based on local concavity and convexity constraints of point clouds and point cloud posture constraints; The dual-arm motion path planning module is used to plan the motion path of the robot's dual arms according to dual-arm decoupling planning, bidirectional search, and a path planning algorithm of a target point gravitational field; The robot movement planning module is used to plan the movement of the robot in the working environment according to the reward and punishment mechanism.

[0072] It should be noted here that the system provided in the above embodiment is only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure is divided into different functional modules to complete all or part of the functions described above. The system is an optimization method for object recognition and obstacle avoidance grasping of a dual-arm robot applied to the above embodiment.

[0073] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0074] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be equivalent replacement methods and are included in the protection scope of the present invention.

Claims

1. An optimization method for object recognition and obstacle avoidance grasping of a dual-arm robot, characterized in that: The steps include: Collect the image of the object to be grasped and repair it based on the diffusion network and image generation network, generate point cloud for the repaired image, and build a point cloud model library of the target object; Generate point cloud for the real-time captured image of the object to be grasped, perform point cloud segmentation and point cloud filtering, and obtain point cloud information after filtering optimization; Performing point cloud registration on the filtered and optimized point cloud information and the target object point cloud model library to determine whether it is an object to be grasped; After identifying the object to be grasped, a grasping posture detection algorithm based on local concavity and convexity constraints of point cloud and point cloud posture constraints is used to obtain the optimal grasping posture; The motion path of the robot's dual arms is planned based on dual-arm decoupling planning, bidirectional search, and the path planning algorithm of the target point gravitational field; Plan the robot's movement in the working environment based on the reward and punishment mechanism.

2. The optimization method for object recognition and obstacle avoidance grasping of a dual-arm robot according to claim 1, characterized in that: The repair based on the diffusion network and the image generation network is specifically as follows: Through the generator 𝐺, from the initial distribution p ( z ) is mapped to the high-dimensional data space to obtain the generator sample x g = G ( z ), z~p ( z ), the distribution is expressed as ; A real dataset obtained from collecting images of objects to be grasped p ( x ) to extract the real data sample 𝑥, x~p ( x ); diffusion Process: At each step, the generator samples x g Add Gaussian noise to the real data sample 𝑥, and get the noisy image sample after 𝑇 steps, specifically: ； in, q ( y | x ) is the mixed distribution in the 𝑇th iteration; 𝑦 is the added random interference noise sample; the size of the step number 𝑇 is affected by β t constraint: ; Mixture components obtained by diffusion q ( y | x , t ) is a Gaussian distribution whose mean is proportional to 𝑥 and whose variance depends on the noise level at the current step 𝑡, expressed as follows: ； in, a t =1- β t , , I is the threshold parameter of the set distribution; The mixed distribution obtained by the diffusion process is used to train a discriminator 𝐷 that can distinguish between real and generated noise samples, and the image judged as true by the discriminator 𝐷 is output as a repaired image.

3. The optimization method for object recognition and obstacle avoidance grasping of a dual-arm robot according to claim 2, characterized in that: The mixed distribution obtained by the diffusion process is used to train the discriminator 𝐷 that can distinguish between real and generated noise samples, specifically: The objective function is defined as: ； in, φ and ϑ Respectively represent the adjustable parameters of the neural network in the discriminator 𝐷 and the generator 𝐺, D φ is the probability that the sample output is false.

4. The optimization method for object recognition and obstacle avoidance grasping of a dual-arm robot according to claim 1, characterized in that: The generated point cloud adopts random sampling consistency segmentation, specifically: Given a parametric model in space, the point cloud in the target space follows the following formula: ax + by + cz = d ； in,( x , y , z ) represents the spatial Cartesian coordinates of the target object point cloud, ( a , b , c ) represents the space normal vector of the target plane, d Represents the distance from the origin of the reference coordinate system to the plane; Randomly select three points in the space as sampling points and substitute them into the parameterized model as follows: a 1 x + b 1 y + c 1 z = d 1； Randomly select another point in the space as the standard point and substitute it into the parameterized model as follows: a 1 x 2+ b 1 y 2+ c 1 z 2= d 2； If the threshold distance ρ =| d 1- d 2| is less than 0.2, the corresponding point is judged as an internal point, that is, a point that satisfies the target model, otherwise it is an external point; After dividing the four points in the above space into inner points and outer points, record the number of inner points, then reselect three other random sampling points and execute the steps of judging inner points and outer points, set the number of iterations, and when the number of iterations is reached, take the model with the largest number of inner points as the optimal fit to the actual target point cloud model, and remove the redundant point clouds.

5. The optimization method for object recognition and obstacle avoidance grasping of a dual-arm robot according to claim 1, characterized in that: The point cloud filtering adopts Euclidean clustering denoising filtering, specifically: Construct a KD-Tree in the point cloud coordinate system and divide the remaining points in the space after random one-time segmentation. n Points are mounted in it; Initialize the point cloud and mark all points in the space as unvisited, ready for clustering; Traverse the point cloud, for each unvisited point, use KD-Tree to find its neighboring points and calculate the Euclidean distance between it and the neighboring points; for any two points in space γ i and γ j , use the following formula to calculate its Euclidean distance: ； In the above formula, d ( γ i - γ j )represent γ i and γ j The Euclidean distance of γ ix represent γ i point x Axis coordinates, γ iy represent γ i point y Axis coordinates, γ iz represent γ i point z Axis coordinates, γ j Similarly; Perform clustering and classification, traverse the entire KD-Tree to obtain n The Euclidean distance of neighboring points d ( γ i - γ j ), and compare it with the set threshold r Compare and mark the points whose Euclidean distance is less than the threshold as the same type of point cloud; After clustering is completed, the final optimized point cloud of the target to be grasped is obtained.

6. The optimization method for object recognition and obstacle avoidance grasping of a dual-arm robot according to claim 1, characterized in that: The grasping posture detection algorithm based on the local concavity and convexity constraint of the point cloud and the point cloud posture constraint is specifically as follows: Taking the target point cloud as input, randomly sample multiple groups of candidate grasping poses at multiple angles of the target point cloud; The force closure index, grasping spinor space, local concavity and convexity constraint of point cloud and point cloud pose constraint are used as the constraint conditions of grasping pose to perform pose screening. The force closure index is specifically: requiring the resultant force and resultant moment acting on the object to be zero, which is used to measure whether the grasping is stable; The grasping spinor space is a collection of force spinors of all contact points. Specifically, the force spinors of the contact points are: the force and torque applied by the contact points to the object when the object is grasped. The force and torque are combined into a force spinor. When the grasping spinor space can completely limit the degrees of freedom of movement of the object to be grasped, the movement of the object is constrained. The local concavity and convexity constraint of the point cloud uses the principal component analysis method to calculate the curvature of the point cloud, specifically: The captured point cloud obtained by point cloud registration M Any point within m i , in its neighborhood m i and its neighbors m ij Perform an analysis of covariance: ； ； C i is a 3*3 covariance matrix, l is the number of points in the neighborhood, is the center of the domain point cloud, and They are C i The eigenvalues ​​and eigenvectors of Assume that the eigenvalues ​​satisfy ,but for m i The normal vector for m i The neighborhood curvature have: ； in, i =1,2,…, L , L Represents the number of sampling points; field curvature For measuring point clouds M The degree of curvature, The smaller it is, the flatter the neighborhood is. The larger the value, the greater the curvature of the neighborhood. For the surface point cloud grasped by the gripper, the local concavity constraint is as follows: F ≤ ≤ G ； in, F and G are constants, F Represents the lower limit of the curvature of the captured surface point cloud, G Represents the upper limit of the curvature of the grasped surface point cloud; The point cloud pose constraints are specifically: After point cloud registration, the target's position in three-dimensional space is obtained. The target position is represented by vector , , and origin V 0 means the origin of the gripper coordinate system a 0, the three orthogonal vectors are used respectively , , express; The grab sampling condition equation is: ； in, μ is a constant, τ To set the parameters that determine the size of the capture range, a 1. a 2. a 3 are the three orthogonal vectors of the gripper , , The value of v 1. v 2. v 3 are the target pose vectors , , The value of The screened candidate grasping postures are input into the grasping evaluation network to score all grasping postures; Output the grasp pose with the highest score.

7. The optimization method for object recognition and obstacle avoidance grasping of a dual-arm robot according to claim 1, characterized in that: The specific dual-arm decoupling plan is: Set the initial posture and target posture of the master arm and slave arm; Perform path planning for the master arm or slave arm; Determine whether a collision occurs: If a collision occurs, the slave arm path is abandoned, and the master arm trajectory information is added to the slave arm status validity judgment, and the slave arm path planning is re-performed; If no collision occurs, the path planning of the master arm and the slave arm is completed.

8. The optimization method for object recognition and obstacle avoidance grasping of a dual-arm robot according to claim 7, characterized in that: The bidirectional search and target point gravitational field are as follows: Perform a bidirectional search and select the direction of tree update. Specifically, place the starting point and the end point in two different diffusion sets at the same time, and perform bidirectional transmission in each direction. Before each round of diffusion, determine the number of nodes in each tree and expand the tree with the smaller number of nodes until the two trees intersect. Select the initial state as the starting node; Randomly sample in the motion space to obtain random nodes q rand ; Finding random nodes q rand The closest node q near ; Calculate initial orientation , where || ⋅ || is the norm in the configuration space, set to L1 or L2 norm; Under the influence of the gravitational field, a new growth direction is obtained d' , and determine the new node q' new , specifically: define the target point for any node q The gravitational function ,in k is the gravitational coefficient, q goal is the target point; Apply the gravitational function to q near Get new growth direction , and the new node ,in q new When the gravity function is not added q near Towards q rand The direction is based on the step length ∆ q Expanded nodes; judge q' new Is it an obstacle point: If it is an obstacle point, return to find a random node q rand The closest node q near steps; such as Not an obstacle point, q' new Add to the node collection; connect q' new and q near , determine whether the connection collides: if a collision occurs, delete q' new And return to find the random node q rand The closest node q near If no collision occurs, determine whether the initial point is connected to the target point; If no node connection appears, return to randomly sampling in the motion space to obtain a random node q rand Steps; If a node connection occurs, a connection path is generated and path planning ends.

9. The optimization method for object recognition and obstacle avoidance grasping of a dual-arm robot according to claim 1, characterized in that: The movement of the robot in the working environment is planned based on the reward and punishment mechanism, specifically: In the action space, the robot can choose a limited and discrete speed, ranging from the interval [0, V max ] are uniformly selected, the heading angle Ω is uniformly distributed in [0,2π], and the robot running speed in space is decomposed along the Cartesian coordinate system into [ V x , V y ], P g , P , are the current spatial positions of the robot, target point, and pedestrian, respectively. Their spatial positions are decomposed into the X and Y axes in the Cartesian coordinate system. The running reward function is set as: S = S pass + S cross + S over ； ； in, S pass Avoid running reward functions that exceed the phenomenon. S cross To avoid the running reward function of the crossing phenomenon, S over To avoid running reward functions that cause pass-by, is the distance between the robot and the target, and the distance between the robot and the pedestrian is , the relative rotation angle between the two robots when they meet is , [ V x , V y ]、[ V x ' , V y ' ] are the current velocities of the two mobile dual-arm robots in space in Cartesian coordinates, and the angle |Θ-Ω| is limited to [-π, π].

10. An optimization system for object recognition and obstacle avoidance grasping of a dual-arm robot, characterized in that: An optimization method for object recognition and obstacle avoidance grasping of a dual-arm robot applied to any one of claims 1-9, comprising a diffusion network image restoration module, a point cloud segmentation and filtering module, a registration recognition module, a grasping posture detection module, a dual-arm motion path planning module, and a robot movement planning module; The diffusion network image restoration module is used to collect the image of the object to be captured and restore it based on the diffusion network and the image generation network, generate a point cloud for the restored image, and build a point cloud model library of the target object; The point cloud segmentation and filtering module is used to generate a point cloud for the real-time collected image of the object to be captured and perform point cloud segmentation and point cloud filtering to obtain point cloud information after filtering optimization; The registration and recognition module is used to perform point cloud registration on the point cloud information after filtering and optimization with the point cloud model library of the target object to determine whether it is an object to be grasped; The grasping posture detection module is used to obtain the optimal grasping posture by adopting a grasping posture detection algorithm based on local concavity and convexity constraints of point clouds and point cloud posture constraints; The dual-arm motion path planning module is used to plan the motion path of the robot's dual arms according to dual-arm decoupling planning, bidirectional search, and a path planning algorithm of a target point gravitational field; The robot movement planning module is used to plan the movement of the robot in the working environment according to the reward and punishment mechanism.

Images