Dual-arm robot object recognition and obstacle avoidance grabbing optimization method

Abstract

The application discloses a kind of bimanual robot object identification and optimization method of barrier avoidance grabbing, the method includes: collecting the image of object to be grabbed and is repaired based on diffusion network and image generation network, generates point cloud, constructs target object point cloud model library;Real-time acquisition of the image of object to be grabbed generates point cloud and carries out point cloud segmentation and point cloud filtering, obtains the point cloud information after filtering optimization and carries out point cloud registration with target object point cloud model library;Optimal gripping pose is obtained using gripping pose detection algorithm based on point cloud local concave-convex constraint and point cloud pose constraint;The movement path of robot double arm is planned based on double-arm decoupling planning, bidirectional search and target point attraction potential field path planning algorithm;The movement of robot in factory in multiple ranges is planned based on reward and punishment mechanism.The application improves the training effect of fuzzy image and uses point cloud filtering and improved pose constraint, help mechanical arm effectively match target object.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of robot object grabbing, and particularly relates to a dual-arm robot object recognition and obstacle avoidance grabbing optimization method. BACKGROUND

[0002] Manufacturing industry is facing the challenge of transformation and upgrading, and improving production efficiency and reducing labor cost has become the goal pursued by enterprises. With the rapid development of intelligent manufacturing and automation technology, dual-arm robots can effectively improve the automation level of production lines due to their flexibility and collaborative work ability, and play an increasingly important role in the fields of industry, logistics, medical treatment and the like. In many application scenarios, robot grabbing operation is a key link to realize an automated process, and driving a dual-arm robot to move and grab different objects autonomously under the command of an external instruction is a common mechanical arm operation mode at present. However, the existing robot grabbing often has problems such as low success rate, poor work efficiency, and easy collision of movement trajectories when dealing with different objects with complex shapes and variable poses.

[0003] In view of the above problems, in recent years, many scholars have proposed corresponding solutions. Wan Qin published an attitude estimation algorithm for different texture workpieces at different points in space in the journal of Control Theory and Applications, but the algorithm can only be applied to specific industrial workpieces and cannot be generalized to other grabbing objects; Jin Kaixuan proposed capturing a surface imaging image of a grabbing object by using a point cloud and transmitting the surface imaging image to an upper computer grabbing control unit, but the grabbing attitude is relatively single, which makes it difficult to change the grabbing attitude to effectively grab irregular parts in a complex environment, and is suitable for mechanical arms with fixed grabbing poses in industry and is not suitable for dual-arm robots applied in daily life; Liu Guanghui proposed a dual-arm motion algorithm using a kinematic model of a mechanical arm and a collision detection rule as a running constraint in the journal of Computer Integrated Systems, but the algorithm lacks node guidance in the search process, which may cause the algorithm to have a local optimal solution property, resulting in low algorithm planning efficiency. SUMMARY

[0004] The main purpose of the present application is to overcome the shortcomings and deficiencies of the prior art, and to provide a dual-arm robot object recognition and obstacle avoidance grabbing optimization method, which reduces the texture blur of fuzzy images in the grabbing data set, improves the case that the object edge is mistaken for the contour of a grabbable object by the grabbing network, thereby improving the generalization ability of the mechanical arm to grab different objects, improving the success rate of the dual-arm robot when processing object grabbing tasks, and improving the safety and flexibility of the spatial motion planning of the dual-arm robot.

[0005] In order to achieve the above purpose, the technical scheme adopted by the present application is as follows:

[0006] In one aspect of the present application, a method for optimizing dual-arm robot object recognition and obstacle avoidance grasping is provided, comprising the following steps:

[0007] Collecting an image of the object to be grasped and repairing the image based on a diffusion network and an image generation network, generating a point cloud from the repaired image, and constructing a target object point cloud model library;

[0008] Generating a point cloud from the real-time collected image of the object to be grasped, and performing point cloud segmentation and point cloud filtering to obtain filtered and optimized point cloud information;

[0009] Performing point cloud registration on the filtered and optimized point cloud information and the target object point cloud model library to determine whether it is the object to be grasped;

[0010] After recognizing the object to be grasped, a grasping pose detection algorithm based on point cloud local concave-convex constraint and point cloud pose constraint is used to obtain the optimal grasping pose;

[0011] A path planning algorithm based on dual-arm decoupling planning, bidirectional search, and target point attractive potential field is used to plan the motion path of the robot dual arms;

[0012] A reward and punishment mechanism is used to plan the movement of the robot in the working environment.

[0013] As a preferred technical solution, the repairing based on the diffusion network and the image generation network is specifically:

[0014] Data z sampled from an initial distribution p ( z ) is mapped to a high-dimensional data space by a generator G to obtain a generator sample x g = G ( z ), z~p ( z ), and the distribution is represented as ;

[0015] Real data samples x are extracted from a real data set p ( x ) obtained from the collected image of the object to be grasped, x~p ( x );

[0016] Diffusion process: Gaussian noise is added to the generator sample x g and the real data sample x in each step, and after T steps, a noisy image sample is obtained, specifically:

[0017] ;

[0018] wherein,q ( y | x ) is a mixture distribution in the T-th iteration; y is a random interference noise sample added; the size of the step number T is determined by β t Constraints: ;

[0019] The mixed component obtained by diffusion q ( y | x , t ) is a Gaussian distribution, the mean value is proportional to x, and the variance depends on the noise level at the current step t, which is expressed as:

[0020] ;

[0021] Where, a t =1- β t , , I is the threshold parameter of the set distribution;

[0022] The mixed distribution obtained by the diffusion process is used to train the discriminator D that can distinguish between real and generated noise samples, and the image judged as real by the discriminator D is output as the repaired image.

[0023] As a preferred technical solution, the mixed distribution obtained by the diffusion process is used to train the discriminator D that can distinguish between real and generated noise samples, specifically:

[0024] The objective function is defined as:

[0025] ;

[0026] Where, φ and ϑ respectively represent the adjustable parameters of the neural network in the discriminator D and the generator G, D φ is the probability of judging the sample output as false.

[0027] As a preferred technical solution, the generated point cloud adopts random sampling consistency segmentation, specifically:

[0028] There is a parameterized model in space, and the point cloud in the target space follows the following formula:

[0029] ax + by + cz = d ;

[0030] Wherein, ( x , y , z ) represent the spatial Cartesian coordinates of the target object point cloud, ( a , b , c ) represent the spatial normal vector of the target plane, d represent the distance from the origin of the reference coordinate system to the plane;

[0031] Randomly select three points in space as sampling points to substitute into the parameterized model, as follows:

[0032] a 1 x + b 1 y + c 1 z = d 1;

[0033] Randomly select another point in space as a standard point to substitute into the parameterized model, as follows:

[0034] a 1 x 2+ b 1 y 2+ c 1 z 2= d 2;

[0035] If the threshold distance ρ =| d 1- d 2| is less than 0.2, the corresponding point is judged to be an inner point, i.e., a point that satisfies the target model, otherwise it is an outer point;

[0036] After dividing the above four points in space into inner points and outer points, the number of inner points is recorded, then three random sampling points are selected again and the steps of judging inner points and outer points are executed, the number of iterations is set, and after the number of iterations is reached, the model with the most inner points is taken as the optimal fitting of the actual target point cloud model, and the redundant point cloud is removed.

[0037] As a preferred technical solution, the point cloud filtering adopts Euclidean clustering denoising filtering, specifically:

[0038] After constructing the KD-Tree in the point cloud coordinate system and after one-time segmentation, the remaining n points in space are mounted in it;

[0039] Initialize the point cloud, mark all points in space as unvisited state, and prepare for clustering;

[0040] Traverse the point cloud, for each unvisited point, find its neighboring points using KD-Tree, and calculate the Euclidean distance between it and its neighboring points; for any two points in space γ i With γ j , the Euclidean distance is calculated using the following formula:

[0041] ;

[0042] In the above formula, d ( γ i - γ j ) represents the Euclidean distance between γ i and γ j , γ ix represents the z-axis coordinate of the point γ i , x the x-axis coordinate of the point γ iy , and γ i the y-axis coordinate of the point y iz , γ i and γ j ; z i Similarly, γ j ;

[0043] Cluster and classify, traverse the entire KD-Tree to get the Euclidean distances n ( d γ i - χ j ) of the neighboring points, and compare them with the set threshold r , mark the points with a Euclidean distance less than the threshold as the same class of point cloud;

[0044] After clustering, the final optimized point cloud of the target to be grasped is obtained.

[0045] As a preferred technical solution, the grasping pose detection algorithm based on point cloud local concave-convex constraint and point cloud pose constraint is specifically:

[0046] Take the target point cloud as input, randomly sample multiple candidate grasping poses at multiple angles of the target point cloud;

[0047] The force closure index, the grasp wrench space, the point cloud local concave-convex constraint, and the point cloud pose constraint are taken as constraint conditions of the grasping pose for pose screening;

[0048] The force closure index is specifically that the resultant force and the resultant moment acting on the object are zero, and is used for measuring whether the grasping is stable;

[0049] The grasp wrench space is a set of force wrenches of all contact points, the force wrench of the contact point is specifically the force and the moment exerted on the object by the contact point when the object is grasped, and the force and the moment are combined into a force wrench; when the grasp wrench space can completely restrict the movement freedom degree of the object to be grasped, the movement of the object is constrained;

[0050] The point cloud local concave-convex constraint calculates the curvature of the point cloud by using the principal component analysis method, and is specifically:

[0051] The point cloud of the grasping point obtained by point cloud registration M Any point in m i , in its neighborhood, to m i and its neighboring points m ij Covariance analysis is performed:

[0052]

[0053]

[0054] C i is a 3*3 covariance matrix, l is the number of points in the neighborhood, is the center of the field point cloud, and are the eigenvalue and eigenvector of C i respectively;

[0055] Suppose the eigenvalue satisfies , then is the normal vector of m i , for m i The neighborhood curvature χ i of m i has:

[0056]

[0057] wherein, i =1,2,…, L ​​​, L representing the number of sampling points; field curvature χ i ( m i ) for measuring the bending degree of a point cloud M , χ i The smaller the value is, the more flat the neighborhood is, and vice versa χ i The larger the value is, the greater the bending fluctuation of the neighborhood is

[0058] For the surface point cloud of the gripper grabbing, the local concave-convex constraint is as follows:

[0059] F ≤ μ i ( m i )≤ G ;

[0060] wherein, F and G are constants, F representing the lower limit of the curvature of the grabbing surface point cloud, G representing the upper limit of the curvature of the grabbing surface point cloud

[0061] The point cloud pose constraint is specifically:

[0062] After point cloud registration, the pose of the target in the three-dimensional space is obtained, and the target pose is represented by the vector , , and the origin V 0, the gripper coordinate system origin a 0, and the three orthogonal vectors are represented by , , respectively

[0063] The grabbing sampling condition equation is:

[0064] ;

[0065] wherein, τ is a constant, Figure 1 is a parameter set to determine the size of the grabbing range, a 1, a 2, a 3 are the values of the three orthogonal vectors , , of the gripper respectively, v 1, v 2, v 3 are the target pose vectors , , The value;

[0066] The selected candidate grasping poses are input into the grasping evaluation network to score all grasping poses;

[0067] Output the capture pose with the highest score.

[0068] As a preferred technical solution, the dual-arm decoupling plan is as follows:

[0069] Set the initial and target poses of the master and slave arms;

[0070] Perform path planning for the master arm or slave arm;

[0071] Determine if a collision has occurred:

[0072] If a collision occurs, discard the slave arm path, add the master arm trajectory information to the slave arm's state validity assessment, and re-plan the slave arm's path.

[0073] If no collision occurs, complete the path planning for the main arm and slave arm.

[0074] As a preferred technical solution, the bidirectional search and the target point gravitational field are specifically as follows:

[0075] Perform a bidirectional search and select the tree update direction. Specifically, place the starting point and the ending point in two different diffusion sets and propagate them bidirectionally towards each other. Before each round of diffusion, determine the number of nodes in each tree and expand the tree with the smaller number of nodes until the two trees intersect.

[0076] Select the initial state as the starting node;

[0077] Random nodes are obtained by random sampling in the motion space. q rand ;

[0078] Finding and Random Nodes q rand nearest node q near ;

[0079] Calculate the initial direction , where || ⋅ || is the norm in the configuration space, set as L1 or L2 norm;

[0080] Under the influence of a gravitational field, it gains a new direction of growth. d' and determine new nodes. q' new Specifically, it means: defining the target point as a pair of nodes. q gravitational function ,in k The gravitational coefficient, q goal The target point is the gravitational function applied to it. q near At that point, a new direction for growth was found. and new nodes ,in q new When no gravitational function is added q near Towards q rand The direction is in step length ∆ q Expanded nodes;

[0081] judge q' new Is it an obstacle point? If it is an obstacle point, return to find a random node. q rand nearest node q near The steps; if it is not an obstacle point, q' new Add to the node collection;

[0082] connect q' new and q near Check for collisions in the connections: if a collision occurs, delete the connection. q' new And return to find the random node. q rand nearest node q near The steps are as follows: if no collision occurs, determine whether the initial point is connected to the target point;

[0083] If no node connection is found, return to randomly sampling in the motion space to obtain random nodes. q rand The steps are as follows: if node connections occur, a connection path is generated, and path planning ends.

[0084] As a preferred technical solution, the method of planning the robot's movement in the working environment based on a reward and punishment mechanism specifically includes:

[0085] In the action space, the robot can choose a limited and discrete speed, with the speed magnitude ranging from [0, ... V max The robot's trajectory is uniformly selected within the Cartesian coordinate system, with the heading angle Ω uniformly distributed in the range [0, 2π]. The robot's velocity in space is decomposed along the Cartesian coordinate system into [...]. V x ,V y ], P g 、 P 、 are the current spatial positions of the robot, target point and pedestrian respectively, whose spatial positions are decomposed into X and Y axes in the Cartesian coordinate system, and the running reward function is set as:

[0086] S = S pass + S cross + S over ;

[0087] ;

[0088] wherein, S pass the running reward function for avoiding the exceeding phenomenon, S cross the running reward function for avoiding the crossing phenomenon, S over the running reward function for avoiding the passing-by phenomenon, is the distance between the robot and the target, the distance between the robot and the pedestrian is , the relative rotation angle between the two robots when they meet is ,[ V x , V y ]、[ V x ' , V y ' are respectively the current speed of the two mobile dual-arm robots in the space in the Cartesian coordinate system, and the angle |Θ-Ω| is limited to [-π, π].

[0089] Another aspect of the present application also provides an optimization system for dual-arm robot object recognition and obstacle avoidance grabbing, which is applied to the dual-arm robot object recognition and obstacle avoidance grabbing optimization method described above, and comprises a diffusion network image repair module, a point cloud segmentation and filtering module, a registration recognition module, a grabbing pose detection module, a dual-arm motion path planning module and a robot movement planning module.

[0090] The diffusion network image repair module is used for collecting the image of the object to be grabbed and repairing the image based on the diffusion network and the image generation network, generating a point cloud for the repaired image, and constructing a target object point cloud model library.

[0091] The point cloud segmentation and filtering module is used for generating a point cloud from a real-time collected object image to be grabbed and performing point cloud segmentation and point cloud filtering to obtain filtered and optimized point cloud information.

[0092] The registration identification module is used for performing point cloud registration on the filtered and optimized point cloud information and a target object point cloud model library to determine whether it is an object to be grabbed.

[0093] The grabbing pose detection module is used for adopting a grabbing pose detection algorithm based on point cloud local concave-convex constraint and point cloud pose constraint to obtain an optimal grabbing pose.

[0094] The dual-arm motion path planning module is used for planning a motion path of the robot dual arms according to a dual-arm decoupling planning, bidirectional search and target point attractive force field path planning algorithm.

[0095] The robot movement planning module is used for planning movement of the robot in a working environment according to a reward and punishment mechanism.

[0096] Compared with the prior art, the present application has the following advantages and beneficial effects:

[0097] (1) The present application adopts a diffusion image generation network to improve image training efficiency, which helps to improve the repair performance of the generative adversarial network, improve the situation that the object edge is mistakenly considered as the contour of the grabbable object by the grabbing network, improve the training effect on the fuzzy image, and help subsequent point cloud matching to improve the generalization ability of the mechanical arm in grabbing different objects.

[0098] (2) The present application adopts point cloud filtering and improved pose constraint to help the mechanical arm effectively match the target object and grab the object in a reasonable grabbing posture, effectively identify the target grabbing object in a complex occlusion environment, execute the grabbing operation, and improve the success rate of the dual-arm robot in handling the object grabbing task.

[0099] (3) The present application adopts a target point guided spatial motion planning algorithm and dual-arm coordinated motion planning criterion to improve the spatial motion planning efficiency of the dual-arm robot, so that it can effectively plan a path in a complex environment in a short time and avoid self-collision of the dual arms or collision of the arm end with the space environment obstacles.

[0100] (4) The present application adopts a running penalty function to ensure that collision with pedestrians or other mobile robots can be avoided when operating in a complex environment, enhance the reliability of the robot in different scenes, and improve the safety and flexibility of the spatial motion of the dual-arm robot. BRIEF DESCRIPTION OF DRAWINGS

[0101] Figure 2 is a flowchart of an optimization method for dual-arm robot object identification and obstacle avoidance grabbing according to an embodiment of the present application.

[0102] Figure 3 is a structural schematic diagram of a traditional image generation model;

[0103] Figure 4 is a structural schematic diagram of a diffusion image generation network of an embodiment of the present application;

[0104] Figure 5 is a tree diagram of an improved mechanical arm motion planning algorithm of an embodiment of the present application;

[0105] Figure 6 is a schematic diagram of a mechanical arm motion planning process of an embodiment of the present application;

[0106] Figure 7 is a flowchart of a dual-arm coordinated motion planning criterion of an embodiment of the present application;

[0107] Figure 8 is a robot operation safety criterion diagram of an embodiment of the present application;

[0108] Figure 1 is a structural schematic diagram of a dual-arm robot object recognition and obstacle avoidance grasping optimization system of an embodiment of the present application. DETAILED DESCRIPTION

[0109] In order for those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0110] Embodiment:

[0111] As shown in the accompanying drawings, Figure 2 the present embodiment provides a dual-arm robot object recognition and obstacle avoidance grasping optimization method, comprising the following steps:

[0112] S1, collect the image of the object to be grasped and repair based on the diffusion network and the image generation network, generate point cloud for the repaired image, and construct a target object point cloud model library.

[0113] A reliable grasping dataset is beneficial for robotic arms to successfully identify objects to be grasped in application environments. Datasets for common everyday objects are already quite comprehensive, but with the increasing application environments of robots, the grasping of irregular objects is also on the rise, such as the need to grasp composite workpieces in industrial applications. However, images of irregular objects are prone to edge texture issues, meaning the boundary between the object and the background is unclear or indistinct. This is mostly caused by uneven lighting during shooting due to the irregular object's geometry or the unevenness of the workpiece surface. This can interfere with the robotic arm's grasping algorithm's accurate detection of object edges. Therefore, this embodiment proposes a method for image enhancement using a diffusion network. By introducing a diffusion model into the image generator, the generated image and the real image undergo a forward diffusion process to obtain a repaired image with Gaussian noise and the real image. These are used as input to the image discrimination module, and the process iteratively obtains the image that is judged as real as the final output repaired image.

[0114] 1.1 Traditional image generation models;

[0115] Traditional image generation models, such as GANs, aim to learn the data distribution of the target dataset by constructing an image generator and an image discriminator in two neural network modules and establishing a mini-maximum game between the two modules. p ( x The generator 𝐺 will generate from a simple prior distribution such as the standard normal or uniform distribution. p ( z Sampling random noise vector z As input, and in an attempt to generate pseudo-samples of the original data. G ( z The input to the discriminator is from... p ( x The extracted real data sample 𝑥 or the pseudo sample generated by 𝐺 G ( z The goal of is to deceive and attempt to correctly classify them as real or spurious samples. The goal of is to deceive and cause to misclassify, while the goal of is to accurately distinguish between them. G ( z ) and 𝑥. Generative adversarial network structures such as z~p As shown.

[0116] 1.2 Diffusion Image Generation Network;

[0117] First, a traditional generator network 𝐺 is used to start from the initial distribution. p ( z The sampled data is mapped to a high-dimensional data space to generate near-realistic samples. x g Generator Samplesx g = G ( z ), x~p ( z ) of the distribution is denoted as . To make the generator more robust, noise is injected into the generated samples x g by a diffusion process that adds Gaussian noise at each step. The diffusion process can be seen as a Markov chain that starts from the initial sample 𝑥 and gradually becomes a noisy image sample after T steps. The size of the number of steps 𝑇 is constrained by β t .

[0118] This embodiment defines the random interference noise sample 𝑦 added at any step of the diffusion process as a mixture distribution q ( y | x ). q ( y | x , t ) is a Gaussian distribution whose mean is proportional to 𝑥 and whose variance depends on the noise level at the current step 𝑡. The real sample Figure 3 ( x ) and the generated sample x g ~p ( x g ) use the same diffusion process and mixture distribution. The mixture distribution of the real image and the generated image after the diffusion process is denoted as:

[0119] (1) ;

[0120] where q ( y | x ) is the mixture distribution in the 𝑇 iterations, the mixture component obtained by diffusion q ( y | x , t ) is denoted as follows, where a t =1- β t , , I is a threshold parameter of the distribution, which can be artificially adjusted to a reasonable interval.

[0121] (2) ;

[0122] ​Sampling y from the mixed distribution can obtain real samples and noise versions of generated samples with different degrees of noise. The more steps taken in the diffusion process, the more noise added to y, and the less information retained from x. Then use this mixed distribution obtained by the diffusion process to train a discriminator D that can distinguish between real and generated noisy samples, and iterate the image loop repair that is judged as fake by the discriminator D, and output the image judged as true as the repair image. The diffusion image generation network framework is as shown in φ .

[0123] The objective function of the diffusion generative adversarial network is defined as follows, where, ϑ and Figure 1 represent the adjustable parameters of the neural networks in the discriminator and the generator respectively, D φ is the probability that the sample output is false.

[0124] (3);

[0125] The training target of the objective function is to make the discriminator's ability to distinguish true and false the strongest, that is, in any diffusion step t, a high probability is assigned to the real image after adding noise, and a low probability is assigned to the generated image after adding noise. The goal of the generator is to generate samples that can most deceive the discriminator in any diffusion step t.

[0126] p(x) is the distribution of real pictures; q ( y | x , t ) is the conditional distribution of the given original data x and the noise sample y with diffusion step length t, and according to the Gaussian re-distribution, y can be simplified as ,σ is a proportional factor, is a Gaussian noise distribution. Similarly, because the generated sample of the diffusion process , can be rewritten as y g . .

[0127] By introducing the diffusion model in the image generation network, the model can focus on the true and false judgment of the image itself, and ignore the edge texture and structure information, thereby improving the repair performance of the generative adversarial network. Secondly, through this method, the object contour can be distinguished, and a relatively complete RGB image can be generated for each complex workpiece information, which can be used to design a point cloud model library for subsequent spatial object grabbing.

[0128] S2, generating a point cloud from the real-time collected object image to be grabbed, and performing point cloud segmentation and point cloud filtering to obtain filtered and optimized point cloud information.

[0129] Currently, the grasp detection based on 2D images has been widely applied in robotic vision grasping, but a single 2D image cannot fully reflect the three-dimensional spatial information of the target to be grasped, and the grasp detection using 3D point cloud can directly obtain the three-dimensional spatial information of the target object. Therefore, after obtaining the spatial information of the grasped target, how to select a reasonable grasping pose is the difficulty of current research. Therefore, in this patent, the data collected by the depth camera is first segmented, and based on the above-mentioned adversarial image generation network, a target object point cloud database is designed. When the robot arm grasps, the corresponding object is segmented in the image, and the point cloud is filtered to obtain the filtered and optimized point cloud information, which is matched with the database to determine whether it is the object to be grasped, and then through the improved grasp pose detection algorithm, the spatial pose of the gripper that is more consistent with the force analysis and the stable grasping attitude are obtained. The overall flow chart is shown in ax .

[0130] 2.1, point cloud segmentation and registration method.

[0131] 2.1.1, random sampling one-time segmentation.

[0132] The random sampling consistency algorithm is a method of estimating the model parameters of a set of discrete data using an iterative method. First, a parameterized model in space is designed, and the point cloud in the target space follows the following model formula:

[0133] by + cz + ρ = d (4);

[0134] In the above formula, x , y , z represent the spatial Cartesian coordinates of the target object point cloud, a , b , c represent the spatial normal vector of the target plane, d represents the distance from the origin of the reference coordinate system to the plane. The random sampling consistency segmentation algorithm assumes that the point cloud distribution in space includes inliers and outliers. The inliers are points that satisfy the target model, and the outliers are otherwise. First, three points in space are randomly selected (at least three points are needed to determine the plane), and the three points are used as sampling points to substitute into the parameterized model to obtain the equation:

[0135] a 1 x + b 1 y + c 1 z = d1(5) ;

[0136] Randomly select another point in space as a standard point into the model a 1 x 2+ b 1 y 2+ c 1 z 2= d 2, by distance threshold ρ to determine whether the point is an inner point:

[0137] ρ =| d 1- d 2| (6) ;

[0138] If the distance threshold γ is less than 0.2, it is determined that the point is an inner point. After dividing the four points in the above space into inner points and outer points, record the number of inner points, then select another three random sampling points and perform the steps of judging inner points and outer points. Set the number of iterations, and after reaching the number of iterations, the model with the most inner points is considered to be the optimal fitting of the actual target point cloud model, and the redundant point cloud is removed.

[0139] 2.1.2, Euclidean clustering denoising filter.

[0140] Using random one-time segmentation can effectively segment the target object, but there are still some discrete point clouds. In order to further clean up these noises and extract the point cloud of the target object, Euclidean clustering denoising filter is used to regard points with similar distances as the same class, which can separate the point cloud of the target object from other noises in space.

[0141] First, construct a KD-Tree in the point cloud coordinate system and mount the remaining n points in the space after random one-time segmentation in it, which can efficiently manage the points in the space and speed up the search process of neighboring points. Then, initialize the point cloud, mark all points in the space as unvisited state, and prepare for clustering. Then, traverse the point cloud, for each unvisited point, find its neighboring points using KD-Tree, and calculate the Euclidean distance between them. For any two points γ i and γ j , calculate their Euclidean distance using the following formula:

[0142] (7) ;

[0143] In the above formula, d ( γ i - γj )represent γ i and γ j European distance, γ ix represent γ i point x Axis coordinates γ iy represent γ i point y Axis coordinates γ iz represent γ i point z Axis coordinates γ j Similarly, clustering and category division are then performed, traversing the entire KD-Tree to obtain... n Euclidean distance between neighboring points d ( γ i - χ j ), and compare it with the threshold r Comparison, threshold r The numerical values ​​can be manually adjusted to mark points with a Euclidean distance less than a threshold as belonging to the same point cloud class. Finally, after clustering is completed, the final optimized point cloud of the target to be captured is obtained.

[0144] S3. Perform point cloud registration between the filtered and optimized point cloud information and the target object point cloud model library to determine whether it is the object to be captured.

[0145] S4. After identifying the object to be grasped, a grasping pose detection algorithm based on point cloud local concavity and convexity constraints and point cloud pose constraints is used to obtain the optimal grasping pose.

[0146] Traditional pose detection algorithms often use deep learning for grasping pose detection, enabling them to predict the optimal grasping posture from a set of 3D point cloud data. This algorithm takes the target point cloud as input. For irregular objects, using a single viewpoint may result in grasping errors. Therefore, multiple sets of candidate grasping poses are randomly sampled from multiple angles of the target point cloud. Then, force closure indices and grasping spinor space are used as constraints for grasping pose selection. The selected candidate grasping poses are then input into a grasping evaluation network, which scores all grasping poses and finally outputs the optimal grasping pose.

[0147] The force closure index is specifically: if an object is grabbed and no matter what direction the external force is, it will not be separated from the robot hand, then this grabbing is called force closure. Force closure requires that the resultant force and moment acting on the object are zero. In the process of grabbing an object, the force closure index is used to measure whether the grabbing is stable, that is, whether the force exerted on the object by the fingers or grabbing tool can completely constrain the motion of the object. For example, in a simple planar grabbing scene, an object is clamped with two fingers. If the force exerted by the fingers can balance the gravity of the object and any possible external force interference, so that the object has no tendency to translate and rotate in the plane, force closure is achieved.

[0148] The grasping screw space is specifically: the space composed of the force screw at all possible contact points. When an object is grasped, a force and a moment will be exerted on the object at each contact point. These forces and moments can be combined into a force screw. The force screws of all contact points together form the grasping screw space. Taking the above-mentioned scene of clamping an object with two fingers as an example, a force screw will be generated at the contact point of each finger and the object. The space spanned by the two force screws is the grasping screw space. As long as the grasping screw space can completely limit the degrees of freedom of the object motion, the object motion can be constrained.

[0149] The grasping scoring network usually only uses two indexes to judge the stability of the candidate grasp, without using the local surface information and global pose information of the point cloud, resulting in more low-quality grasps generated, affecting the execution efficiency and accuracy of the algorithm. Therefore, the present patent adds point cloud local concave-convex constraint and point cloud pose constraint to improve the grasp generation constraint on the basis of the traditional method.

[0150] The constraint of the local concave-convex of the point cloud can be realized by analyzing the curvature of the point cloud in the local coordinate system of the candidate grasp. For calculating the curvature of the point cloud, the principal component analysis method can be used. For any point P in the grasping point cloud M m i , covariance analysis is performed on m i and its neighboring points m ij

[0151] (8);

[0152] (9);

[0153] C i is a 3*3 covariance matrix, l is the number of points in the neighborhood, ​​is the center of the field point cloud, and are the eigenvalues and eigenvectors of C i respectively. Assuming the eigenvalues satisfy then is the normal vector of m i for m i the neighborhood curvature of χ i ( m i ) has:

[0154] (10);

[0155] where i =1,2,…, L , L denotes the number of sampling points. The field curvature can measure the bending degree of the point cloud M , χ i the smaller the neighborhood curvature is, the flatter the neighborhood is, and vice versa, χ i the larger the neighborhood curvature is, the greater the bending fluctuation of the neighborhood is. For the surface point cloud of the gripper grabbing, the following constraints are sampled as its local concave-convex constraints:

[0156] F ≤ μ i ( m i )≤ G (11);

[0157] where F and G are constants, F denotes the lower limit of the curvature of the surface point cloud to be grabbed, G denotes the upper limit of the curvature of the surface point cloud to be grabbed, and the two values can be adjusted according to the actual grabbing effect.

[0158] In addition to using the local point cloud of the surface of the target to be grabbed, the overall pose of the point cloud can also be used to screen the candidate grabbing, i.e. the point cloud pose constraint. After point cloud registration, the pose of the target in the three-dimensional space is obtained, and the target pose can be represented by the vector , , and the origin V 0, the origin of the gripper coordinate system a 0, and the three orthogonal vectors are represented by , , .

[0159] Take the two-finger gripper at the end of the mechanical arm as an example, its pose with the target object should satisfy the following constraints. First, the two-finger gripper should be distributed on both sides of the main shaft. Second, the two-finger gripper should be parallel to the surface of the target to be grasped, that is, perpendicular to the surface normal of the grasped target. Finally, the two-finger gripper should be as close as possible to the center of mass of the object to be grasped for stability. Assuming uniform mass, the center of the target can replace the center of mass.

[0160] The above three constraints are used as the conditions of the point cloud pose constraints, and the grasping sampling condition equation can be generated as:

[0161] (12);

[0162] τ is a constant, a 1、 a 2、 a 3 are the values of the three orthogonal vectors 、 、 of the gripper, v 1、 v 2、 v 3 are the values of the target pose vectors 、 、 ; τ is a parameter set to determine the size of the grasping range, which can be adjusted according to actual conditions. When τ is small, the grasping will be sampled in the area close to the center of mass, and the grasping will be more stable but the grasping range will be smaller; when Figure 4 is large, the grasping will be sampled in the area far from the center of mass, and the grasping range will be larger.

[0163] S5, the path planning algorithm based on dual-arm decoupling planning, bidirectional search and target point attractive potential field is used to plan the motion path of the robot dual-arm.

[0164] 5.1, traditional mechanical arm motion planning algorithm.

[0165] For a complex system with multiple degrees of freedom such as a mechanical arm, if only the path in the workspace is planned, only the end effector of the mechanical arm can avoid obstacles, but the body of the mechanical arm cannot avoid obstacles. Therefore, in the past, scholars introduced the concept of C space (configuration space) in the motion space of the mechanical arm, which is a space related to the configuration of the robot. All robots in the C space can be regarded as a point robot, so the problem of path planning for a robot with complex configuration in the workspace is transformed into a problem of path planning for a point in the C space.

[0166] The RRT algorithm is a common path planning algorithm, and its search process is as follows:

[0167] (1) Given the starting point q start , the target point q goal and the algorithm step size Δ q , a random point q rand1 is generated in the C space.

[0168] (2) q rand1 The direction of the vector formed by the parent node is the direction of the first expansion of the random tree. Take q start as the starting point and Δ q as the step size to expand the first child node q new1 .

[0169] (3) The state validity of the node q new1 is judged. If it is determined to be a valid point, the node q new1 is added to the node set and edge set of the random tree. If it does not pass, it is discarded, and a random point is selected again and the above process is repeated.

[0170] (4) A random point q rand2 is generated in the space.

[0171] (5) From the previously generated nodes, select the nearest point to q rand2 , such as q new1 , expand the second child node q new2 in the direction from to q new2 , taking q new1 as the starting point and Δ q as the step size.

[0172] (6) Repeat the above steps until the distance from the new node to the target point is less than Δ q , and the search is completed.

[0173] (7) Starting from q goal , backtrack upwards in turn, and find the corresponding relationship between the parent node and the child node until the found parent node is q startTo stop, then the found nodes in order, can get a start point to the target point of no collision path.

[0174] The traditional algorithm by random sampling in the sampling space, constantly expanding tree structure, but because the node guiding random tree growth is randomly generated in space, and for one-way search, sampling points will be throughout the sampling space, random tree has quite a part of the time in the direction of growing away from the target point, in high-dimensional space or in the environment of more obstacles, will lead to search efficiency greatly discounted.

[0175] 5.2, improve the motion planning algorithm of mechanical arm.

[0176] For the problem of RRT search in 5.1, this embodiment proposes a two-armed global bidirectional planning algorithm combined with target point induced potential field, the specific algorithm process is introduced as follows.

[0177] As Figure 5 shown, bidirectional search refers to placing the start point and the end point in two different diffusion sets during the search process, and performing bidirectional transmission towards each other, judging the number of nodes of each tree before each diffusion round, expanding the tree with smaller number of nodes, until the two trees intersect, that is, a complete trajectory curve is generated.

[0178] As Figure 6 shown, the target function diffusion is explained with the start point as the tree starting direction, in the diffusion book of the start point, q start is the initial node, q goal is the target node, q rand is a random node generated by the random tree in the configuration space, and the parent node is q near Without adding the gravitational function, q near a new node q rand will be expanded in the direction of step length q q new , and the growth direction is:

[0179] (13);

[0180] Where || ⋅ || is the norm in the configuration space, which can be set as L1 or L2 norm, and a gravitational field is set at the target point q goal , and the gravitational coefficient is k , which is self-defined, q goal in​q near The gravitational potential energy at that point is U and q goal exist q near The gravitational force is:

[0181] (14);

[0182] (15);

[0183] This leads to the target point being paired with any node. q Gravitational function:

[0184] (16);

[0185] Applying the gravitational function to q near Above, making q near The point will be subject to the gravitational pull from the target point, and will be biased to a certain extent. q goal This leads to new growth directions. d' and new nodes q' new :

[0186] (17);

[0187] (18);

[0188] because F ( q Due to the influence of gravity, the random tree will tend to grow towards the target direction. If you want to change the magnitude of gravity at the target point, you only need to change the gravity coefficient. k That's all. When k When the target node is larger, its attraction to the random tree is greater, guiding the random tree to grow more towards the target node; when... k When the tree is smaller, the random tree grows more divergently, tending to expand outwards into the surrounding environment, which is beneficial for avoiding obstacles and searching for reachable paths. This can be based on... Adjust the size of the coefficient k To ensure that the algorithm is q goal No oscillations occur nearby. By introducing a gravitational function field and a bidirectional search strategy into the RRT algorithm, the generation of random trees is accelerated. Similarly, the direction of the tree start is determined by the endpoint, under the influence of gravity... q' near Generate orientation q start intermediate tree nodesq'' new , as the search trees are updated to converge on each other until they are connected.

[0189] As Figure 7 shown, in order to avoid the curse of dimensionality brought by centralized planning, a decoupled way is adopted to plan the coordinated motion of the dual arms. The planning process is shown in the following figure. Now it is defined that one of the arms is the master arm and the other is the slave arm, and here the left arm is taken as the master arm and the right arm as the slave arm as an example. First, the path of the master arm is planned by using the improved RRT algorithm. The master arm does not consider the state of the slave arm when planning the path, and only considers the static obstacles in the environment. After the master arm plans a feasible path, the path of the slave arm is planned. The slave arm only considers the obstacles in the environment when initially planning the path, and can get a collision-free path of the slave arm. At this time, time information is given to the path points of the two paths, and collision detection is carried out for the path points at the same time. If there is no collision point between the two arms after the detection is completed, the motion planning process is ended; if some points in the middle are detected to have collision, the path of the slave arm is discarded, and the trajectory information of the master arm is added to the collision library of the slave arm, and the path of the slave arm is re-planned until the slave arm plans a path that does not collide with the master arm and the obstacles in the environment.

[0190] In order to prove the effectiveness of the above-mentioned guidance algorithm, a simulation environment of a dual-arm robot is constructed in the software Pybullet, ten obstacles are randomly generated as environment one, thirty obstacles are randomly generated as environment two, a space Cartesian coordinate point is given as a target point, and the obstacle avoidance ability of the dual arms to move to the target point is tested. Under the controller of the same algorithm, it is continuously run for ten times, and the average time and error cost are taken as evaluation indexes. The constructed environment and the test result data are shown as follows:

[0191] ;

[0192] Table 1. Environment one experimental data.

[0193] ;

[0194] Table 2. Environment two experimental data.

[0195] Among the above results, RRT (attractive) represents the attractive optimization RRT algorithm, and RRT (attractive -dual) represents the attractive optimization RRT algorithm combined with the dual-arm coordination mode. As can be seen from the above results, the improved algorithm RRT (attractive -dual) proposed in the present application has the lowest time consumption and error in various complex environments, and the optimization performance of the algorithm gradually increases compared with the traditional RRT algorithm as the number of obstacles in the complex environment increases. For example, in the first environment, the average motion time of RRT and RRT (attractive) is the same, but in the second environment with increased obstacles, the time required by RRT (attractive) is significantly shorter than that of the traditional RRT algorithm.

[0196] S6, planning the movement of the robot in multiple ranges in the factory based on the reward and punishment mechanism.

[0197] In industrial applications, a mobile chassis is often matched with a dual-arm robot for movement in multiple ranges in the factory. In order to avoid collision, the dual-arm robot needs to follow simple movement rules. Although the rules are intuitive, they are difficult to quantify accurately. For example, when the robot passes through a pedestrian, the turning time and turning angle are randomly strong, making the problem more complex. Therefore, the reward and punishment mechanism is used to induce the robot to follow the habits of factory operation, which is convenient for standardizing the safety of robot operation.

[0198] In the action space, the speed of the robot is limited and discrete. The speed size is uniformly selected from the interval [0, V max ] and the heading angle Ω is uniformly distributed in [0, 2π]. The running speed of the robot in space can be decomposed into[ V x , V y ] along the Cartesian coordinate system, P g 、 P 、 are the current spatial positions of the robot, the target point, and the pedestrian, respectively, which can be decomposed into X and Y axes in the Cartesian coordinate system. The running reward function is set as:

[0199] S S pass S cross S over (19);

[0200] (20);

[0201] In the above formula,​​​S pass an operating reward function for avoiding the occurrence of the exceeding phenomenon, S cross an operating reward function for avoiding the occurrence of the crossing phenomenon, S over an operating reward function for avoiding the occurrence of the passing-by phenomenon, is the distance between the robot and the target, the distance between the robot and the pedestrian is , the relative rotation angle between the two robots when they meet is , V x , V y ]、[ V x ' , V y ' ] respectively represent the current speed of the two moving dual-arm robots in the space in the Cartesian coordinates, and the angle |Θ-Ω| is limited to [-π, π]. The diagram of the penalty set is shown in Figure 8 , which is divided into three behaviors according to the relative position and speed between the robot and the pedestrian, and their target directions. If the robot appears in the shadow area, it will be punished, which corresponds to the three situations of exceeding, passing-by and crossing respectively.

[0202] As shown in ​ , in another embodiment of the present application, an optimization system for dual-arm robot object recognition and obstacle avoidance grabbing is provided, which comprises a diffusion network image repair module, a point cloud segmentation and filtering module, a registration identification module, a grabbing pose detection module, a dual-arm motion path planning module, and a robot movement planning module;

[0203] The diffusion network image repair module is used for collecting an image of an object to be grabbed and repairing the image based on a diffusion network and an image generation network, generating a point cloud from the repaired image, and constructing a target object point cloud model library;

[0204] The point cloud segmentation and filtering module is used for generating a point cloud from a real-time collected image of an object to be grabbed and performing point cloud segmentation and point cloud filtering to obtain filtered and optimized point cloud information;

[0205] The registration identification module is used for performing point cloud registration on the filtered and optimized point cloud information and the target object point cloud model library to determine whether it is an object to be grabbed;

[0206] The grabbing pose detection module is used for adopting a grabbing pose detection algorithm based on point cloud local concave-convex constraint and point cloud pose constraint to obtain an optimal grabbing pose;

[0207] The dual-arm motion path planning module is configured to plan a motion path of the dual arms of the robot according to a dual-arm decoupling planning, bidirectional search, and target point attractive potential field path planning algorithm.

[0208] The robot movement planning module is configured to plan movement of the robot in the working environment according to a reward and punishment mechanism.

[0209] It should be noted that the system provided by the above embodiments is only exemplified by the division of the above functional modules, and in actual application, the above functions can be completed by different functional modules according to needs, that is, the internal structure is divided into different functional modules to complete all or part of the above described functions. The system is a kind of dual-arm robot object identification and obstacle avoidance grabbing optimization method applied to the above embodiments.

[0210] It should be understood that parts of the present application can be realized by hardware, software, firmware or their combination. In the above embodiments, a plurality of steps or methods can be realized by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if realized by hardware, and as in another embodiment, it can be realized by any one or their combination of the following technologies known in the art: discrete logic circuit with logic gate circuit for implementing logic function on data signal, application specific integrated circuit with suitable combination logic gate circuit, programmable gate array (PGA), field programmable gate array (FPGA) and the like.

[0211] The above embodiments are the preferred embodiments of the present application, but the embodiments of the present application are not limited to the above embodiments, and any changes, modifications, substitutions, combinations, simplifications made without departing from the spirit and principles of the present application are equivalent replacement methods and are included in the protection scope of the present application.

Claims

1. An optimization method for dual-arm robot object recognition and obstacle avoidance grasping, characterized in that, Comprise the following steps: Collect the image of the object to be grabbed and repair based on the diffusion network and the image generation network, generate point cloud based on the repaired image, and construct the target object point cloud model library; Generate point cloud based on the real-time collected object to be grabbed, and perform point cloud segmentation and point cloud filtering to obtain filtered and optimized point cloud information; The filtered and optimized point cloud information is registered with the target object point cloud model library to determine whether it is the object to be grabbed; After recognizing the object to be grabbed, a grasping pose detection algorithm based on point cloud local concave-convex constraint and point cloud pose constraint is used to obtain the optimal grasping pose; The motion path of the robot arms is planned based on dual-arm decoupling planning, bidirectional search and target point attractive potential field path planning algorithm; the bidirectional search and target point attractive potential field are specifically: Bidirectional search is performed to select tree update direction, specifically: the starting point and the ending point are placed in two different diffusion sets at the same time, and bidirectional transmission is performed towards each other, the node number of each tree is judged before each diffusion round, and the tree with fewer nodes is expanded until the two trees intersect; The initial state is selected as the starting node; randomly sampling a random node in a motion space q rand ; Find closest node to random node q rand Closest node to random node q near ; Computing initial direction where || · || is a norm in the configuration space, set to the L1 or L2 norm; Get a new growth direction under the action of the potential field d' And determine the new node q' new Specifically: define the target point q Gravitational function of any node Where k Gravitational coefficient, q goal Target point; The gravitational function is applied to q near The new growth direction is obtained at And the new node Where q new Without the gravitational function q near The node is expanded in the direction of q rand With a step length of q The node is expanded in the direction of determine q' new if it is an obstacle point, return to find a random node q rand the nearest node q near the steps; if not a barrier point, will q' new be added to the node set; connect q' new and q near Check for collisions in the connections: if a collision occurs, delete the connection. q' new And return to find the random node. q rand nearest node q near The steps are as follows: if no collision occurs, determine whether the initial point is connected to the target point; If no node connections appear, return a randomly sampled node in the motion space q rand of steps; If the node is connected, a connection path is generated, and the path planning is ended; The movement of the robot in the working environment is planned based on the reward and punishment mechanism.

2. The method of claim 1, wherein, The repair based on the diffusion network and the image generation network is specifically: By the generator G, data z sampled from the initial distribution p ( z ) is mapped into a high-dimensional data space to obtain a generator sample x g = G ( z ), z~p ( z ), the distribution is represented as ; Real dataset obtained from images of objects to be picked up p ( x ) extracting real data samples x from a real dataset, x~p ( x ) Diffusion Process: In each step, the generator samples x g And add Gaussian noise to the real data samples x, get the noisy image samples after T steps, which is: ； wherein, q y x is the mixture distribution in the 𝑇th iteration; 𝑦 is an added random disturbance noise sample; the size of the step number 𝑇 is subject to β t Constraints: ;​​ Mixed components obtained by diffusion q ( y | x , t ) is a Gaussian distribution with mean proportional to x and variance depending on the noise level at the current step t, represented as follows: ； wherein a t =1- β t , , I is a threshold parameter of the set distribution; The mixed distribution obtained by the diffusion process is used to train a discriminator D that can distinguish between real and generated noise samples, and the image judged as real by the discriminator D is output as the repaired image.

3. The method of claim 2, wherein, The mixed distribution obtained by the diffusion process is used to train a discriminator D that can distinguish between real and generated noise samples, and the image judged as real by the discriminator D is output as the repaired image. The objective function is defined as: ； wherein, φ and ϑ respectively represent the neural network adjustable parameters in the discriminator D and the generator G, D φ is the probability that the sample output is false.

4. The method of claim 1, wherein, The generated point cloud uses random sample consensus segmentation, specifically: There is a parameterized model in space, and the point cloud in the target space follows the following formula: ax + by + cz = d ； wherein, x , y , z ) represent the spatial Cartesian coordinates of the target object point cloud, ( a , b , c ) represent the spatial normal vector of the target plane, d represents the distance from the origin of the reference coordinate system to the plane; Randomly select three points in space as sampling points and substitute them into the parameterized model, as follows: a 1 x + b 1 y + c 1 z = d 1； Randomly select another point in space as a standard point and substitute it into the parameterized model, as follows: a 1 x 2+ b 1 y 2+ c 1 z 2= d 2； If the threshold distance ρ | d 1- d 2| is less than 0.2, the corresponding point is judged as an in-point, i.e. a point satisfying the target model, otherwise as an out-point. After dividing the four points in the above space into inliers and outliers, record the number of inliers, then reselect another three random sampling points and perform the steps of judging inliers and outliers, set the number of iterations, and after reaching the number of iterations, the model with the most inliers is selected as the optimal fitting of the actual target point cloud model, and the redundant point cloud is removed.

5. The method of claim 1, wherein, The point cloud filtering uses Euclidean clustering denoising filtering, specifically: KD-Tree is constructed in the point cloud coordinate system and the remaining points in the space are mounted in it after a one-time segmentation by random n ​ Initialize the point cloud, mark all points in space as unvisited state, and prepare for clustering; Traverse the point cloud, for each unvisited point, find its neighboring points by KD-Tree, and calculate the Euclidean distance between it and its neighboring points; for any two points in space γ i With γ j , calculate its Euclidean distance using the following formula: ； In the above formula, d γ i γ j represents γ i the Euclidean distance of γ j γ ix represents γ i the point x axis coordinate, γ iy represents γ i the point y axis coordinate, γ iz represents γ i the point z axis coordinate, γ j Similarly;​​​ The clustering and classification are performed, the Euclidean distances of the neighboring points are obtained by traversing the entire KD-Tree n d ( γ i - γ j ), and the Euclidean distances are compared with a set threshold r , and the points with the Euclidean distances less than the threshold are marked as the same class of point clouds;​ After clustering, the final optimized point cloud of the target to be grabbed is obtained.

6. The method of claim 1, wherein, The grasping pose detection algorithm based on point cloud local concave-convex constraint and point cloud pose constraint is specifically: Take the target point cloud as input, randomly sample multiple candidate grasping poses at multiple angles of the target point cloud; Use the force closure index, grasping screw space, point cloud local concave-convex constraint and point cloud pose constraint as constraint conditions for grasping pose, and perform pose screening; The force closure index is specifically that the resultant force and the resultant moment acting on the object are zero, and is used to measure whether the grasping is stable; The grasping screw space is a set of force screws of all contact points, and the force screw of the contact point is specifically that the force and the moment exerted on the object by the contact point when the object is grasped, which are combined into a force screw; when the grasping screw space can completely restrict the movement degree of freedom of the object to be grasped, the movement of the object is constrained; The point cloud local concave-convex constraint adopts a principal component analysis method to calculate the curvature of the point cloud, and is specifically as follows: to the grasp point cloud obtained by point cloud registration M any point within the inner m i , within its neighborhood to m i and its neighboring points m ij perform a covariance analysis: ； ； C i is a 3*3 covariance matrix, l is the number of points in the neighborhood, is the center of the point cloud, and are the eigenvalues and eigenvectors of C i respectively. Assume the eigenvalues satisfy Then is m i the normal vector of m i the neighborhood curvature of has: ； in, i =1,2,…, L , L Represents the number of sampling points; neighborhood curvature Used for measuring point clouds M The degree of curvature, A smaller value indicates a flatter neighborhood, and vice versa. The larger the value, the greater the curvature and undulation of the neighborhood; For the surface point cloud of the gripper grasping, the local concave-convex constraint is as follows: F ≤ ≤ G ； wherein, F and G are constants, F represents a lower bound of curvature of the grasped surface point cloud, G represents an upper bound of curvature of the grasped surface point cloud; The point cloud pose constraint is specifically as follows: After the point cloud registration, the pose of the target in the three-dimensional space is obtained, and the target pose is represented by a vector , , and the origin V 0, the origin of the gripper coordinate system a 0, and three orthogonal vectors are represented by , , respectively. The grasping sampling condition equation is as follows: ； in, μ It is a constant. τ The parameters set determine the size of the crawling range. a 1. a 2. a 3 represents the three orthogonal vectors of the gripper. , , The value, v 1. v 2. v 3 represent the target pose vectors. , , The value; The filtered candidate grasping pose is input into the grasping evaluation network, and all grasping poses are scored; The grasping pose with the highest score is output.

7. The method of claim 1, wherein, The dual-arm decoupling planning is specifically as follows: The initial pose and the target pose of the master arm and the slave arm are set; The path planning of the master arm or the slave arm is performed; Whether a collision occurs is judged: If a collision occurs, the slave arm path is discarded, the master arm trajectory information is added to the state validity judgment of the slave arm, and the path planning of the slave arm is re-performed; If no collision occurs, the path planning of the master arm and the slave arm is completed.

8. The method of claim 1, wherein, The robot movement planning based on the reward and punishment mechanism is specifically as follows: In the action space, the robot can choose a limited and discrete speed, the speed size is uniformly selected from the interval [0, V max ] and the heading angle Ω is uniformly distributed in [0, 2π]. The running speed of the robot in space is decomposed along the Cartesian coordinate system as [ V x , V y ], P g 、 P 、 are the current spatial positions of the robot, the target point and the pedestrian respectively, and their spatial positions are decomposed into X and Y axes in the Cartesian coordinate system. The running reward function is set as: S = S pass + S cross + S over ； ； where, S pass The running reward function for avoiding the exceeding phenomenon, S cross The running reward function for avoiding the crossing phenomenon, S over The running reward function for avoiding the passing phenomenon, is the distance between the robot and the target, the distance between the robot and the pedestrian is is the relative rotation angle between two robots when they meet, V x V y V x ' V y ' are the current velocities of two moving dual-arm robots in space, respectively, and the angle |Θ-Ω| is limited in [-π, π].​​​​ 9. An optimized system for object recognition, obstacle avoidance, and grasping of a dual-arm robot, characterized in that, The optimization method for dual-arm robot object identification and obstacle avoidance grasping is applied to any one of claims 1-8, comprising a diffusion network image repair module, a point cloud segmentation and filtering module, a registration identification module, a grasping pose detection module, a dual-arm motion path planning module, and a robot movement planning module; The diffusion network image repair module is used to collect object images to be grasped and repair based on a diffusion network and an image generation network, generate point clouds from the repaired images, and construct a target object point cloud model library; The point cloud segmentation and filtering module is used to generate point clouds from real-time collected object images to be grasped and perform point cloud segmentation and filtering to obtain filtered and optimized point cloud information; The registration identification module is used to perform point cloud registration on the filtered and optimized point cloud information and the target object point cloud model library to determine whether it is an object to be grasped; The grasping pose detection module is used to obtain the optimal grasping pose by using a grasping pose detection algorithm based on point cloud local concave-convex constraint and point cloud pose constraint; The dual-arm motion path planning module is used to plan the motion path of the robot dual arms according to the dual-arm decoupling planning, bidirectional search, and target point attractive force field path planning algorithm; The robot movement planning module is used to plan the movement of the robot in the working environment according to the reward and punishment mechanism.

Images