Universal six-degree-of-freedom grabbing pose detection method and system for robot

Abstract

The invention belongs to the technical field of AI vision, and particularly relates to a universal six-degree-of-freedom grabbing pose detection method and system for a robot. According to the detection method, the point cloud feature extraction process is optimized by introducing an attention mechanism, so that the prediction precision of the detection head on the six-degree-of-freedom grabbing pose is improved. The method comprises the following steps: firstly, carrying out serialization processing on unstructured, sparse and non-uniform point cloud data generated from an RGB-D image, and then designing a multi-encoder-decoder cascaded backbone network based on a Transform architecture so as to effectively extract rich features of the point cloud. On the basis, a network module special for six-degree-of-freedom grabbing pose detection is designed, and the predicted six-degree-of-freedom grabbing pose is optimized in combination with antipodal grabbing constraints, so that it is ensured that the six-degree-of-freedom grabbing pose conforms to actual physical rules. Through verification of a grabbing experiment on an actual Franka robot, the general grabbing pose detection performance of the method on any object in actual application is further proved.

Description

Technical Field

[0001] The present invention belongs to the technical field of AI vision, and particularly relates to a general six-degree-of-freedom grasping pose detection method and system for robots. Background Art

[0002] With the rise of embodied intelligence, the grasping pose detection technology based on visual data has become an important means to achieve autonomous grasping of robots in unknown environments. This technology uses visual sensors to perceive the environment around the robot and applies computer vision technology to detect the pose of the object grasped by the robot from the perceived data. Improving the accuracy of grasping pose detection has always been the goal pursued by researchers in this field. Traditional methods usually first predict the pose of the object in the image and then compare it with the object CAD model with grasping pose annotations to calculate the grasping pose of the object at this pose. However, these methods are mainly applicable to objects with known three-dimensional models and rely strongly on the accurate estimation of the object pose. Therefore, it is difficult to be widely applied to most real-world scenarios. In recent years, with the progress of RGB-D sensors and three-dimensional computer vision technology, data-driven grasping pose detection methods have gradually become a promising research direction in this field due to their strong generalization ability, especially when facing unknown scenarios and unknown objects.

[0003] Currently, data-driven grasping pose detection methods can be divided into rectangular box grasping pose detection methods and six-degree-of-freedom (6-DoF) grasping pose detection methods according to the representation of the grasping pose. The rectangular box grasping pose detection method determines the pose by detecting the grasping bounding box of the object. Since this method requires fewer grasping parameters to be predicted and usually has a higher processing speed, it is applicable to real-time application scenarios. In contrast, the 6-DoF grasping pose detection method provides more comprehensive grasping pose parameters by predicting the three-dimensional position and rotation angle of the gripper when grasping the object. This method has stronger adaptability than the rectangular box method and can handle almost all types of grasping scenarios.

[0004] However, when using the current six-degree-of-freedom grasping pose detection, problems such as low detection accuracy are likely to occur. As is well known, for data-driven methods, the accuracy and generality of the grasping pose detection model constructed and trained based on the grasping pose detection dataset (such as GraspNet-1Billion) largely depend on the quality of the network architecture. Similar to the goals of other deep learning models in pursuing high accuracy and high generality, the optimization of model performance involves multiple directions, including but not limited to the improvement of the network architecture, the optimization of the loss function, and the optimization of the training strategy, etc.

[0005] Regarding existing six-degree-of-freedom (6DOF) grasping pose detection methods, the vast majority of research mainly focuses on optimizing the grasping pose detection head, aiming to more accurately predict parameters such as the grasping position, grasping direction, and rotation angle from the extracted point cloud features through classification or regression. In addition, some research has proposed innovative tricks to optimize the predicted grasping parameters or select the optimal grasping pose from multiple predictions. Although these studies have made important contributions to the performance improvement of the grasping pose detection model, for the deep learning model to achieve better performance, it is crucial to construct a more efficient and powerful backbone network. Because an excellent backbone network is a prerequisite for the detection model to achieve high levels of performance in terms of accuracy, generality, and robustness.

[0006] Six-degree-of-freedom grasping pose detection based on computer vision technology is an effective way to solve the problem of robots grasping arbitrary objects. Most existing six-degree-of-freedom grasping pose detection methods utilize existing backbone networks for point cloud feature extraction. Their focus is on designing better detection heads or tricks to predict the six-degree-of-freedom grasping pose from point cloud features, but they neglect the importance of the backbone network in high-precision grasping pose detection. Summary of the Invention

[0007] The present invention proposes a general six-degree-of-freedom grasping pose detection method and system for robots, which solves the problem of low detection accuracy that easily occurs when using current six-degree-of-freedom grasping pose detection.

[0008] The technical solution of the present invention is implemented as follows:

[0009] A general six-degree-of-freedom grasping pose detection method and system for robots includes the following steps:

[0010] S1. Use a depth camera to capture RGB-D images, convert them into point clouds in the configured host computer, and perform preprocessing.

[0011] S2. Call the PSAGrasp detection method deployed in the host computer to perform grasping pose detection of the target object, and select the grasping pose with the highest confidence among all the predicted grasping poses as the target grasping pose of the robot; the PSAGrasp detection method includes point cloud feature extraction, six-degree-of-freedom grasping pose detection, and grasping pose optimization based on antipodal grasping constraints.

[0012] S3. Infer the actions of the robot at each time step from the initial position to the predicted grasping position. After the robot moves to the target position, drive the robot to adjust its posture and control the gripper to the predicted grasping width to perform object grasping.

[0013] S4. Continuously repeat S1 - S3 until no grasping pose can be predicted in the scene.

[0014] Optionally, in step S1, the preprocessing method is as follows: construct scene point cloud data using RGB-D images, and obtain the coordinates, colors, and normal vector information of these sampled points by randomly sampling N points from the constructed point cloud as the input of PSA Grasp.

[0015] Optionally, in step S2, the processing method for point cloud feature extraction is: a serial point cloud feature extraction backbone network cascaded with multiple encoders and decoders based on the Transformer architecture. This backbone includes N1 encoders with attention mechanisms and N2 cascaded decoders. Use the Z-order space filling curve to serialize the point cloud, and input the serialized point cloud data into the point cloud serialization feature extraction network cascaded with multiple encoders and decoders based on the Transformer architecture for feature extraction.

[0016] Optionally, in step S2, point cloud feature extraction includes point cloud serialization, and the processing method is as follows:

[0017] Set a fixed network size g size , and convert the three-dimensional real coordinates c in =(x, y, z) of the input point cloud into the spatial index coordinates c g of the grid where the point is located;

[0018]

[0019] Use the binary representation of the grid coordinates c g of each point for sequence encoding, and obtain the binary representation of each point sequence code as:

[0020] E(x g , y g , z g ) = B m B m-1 ...B2B1x d y d z d x d-1 y d-1 z d-1 ...x2y2z2x1y1z1;

[0021] where B i , x i , y i and z i are the batch size B, the grid coordinate value x g , y g and z gThe value of the i-th binary bit from the low bit to the high bit, m is the number of binary bits of the batch size, and d represents the number of binary bits corresponding to the maximum grid coordinate value in any coordinate dimension among N points.

[0022] Through the above technical solution,

[0023] Optionally, in step S2, point cloud feature extraction includes sequence pooling and sequence unpooling, and the processing method is as follows:

[0024] During the sequence pooling process, for the sequence code of any point, the following operations are performed:

[0025] E new (x g <<1, y g <<1, z g <<1) = E(x g , y g , z g ) << 3;

[0026] where E new represents the sequence code of the pooled point, and "<<" represents the left shift operation;

[0027] The three-dimensional real coordinates of the pooled point corresponding to the j-th cluster s j are:

[0028]

[0029] where (x i , y i , z i ) represents the three-dimensional real coordinates of the i-th point in the cluster s j , and |s j | represents the number of points in the cluster s j ;

[0030] The feature of the pooled point is obtained by performing a max pooling operation on the features of all points in the current cluster s j :

[0031]

[0032] where f i represents the feature of the i-th point in the cluster s j , and L prj ∈R C×C′ is a linear mapping layer;

[0033] During the sequence unpooling process, the feature values of all points in the cluster s j are not calculated by interpolation method, but directly copy the feature of the corresponding green point to each point in the cluster;

[0034] f i uppooling = f j , if (x i , y i , z i ) ∈ s j ;

[0035] f j represents the feature of point j before inverse pooling.

[0036] Optionally, in step S2, point cloud feature extraction includes serializing point cloud chunking, and the processing method is as follows:

[0037] The number of small chunks that the serialized point cloud can be divided into is:

[0038]

[0039] where p size represents the number of points contained in each small chunk. For the last small chunk that cannot be completely divided, the number of missing points is:

[0040] p deficit = p size - (N mod p size ).

[0041] Optionally, in step S2, point cloud feature extraction includes serializing point cloud multi-head attention, and the processing method is as follows:

[0042] Directly introduce a sparse convolutional layer with a skip connection for position encoding:

[0043] F = F + batch_norm(sparse_conv(F));

[0044] where F is the feature of the point cloud. The position encoding of the feature F of the point is performed through three-dimensional sparse convolution and batch normalization operations. The point cloud feature F itself contains the three-dimensional coordinate information of each point;

[0045] For each attention head i (i = 1, 2,..., H), the point cloud feature F is mapped to a feature vector through a linear mapping layer: query Q i , key K i and value V i ;

[0046]

[0047] where and are the weight matrices of the query, key, and value respectively, and the corresponding dimensions are For each head, its corresponding attention is as follows:

[0048]

[0049] The combined output of multi-head attention is:

[0050] MultiHead(F) = Concat(Head1, ……, Head H )W O .

[0051] Optionally, in step S2, the processing method for six-degree-of-freedom grasping pose detection is as follows:

[0052] Using the extracted point cloud features, GPD-Net predicts the objectness o and grasping score s of each point in the point cloud, and the cross-entropy loss L o is used for classifying object points, and the smooth L1 loss L g is used for regressing the grasping score:

[0053]

[0054] where and are the true labels of the corresponding objectness and grasping scores respectively, and o i , s i represent the objectness and grasping scores of the predicted point i respectively;

[0055] For grasping candidate points, the grasping direction score q of each grasping candidate point is predicted i , and regression is performed through the smooth L1 loss L e . The specific loss function is as follows:

[0056]

[0057] where is the true grasping direction score of the i-th grasping candidate point;

[0058] For the grasping width, regression is also performed using the smooth L1 loss;

[0059]

[0060] where s ij and w ij are the score and grasping width under the predicted j-th combination, while and are the corresponding true labels;

[0061] The total loss of the PC-Attention backbone and GPD-Net can be expressed as:

[0062] L GPD-Net = L o + L g + L e + λ·L rdw ;

[0063] Wherein, since L rdw is the cumulative sum of losses in the case of 48 sets of grasping parameters for each grasping candidate point.

[0064] Optionally, in step S2, the processing method for optimizing the grasping pose based on the antipodal grasping constraint is as follows:

[0065] During the actual grasping process, the antipodal grasping rule that two points (c1, c2) where the gripper contacts any object need to satisfy can be expressed as:

[0066]

[0067] Wherein, and respectively represent the unit tangent vectors of the contact points c1 and c2 with the object surface, while and represent the unit normal vectors of the contact points with the object surface;

[0068] Define a constraint function to quantify and penalize the grasping poses that do not satisfy the antipodal grasping rule:

[0069]

[0070] Since the antipodal grasping rule is independent of the length, the simplified equivalent constraint loss is:

[0071]

[0072] For and to determine the two vectors, the coordinates of the contact points c1 and c2 need to be solved first;

[0073] Take the contact point c1 as an example;

[0074]

[0075] Convert the above vectors in the world coordinate system into coordinate form as:

[0076]

[0077] Wherein, O c1 and G c1 respectively represent the coordinate values of the contact point c1 in the world coordinate system and the gripper coordinate system, OG represents the coordinate value of the G point in the world coordinate system, represents the rotation matrix of the gripper coordinate system relative to the world coordinate system.

[0078] Optionally, in step S2, in the grasping pose optimization based on the antipodal grasping constraint, another representation of the predicted six-degree-of-freedom grasping pose is g = [R, T, w], then it can be further transformed into:

[0079]

[0080] where R and T respectively represent the rotation and translation of the gripper relative to the world coordinate system, and w represents the grasping width;

[0081] To solve for the normal vector and construct a signed truncated distance function volume for the point cloud scene;

[0082]

[0083] The total loss of the PSAGrasp model technical network architecture is:

[0084] L total = L GPD-Net + βL antipodal ;

[0085] where the hyperparameter β is used to balance the importance of L GPD-Net and L antipodal of the two losses.

[0086] After adopting the above technical solution, the beneficial effects of the present invention are:

[0087] (1) The present invention proposes a point cloud serialization feature extraction network based on a multi-encoder and decoder cascade of the Transformer architecture. This network serializes the point cloud generated by the RGB-D image through a space-filling curve and efficiently extracts rich point cloud features by using positional encoding and multi-head attention mechanisms, laying a foundation for the grasping pose detection of any object.

[0088] (2) The present invention designs a six-degree-of-freedom grasping pose detection head (GPD-Net) and optimizes the grasping pose in combination with the antipodal grasping constraint, improving the existing six-degree-of-freedom grasping pose detection accuracy while avoiding predictions that may not conform to actual physical conditions in the existing methods.

[0089] (3) The present invention proposes a general six-degree-of-freedom grasping pose detection method for robots, PSAGrasp, which is evaluated on the GraspNet-1Billion benchmark and a real Franka robot, verifying the effectiveness and high performance of the PSAGrasp method.

[0090] (4) The general grasping pose detection method for robots, PSAGrasp, proposed by the present invention realizes the general grasping pose detection for any type and any object, covering the grasping pose detection of any household daily necessities, any industrial small objects, and any small objects with various shapes in life.

[0091] (5) The general grasping pose detection method for robots, PSAGrasp, proposed by the present invention includes three steps: point cloud feature extraction, six-degree-of-freedom grasping pose detection, and grasping pose optimization based on antipodal grasping constraints. Among them, point cloud feature extraction includes four steps: point cloud serialization, sequence pooling and sequence unpooling, serialized point cloud chunking, and serialized point cloud multi-head attention. The operation and formula of point cloud serialization can efficiently convert disordered, cluttered, sparse, and uneven point cloud data into an ordered one-dimensional sequence, while retaining the local spatial relationship between adjacent points, providing support for the extraction of object geometric structure features. The operations and formulas of sequence pooling and sequence unpooling can achieve downsampling of the point cloud without the need for adjacent point retrieval, which can greatly reduce the computational complexity of point cloud feature extraction and improve the six-degree-of-freedom grasping pose detection speed. The operation and formula of serialized point cloud chunking can increase the receptive field of point cloud feature extraction and achieve the precise extraction of the overall geometric structure information of objects in the scene. The operation and formula of serialized point cloud multi-head attention can learn multi-scale rich point cloud features and improve the extraction performance of object grasping area features. The operation and formula of six-degree-of-freedom grasping pose detection can achieve the precise detection of six-degree-of-freedom grasping pose parameters of the robot. The operation and formula of grasping pose optimization based on antipodal grasping constraints can optimize the detected six-degree-of-freedom grasping pose through physical grasping rules, avoid predicting grasping poses that do not meet the actual physical grasping conditions, and achieve general-level six-degree-of-freedom grasping pose detection performance. Brief Description of the Drawings

[0092] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0093] Figure 1 It is a schematic diagram of the technical architecture of a general six-degree-of-freedom grasping pose detection technology for robots in the embodiment;

[0094] Figure 2 It is a schematic diagram of sequence pooling and sequence unpooling operations in the embodiment;

[0095] Figure 3 It is a schematic diagram of serialized point cloud chunking operation in the embodiment;

[0096] Figure 4 It is a schematic diagram of multi - head attention operation in the embodiment;

[0097] Figure 5 It is a schematic diagram of coordinate solution of the grasping contact point in the base coordinate system and the antipodal grasping constraint in the embodiment;

[0098] Figure 6 It is a general six - degree - of - freedom grasping pose detection system of the robot built in the embodiment;

[0099] Figure 7 It is the visualization result of the real robot grasping in the embodiment. Detailed implementation manners

[0100] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0101] The embodiments of the present application disclose a general six - degree - of - freedom grasping pose detection method and system for a robot.

[0102] Embodiment

[0103] According to Figures 1 to 7 As shown, a general six - degree - of - freedom grasping pose detection method and system for a robot includes the following content.

[0104] 1. Related work

[0105] 1.1 Perception of the scene to be grasped

[0106] In the current target grasping scene perception, visual sensors and tactile sensors are two main types of perception devices, which respectively exhibit unique advantages in scene perception due to their functional differences. Visual sensors use devices such as cameras and Light Detection and Ranging (LiDAR) to obtain the position, shape, size, posture of objects and the structural information of the surrounding environment, thus providing a macroscopic scene perception for robot grasping. In contrast, tactile sensors can more finely perceive the structure and surface characteristics (such as smoothness, roughness, softness and hardness, etc.) of objects by recording the instantaneous force changes when contacting the objects, and provide richer physical feedback information for the stable grasping of the gripper. At the same time, combining the respective advantages of the two sensors to achieve visual-tactile fusion scene perception is also a current research hotspot.

[0107] The research goal of this invention is the grasping pose detection of target objects in an unstructured scene. Usually, this task is achieved by pure vision methods. The grasping pose detection method based on pure vision mainly relies on data formats such as RGB, RGB-D, and point cloud. Among them, using RGB images to perceive the scene to be grasped, although it can utilize powerful 2D computer vision technologies, due to the lack of 3D information in its data, it is usually only applicable to the grasping pose detection of flat surfaces with a fixed height. In contrast, RGB-D or point cloud data can provide explicit three-dimensional information and has become a research hotspot for the perception of the scene to be grasped in recent years. This invention uses a depth camera to obtain RGB-D images and converts them into point cloud data as input for model processing and analysis.

[0108] 1.2 Point cloud feature learning

[0109] In the field of 3D understanding, researchers have explored three methods for point cloud feature learning: projection-based methods, voxel-based methods, and point-based methods. Projection-based methods project point cloud data onto multiple viewpoints to generate 2D images and use highly mature 2D Convolutional Neural Networks (CNNs) to extract image features. This method is suitable for processing large-scale data, but some three-dimensional spatial information may be lost during the projection process. Voxel-based methods convert point clouds into three-dimensional voxel grids and effectively capture spatial structure information through dense voxels. Although they perform well in local feature extraction, they often face low computational efficiency and high memory consumption when dealing with sparse point clouds. In contrast, point-based methods directly process the original point cloud data, can accurately retain the geometric structure of the point cloud, and thus effectively capture the local and global features of objects. This invention adopts a point-based method for point cloud feature extraction in the grasping scene and attempts to combine the currently excellent Transformer architecture in order to more accurately extract the geometric structure and related features of objects in the grasping scene.

[0110] 1.3 Six-degree-of-freedom grasping pose detection

[0111] With the increasing demand for robots to grasp objects in arbitrary poses in unknown scenarios, six-degree-of-freedom (6-DOF) grasping pose detection based on RGB-D images or point cloud data has gradually become a popular research direction. Currently, 6-DOF grasping pose detection methods can be mainly divided into two categories: the two-stage strategy based on sampling-evaluation and the end-to-end single-step strategy. The grasping pose detection method based on the sampling-evaluation strategy usually includes two steps: first, a number of candidate grasping poses are uniformly sampled from the point cloud data, and then a neural network is used to score these candidate poses, and the grasping poses with high scores are selected from them. Due to the time-consuming of the grasping candidate sampling process and the performance being easily affected by objects (or parts) with a relatively thin shape, more and more researchers have tried to use the end-to-end strategy. The end-to-end grasping pose detection method directly predicts feasible grasping poses from the point cloud data through a network model, which is not only more efficient but also more convenient. Similarly, the present invention proposes an end-to-end point cloud serialized attention six-degree-of-freedom grasping pose detection method, which can directly predict feasible six-degree-of-freedom grasping poses from the input point cloud data.

[0112] 2. Detection method in the present invention

[0113] 2.1 Overall technical framework

[0114] In order to improve the 6-DOF grasping pose detection accuracy of objects in arbitrary poses in unknown scenarios, the present invention proposes a general 6-DOF grasping pose detection technology for robots (PSAGrasp), and its overall framework is as Figure 1 shown. This technology uses the RGB and depth images captured by an RGB-D camera to construct the scene point cloud data. By randomly sampling N points from the constructed point cloud, the coordinate, color, and normal vector information of these sampled points are obtained as the input of PSAGrasp. Aiming at the high-dimensional sparsity of the point cloud data, the present invention adopts the sparse tensor technology to convert the input feature f in of the point cloud into the sparse feature f s , thus effectively saving memory and improving the training and inference speed.

[0115] Given the crucial role of point cloud feature learning in grasping pose detection, the features of the extracted scene and target object have an important impact on the quality of grasping pose detection. Therefore, the present invention designs a serialized point cloud feature extraction backbone network (PC-Attention) based on the cascading of multiple encoders and decoders of the Transformer architecture. This backbone consists of N1 encoders with attention mechanisms and N2 cascaded decoders. Considering that neural networks based on the Transformer architecture require sequential format data input, the present invention uses the Z-order space filling curve to serialize the point cloud, and inputs the serialized point cloud data into the PC-Attention backbone network for feature extraction. At the same time, a six-degree-of-freedom grasping pose detection head (GPD-Net) is designed to predict the six-degree-of-freedom grasping pose of the object.

[0116] In addition, aiming at the problem that the network model may predict unrealistic grasping poses in the absence of physical law guidance, the present invention also designs an optimization strategy based on plantar grasping constraints to correct the predicted grasping poses, so as to ensure obtaining high-quality grasping poses that conform to two-finger gripper grasping.

[0117] 2.2 Point cloud serialization

[0118] Since the PC-Attention backbone network with attention mechanism requires the input data to be an ordered sequence, while point cloud data usually exists in the form of an unordered set of three-dimensional points, lacking a fixed order and structure, it is impossible to directly use the PC-Attention network for point cloud feature extraction. To solve this problem, the present invention serializes the scene point cloud composed of N randomly sampled points to generate an ordered sequence.

[0119] To facilitate point cloud serialization, the present invention sets a fixed network size g size , and converts the three-dimensional real coordinates c in =(x, y, z) of the input point cloud into the spatial index coordinates c g of the grid where the point is located.

[0120]

[0121] Using the binary representation of the grid coordinates c g of each point for sequence encoding, the binary representation of the sequence code of each point is obtained as:

[0122] E(x g , y g , z g ) = B m B m-1 ...B2B1xd y d z d x d-1 y d-1 z d-1 ...x2y2z2x1y1z1;

[0123] wherein, B i 、x i 、y i and z i are respectively the values of the i-th binary digit from the low order to the high order of the batch size B, the grid coordinate values x g 、y g and z g m is the number of binary digits of the batch size, and d represents the number of binary digits corresponding to the maximum grid coordinate value in any coordinate dimension of N points. The sequence code of each point is an integer of Int type (note that the sequence value should be ensured to be less than the maximum storage bit number 64 of the computer), and all points in the same grid have the same sequence code. By sorting these sequence code values in ascending order, the unordered point cloud data can be converted into an ordered point cloud sequence.

[0124] 2.3 Sequence Pooling and Sequence Unpooling

[0125] Traditional sampling-based pooling processes use techniques such as farthest point sampling or grid sampling to select encoded points. These methods generally use the KNN method to retrieve the neighboring points of the sampled points, and the retrieval efficiency is restricted by the number of sampled points, the retrieval range, etc. To improve the efficiency of point cloud pooling, the present invention proposes a more efficient sequence pooling and unpooling method based on serialized point clouds, as Figure 2 shown.

[0126] In the sequence pooling process, the present invention obtains the sequence code of the pooled point cloud by shifting the sequence code of the point cloud to be pooled left by 3 bits. Specifically, for the sequence code of any point, the following operations are performed:

[0127] E new (x g <<1,y g <<1,z g <<1) = E(x g ,y g ,z g )<<3;

[0128] According to the new sequence code, the point cloud to be pooled is divided into multiple clusters {s1, s2,... s j ,...}, where the red points with the same new sequence code form a cluster s j, the number of clusters is equal to the number of new sequence codes after deduplication. The present invention aggregates the information of all points within the same cluster to obtain a pooled point. The grid coordinates of the pooled point are (x g <<1,y g <<1,z g <<1), and its three-dimensional real coordinates are the mean of the three-dimensional real coordinates of all points within the cluster. For example, the three-dimensional real coordinates of the pooled point corresponding to the j-th cluster s j are:

[0129]

[0130] where (x i ,y i ,z i ) represents the three-dimensional real coordinates of the i-th point in the cluster s j , and |s j | represents the number of points in the cluster s j . The features of the pooled point are obtained by performing a max pooling operation on the features of all points in the current cluster s j :

[0131]

[0132] where f i represents the feature of the i-th point in the cluster s j . L prj ∈ R C×C′ is a linear mapping layer used to adjust the dimension of the features. For the pooled points, after sorting them according to the size of their corresponding sequence codes, a new serialized point cloud is obtained as the input for the subsequent process.

[0133] In the sequence unpooling process, the coordinates and sequence codes of the obtained points directly come from the coordinates and sequence codes of the points to be pooled recorded in the corresponding pooling operation. The feature values of all points within the cluster s j are not calculated by interpolation method, but directly copy the features of the corresponding green points to each point within the cluster.

[0134] f i uppooling = f j ,if(x i ,y i ,z i )∈s j ;

[0135] The present invention obtains the features of the points after unpooling in this way, considering the following two aspects: On the one hand, interpolation methods usually require retrieving the features of multiple neighboring points of the point to be interpolated, which will undoubtedly bring higher computational costs for a large number of points that need to be upsampled. On the other hand, during the pooling process, the point information within the cluster is aggregated into a single pooled point. When unpooling, the present invention believes that the features of this point should directly originate from this aggregated point without relying on the pooled point information of other clusters.

[0136] 2.4 Serialized Point Cloud Chunking

[0137] Dividing the serialized point cloud into small chunks of equal size is a crucial step in inputting into the attention module. The local information contained in each small chunk (such as the local features of the object surface) and the spatial relationship between small chunks (such as the relative positions between different objects) are crucial for point cloud feature extraction, as Figure 3 shown.

[0138] Through the chunking operation, the PC-Attention backbone network can focus on local features when processing each small chunk and combine this local information with global information through the self-attention mechanism. The number of small chunks that the serialized point cloud can be divided into is:

[0139]

[0140] where p size represents the number of points contained in each small chunk. For the last small chunk that cannot be completely divided, the number of missing points is:

[0141] p deficit = p size -(N mod p size );

[0142] The present invention supplements this incompletely divided small chunk by copying the missing p deficit points from the end of its adjacent small chunk to ensure that all small chunks have the same size. It should be noted that the size of the small chunk has a direct impact on the local receptive field and computational efficiency. Specifically, increasing the size of the small chunk can expand the local receptive field and reduce the total number of small chunks, thereby improving computational efficiency. However, an overly large small chunk may lead to the loss of local detailed information, thereby affecting the model's ability to capture fine structures. Therefore, selecting an appropriate small chunk size (p size ) is a key factor in balancing the local receptive field, computational efficiency, and detail retention.

[0143] 2.5 Serialized Multi-Head Attention

[0144] Position Encoding: Position encoding is crucial for the attention mechanism of point clouds, which can help the model distinguish the point cloud features at different spatial positions. Aiming at the problem that relative position encoding is complex and inefficient, the present invention adopts a more efficient conditional position encoding. Specifically, before the attention mechanism, the present invention directly introduces a sparse convolutional layer with skip connections for position encoding:

[0145] F = F + batch_norm(sparse_conv(F));

[0146] where F is the feature of the point cloud, and the position encoding of the point cloud is performed through three-dimensional sparse convolution and batch normalization operations. The point cloud feature F itself contains the three-dimensional coordinate information of each point. Therefore, the present invention directly uses the convolution operation used in the conditional position encoding to perform position encoding on the point cloud feature. This method effectively avoids the high time cost required for calculating relative positions using the three-dimensional coordinates of points in relative position encoding.

[0147] Multi-Head Attention: The multi-head attention of the present invention is constructed based on scaled dot-product attention. The input point cloud feature F ∈ R N×C , where C is the number of channels of the feature, as Figure 4 shown.

[0148] For each attention head i (i = 1, 2,..., H), the present invention maps the point cloud feature F to a feature vector through a linear mapping layer: query Q i , key K i , and value V i .

[0149]

[0150] where and are the weight matrices of the query, key, and value respectively, and the corresponding dimensions are For each head, its corresponding attention is:

[0151]

[0152] In summary, the combined output of the multi-head attention is:

[0153] MultiHead(F) = Concat(Head1, ……, Head H )W O ;

[0154] Use a linear transformation layer to fuse the point cloud information captured by different attention heads, so as to generate a richer representation of the point cloud features, where W O ∈R C×C is the weight parameter of this layer.

[0155] 2.6 Six-Degree-of-Freedom Grasp Pose Detection Network

[0156] For the point cloud features extracted by the PC-Attention backbone network, the present invention proposes a six-degree-of-freedom grasp pose detection head (GPD-Net) for predicting the grasp pose g = [p, e, r, d, w], where p is the three-dimensional coordinate of the grasp point, e is the grasp direction vector, r is the rotation angle of the gripper around the e-axis, d is the grasp distance along the e-axis, and w is the grasp width. Using the extracted point cloud features, GPD-Net predicts the objectness o and graspability score s for each point in the point cloud. The cross-entropy loss L o of the present invention is used for classifying object points, and the smooth L1 loss L g is used for regressing the graspability score:

[0157]

[0158] where and are the true labels of the corresponding objectness and graspability scores respectively. Next, from the classified object points, the present invention samples M grasp candidate points with graspability scores greater than a certain threshold using the farthest point sampling method. For these grasp candidate points, the grasp direction score q i of each grasp candidate point is predicted, and regression is performed through the smooth L1 loss L e . The specific loss function is as follows:

[0159]

[0160] where is the true grasp direction score of the i-th grasp candidate point. For the rotation angle and grasp distance, since the GraspNet-1Billion dataset only contains 48 possible combinations (the rotation angle is equally divided into 12 classes between 0° and 180°, and the grasp distance is discretized into 4 fixed values: 0.01 m, 0.02 m, 0.03 m, and 0.04 m), the present invention uses the smooth L1 loss function to predict the score of each combination. For the grasp width, the smooth L1 loss is also used for regression.

[0161]

[0162] where s ij and w ij are the score and grasp width predicted under the j-th combination, while and are the corresponding true labels. In summary, the total loss of the PC-Attention backbone and GPD-Net can be expressed as:

[0163] L GPD-Net = L o + L g + L e + λ·L rdw ;

[0164] Wherein, since L rdw is the cumulative sum of the losses in the case of 48 sets of grasping parameters for each grasping candidate point. Therefore, in order to balance the proportion of the losses predicted by different parameters in the total loss, the present invention sets the hyperparameter λ to a number less than 1, and through grid search experiments, it is found that when λ = 0.2, the predicted grasping pose is the most accurate.

[0165] 2.7 Grasping Pose Constraint

[0166] Figure 5 Solving the coordinates of the grasping contact point in the base coordinate system and the antipodal grasping constraint. Among them, the O coordinate system is defined as the world coordinate system, and the G coordinate system is the gripper coordinate system.

[0167] The RGB-D images obtained by the vision system may be affected by various factors such as illumination changes, shadows, reflections, and object surface textures, resulting in a large amount of visual noise and errors in the generated point cloud. In addition, the grasping poses directly detected by the network model sometimes do not conform to the physical laws of grasping in the real world, which makes the predicted grasping poses have a large gap in practical applications. Therefore, in order to effectively constrain the grasping poses predicted by the model, the present invention introduces the antipodal grasping rule to strongly constrain the generated grasping poses to satisfy force-closure grasping, as Figure 5 shown. For the two-finger parallel gripper robot studied in the present invention, during the actual grasping process, the antipodal grasping rule that the two points (c1, c2) where the gripper contacts any object need to satisfy can be expressed as:

[0168]

[0169] Wherein, and respectively represent the unit tangent vectors of the contact points c1 and c2 with the object surface, while and represent the unit normal vectors of the contact points with the object surface. The present invention uses this antipodal grasping rule to constrain the grasping poses output by GPD-Net, defines the corresponding constraint function, and quantifies and penalizes the grasping poses that do not satisfy the antipodal grasping rule:

[0170]

[0171] Since the antipodal grasping rule is independent of the length, the simplified equivalent constraint loss is:

[0172]

[0173] For and the determination of two vectors, the coordinates of the contact points c1 and c2 need to be solved first. Taking the contact point c1 as an example, the present invention uses the triangle rule of vectors to deduce its coordinates in the world coordinate system, as Figure 2 shown.

[0174]

[0175] Converting the above vectors in the world coordinate system into coordinate form is:

[0176]

[0177] Among them, and respectively represent the coordinate values of the contact point c1 in the world coordinate system and the gripper coordinate system, O G represents the coordinate value of point G in the world coordinate system, represents the rotation matrix of the gripper coordinate system relative to the world coordinate system. Another representation of the six-degree-of-freedom grasping pose predicted by the present invention is g = [R, T, w], then it can be further transformed into:

[0178]

[0179] Among them, R and T respectively represent the rotation and translation of the gripper relative to the world coordinate system, and w represents the grasping width. Therefore, using the above formula and the grasping pose predicted by GPD-Net, the coordinates of the contact point c1 can be deduced, and the coordinates of c2 can be obtained in the same way.

[0180] To solve the normal vectors and the present invention constructs a signed truncated distance function volume for the point cloud scene. By calculating the gradient partial derivative of the SDF value of each contact point, the normal vector of this point can be directly obtained Since the tangent vectors and are perpendicular to the corresponding normal vectors, they can be directly obtained after obtaining the normal vectors. In addition, the formula can also be further transformed to only retain the normal vector part:

[0181]

[0182] In summary, the total loss of the PSAGrasp technology network architecture of the present invention is:

[0183] L total = L GPD-Net+βL antipodal ;

[0184] Among them, the hyperparameter β is used to balance the importance of L GPD-Net and L antipodal The importance of the two losses. Through grid experiments, it is found that when β = 0.1, the performance of PSAGrasp reaches the optimal.

[0185] 2.8 Robot General Six-Degree-of-Freedom Grasp Pose Detection System

[0186] Figure 6 It is a robot general six-degree-of-freedom grasp pose detection system built. The robot model used in the robot grasping system of the present invention is Franka Emika Panda 7. The visual camera in the robot grasping system uses a depth binocular camera, and the model is Intel RealSense D435i. The general six-degree-of-freedom grasp pose detection method in the robot grasping system is deployed in a host computer, and the hardware configuration of the host computer is an NVIDIA RTX 3090 graphics card and an Intel i7-10700CPU@2.90GHz processor.

[0187] The host computer in the robot grasping system is installed with the Ubuntu 20.04 version of the system, and in order to achieve real-time control of the Franka Emika Panda robot, the host computer is installed with a real-time kernel version. In the Ubuntu20.04 real-time kernel version, the Noetic version of ROS is installed, and the libfranka and franka_ros libraries required to control the Franka Emika Panda robot are installed. On this basis, Moveit is installed and the panda_moveit_config function package for motion planning and grasping planning of the Franka Emika Panda robot is installed.

[0188] Since the robot grasping platform built in the present invention is a visual servoing one, when building the grasping platform based on the Franka Emika Panda robot, an Intel RealSense D435i depth binocular camera with depth perception is installed at the end effector of the Franka Emika Panda robot. To drive the RealSense D435i camera to take pictures, the realsense sdk is installed in the host computer. At the same time, in order to control the RealSense D435i camera in ROS, librealsense and realsense-ros are installed in the host computer. In order to obtain the pose of the installed RealSense D435i camera relative to the end effector of the Franka Emika Panda robot, the aruco_ros package is installed and the camera calibration work is carried out by using the "eye-in-hand" hand-eye calibration method. The hand-eye calibration matrix is solved by using OpenCV to obtain the 4×4 transformation matrix of the RealSense D435i camera relative to the end effector. Subsequently, this transformation matrix can be used to transform the grasping pose in the camera coordinate system predicted by the grasping pose detection method to the world coordinate system (i.e., the base coordinate system of the Franka Emika Panda robot).

[0189] The RGB-D images are taken by the RealSense D435i depth camera and converted into point clouds and preprocessed in the configured host computer.

[0190] The PSAGrasp method deployed in the host computer is called to detect the grasping pose of the target object, and the one with the highest confidence among all the predicted grasping poses is selected as the target grasping pose of the robot.

[0191] The actions of the real Franka Emika Panda robot from the initial position to the predicted grasping position at each time step are deduced, and these actions are executed in real time by calling the panda_moveit_config function package of MoveIt in Ros. After the Franka Emika Panda robot moves to the target position, the robot is driven to adjust its posture and control the gripper to the predicted grasping width to grasp the object. The above three steps are continuously repeated until no grasping pose can be predicted in the scene. This process realizes the robot intelligent grasping closed loop of "scene perception → grasping pose prediction → manipulator motion planning" in an unstructured environment.

[0192] 3 Experiments

[0193] 3.1 Experimental Setup

[0194] For all methods in the experiment, the datasets and evaluation metrics used for training and testing are from GraspNetbenchmark. The training set of the GraspNet-1Billion dataset contains 100 scenes, and each of the three test sets (Seen dataset, Similar dataset, and Novel dataset) contains 30 scenes. The evaluation metric AP used in the present invention μ represents the average prediction accuracy under different static frictions. The overall average accuracy AP is the average of [AP 0.2 , AP 0.4 , AP 0.6 , AP 0.8 , AP 1.0 in five cases.

[0195] 3.2 Detailed experimental conditions

[0196] The hardware resources for training PSAGrasp in the present invention are: Intel(R) Xeon(R) Gold 6330 cpu, NVIDIA A100 gpu. The Adam optimizer was used for training, and a total of 3600 training epochs were conducted. During the training process, the OneCycleLR learning rate scheduler was adopted to dynamically adjust the learning rate. Among them, the maximum learning rate was set to 0.001, as Figure 3 shown. The number of point clouds N = 20480 adopted from the original point cloud, and the network size g size was set to 0.005 meters when the point cloud coordinates were gridded, and the number of candidate grasps sampled from it was M = 1024.

[0197] 3.5 Real robot grasping

[0198] 3.5.1 Grasping robot configuration

[0199] The grasping experiment in the real scene was conducted on a Franka Emika robot, and all target objects were randomly placed within the grasping range of the robotic arm. The objects to be grasped were 55 types of objects selected from daily life that did not appear in the GraspNet-1Billion dataset. The grasping experiment system platform built in the present invention includes: a PC (Intel i7-10700 CPU@2.90GHz, Nvidia RTX 3090), a Franka robotic arm, a RealSense D435i depth camera, and a storage box, as Figure 6 shown.

[0200] 3.5.2 Visualization of real grasping experiments

[0201] To verify the effectiveness of PSAGrasp in the real scene, the present invention carried out single-object and multi-object grasping experiments. Figure 7Shows the visualization results of a set of multi-object grasping experiments, where each row represents the entire process of a single object in the scene from perception, to grasping, and then to placement. For each scene to be grasped, before each grasping, first, a depth camera is used to capture the scene and construct a point cloud; then, the point cloud is input into a host computer equipped with the PSAGrasp method for grasping pose prediction; secondly, the Top 1 grasping pose with the highest confidence is selected from the predicted grasping poses, and this grasping pose is transformed into the robot coordinate system using coordinate transformation, and the transformed grasping pose is used as the grasping pose to be executed by the robot; finally, the robot control instruction is called to drive the gripper to reach this pose to implement the grasping of the object. From Figure 6 It can be observed that PSAGrasp can detect the grasping poses of objects placed arbitrarily in the scene and can provide accurate grasping poses for the grasping of real robots.

[0202] 3.5.3 Quantitative Comparison of Real Grasping Experiments

[0203] Table 1 Performance Comparison of Grasping Poses Detected by Different Methods for Robots

[0204]

[0205] To quantitatively analyze the effectiveness of PSAGrasp in real robot grasping, the present invention is compared with some representative grasping pose detection methods, and the results are shown in Table 1. For single-object scenes, each method grasps 33 objects in arbitrary poses. For multi-object scenes, each method tests 30 scenes (note that the present invention can only ensure that the types and numbers of objects tested by each method are the same, but it is impossible to keep the object poses consistent in 5 experiments). The experimental results show that the robot grasping platform configured with the trained PSAGrasp shows better performance, with a success rate of 87.88% in single-object grasping and a probability of successfully grasping all objects in the scene of 80% in multi-object grasping, showing better performance than other methods.

[0206] In summary, the present invention draws on the excellent Transformer network architecture in the fields of natural language processing and computer vision, proposes a serialized point cloud feature extraction backbone network with cascaded multi-encoders and decoders, and based on this, designs a general six-degree-of-freedom grasping pose detection method for robots (PSAGrasp). The present invention serializes the point cloud scene constructed from RGB-D images using a space-filling curve, and then uses the PC-Attention backbone network to perform serialized chunking, positional encoding, and multi-head attention mechanism processing on the serialized point cloud data to extract rich point cloud features.

[0207] To accurately detect the six-degree-of-freedom (6-DOF) grasping pose parameters, the present invention designs a six-degree-of-freedom grasping pose detection head (GPD-Net). However, considering that GPD-Net may generate unrealistic grasping pose predictions without the guidance of physical laws, the present invention further designs an antipodal grasping constraint to optimize the predicted grasping poses and ensure the generation of high-quality poses that meet the requirements of actual two-finger robot grasping. To verify the effectiveness of the proposed method, the present invention compares with a variety of existing state-of-the-art six-degree-of-freedom grasping pose detection methods on the GraspNet-1Billion benchmark. Experimental results show that the proposed PSAGrasp method exceeds the current state-of-the-art methods by 9.56%, 5.22%, and 4.49% on the "seen", "similar", and "novel" test sets, respectively. In addition, through the grasping experiment verification on a real Franka robot, the results show that the detection method proposed by the present invention can exhibit excellent grasping pose detection performance when deployed in an actual scenario.

[0208] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the technical scope of the present invention shall be included in the protection scope of the present invention.

Claims

1. A general six-degree-of-freedom grasping pose detection method for robots, characterized in that, Specifically, it includes the following steps: S1. Use a depth camera to capture RGB-D images, convert them into point clouds in the configured host computer, and perform preprocessing. S2. Call the PSAGrasp detection method deployed in the host computer to detect the grasping pose of the target object, and select the one with the highest confidence among all the predicted grasping poses as the target grasping pose of the robot. The PSAGrasp detection method includes point cloud feature extraction, six-degree-of-freedom grasping pose detection, and grasping pose optimization based on antipodal grasping constraints. S3. Infer the actions of the robot at each time step from the initial position to the predicted grasping position. After the robot moves to the target position, drive the robot to adjust its posture and control the gripper to the predicted grasping width to grasp the object. S4. Continuously repeat S1 - S3 until no grasping pose can be predicted in the scene.

2. The general six-degree-of-freedom grasping pose detection method for a robot according to claim 1, wherein In step S1, the preprocessing method is as follows: Use the RGB-D image to construct the scene point cloud data, and randomly sample N points from the constructed point cloud to obtain the coordinate, color, and normal vector information of these sampled points as the input of PSAGrasp.

3. A general six-degree-of-freedom grasping pose detection method for a robot according to claim 1, characterized in that, In step S2, the processing method of point cloud feature extraction is as follows: A serialized point cloud feature extraction backbone network cascaded by multi-encoders and decoders based on the Transformer architecture. The backbone includes N1 encoders with attention mechanisms and N2 cascaded decoders. Use the Z-order space filling curve to serialize the point cloud, and input the serialized point cloud data into the point cloud serialization feature extraction network cascaded by multi-encoders and decoders based on the Transformer architecture for feature extraction.

4. A general six-degree-of-freedom grasping pose detection method for a robot according to claim 1, characterized in that In step S2, point cloud feature extraction includes point cloud serialization, and the processing method is as follows: Set a fixed network size g size , and convert the three-dimensional real coordinates c in = (x, y, z) of the input point cloud into the spatial index coordinates c g of the grid where the point is located; Using the binary representation of the grid coordinates c of each point g for sequence encoding, the binary representation of each point sequence code is obtained as follows: E(x g ,y g ,z g ) = B m B m-1 ...B2B1x d y d z d x d-1 y d-1 z d-1 ...x2y2z2x1y1z1; Among them, B i , x i , y i and z i are respectively the values of the i-th binary bit from the low bit to the high bit of the batch size B, the grid coordinate values x g , y g and z g , m is the number of binary digits of the batch size, and d represents the number of binary digits corresponding to the maximum grid coordinate value in any coordinate dimension among the N points.

5. A general six-degree-of-freedom grasping pose detection method for a robot according to claim 4, characterized in that, In step S2, point cloud feature extraction includes sequence pooling and sequence unpooling, and the processing method is as follows: During the sequence pooling process, for the sequence code of any point, perform the following operations: E new (x g <<1,y g <<1,z g <<1) = E(x g ,y g ,z g )<<3; Among them, E new represents the sequence code of the pooled points, and "<<” represents the left shift operation; The j-th cluster s j The three-dimensional real coordinates of the corresponding pooling point are as follows: Among them, (x i , y i , z i ) represents the three-dimensional real coordinates of the i-th point in the cluster s j , and |s j | represents the number of points in the cluster s j ; Pooled point The feature of j is obtained by performing a max pooling operation on the features of all points in the current cluster s: Among them, f i represents the feature of the i-th point in the cluster s j , and L prj ∈R C×C′ is a linear mapping layer; During the sequence unpooling process, for cluster s j the eigenvalues of all points within it are not calculated by interpolation method, but the features of the corresponding green points are directly copied to each point within the cluster; f j Represents the feature of point j before inverse pooling.

6. A general six-degree-of-freedom grasping pose detection method for a robot according to claim 1, characterized in that, In step S2, point cloud feature extraction includes serialized point cloud chunking, and the processing method is as follows: The number of small chunks that the serialized point cloud can be divided into is: Among them, p size represents the number of points included in each small block. For the last small block that cannot be completely divided, the missing number of points is: p deficit = p size - (N mod p size )。 7. A general six-degree-of-freedom grasping pose detection method for a robot according to claim 1, characterized in that, In step S2, point cloud feature extraction includes serialized point cloud multi-head attention, and the processing method is as follows: Directly introduce a sparse convolutional layer with skip connections for position encoding: F = F + batch_norm(sparse_conv(F)); Among them, F is the feature of the point cloud. The position encoding of the feature F of the point is performed through three-dimensional sparse convolution and batch normalization operations. The point cloud feature F itself contains the three-dimensional coordinate information of each point. For each attention head i (i = 1, 2, …, H), the point cloud feature F is mapped to feature vectors: query Q, key K, and value V through a linear mapping layer; i key K i and value V i ; Among them, and are the weight matrices for queries, keys, and values, respectively, with corresponding dimensions For each head, its corresponding attention is: The combined output of multi-head attention is: MultiHead(F) = Concat(Head1,……,Head H )W O .

8. A general six-degree-of-freedom grasping pose detection method for a robot according to claim 1, characterized in that In step S2, the processing method of six-degree-of-freedom grasping pose detection is as follows: Using the extracted point cloud features, GPD-Net predicts the objectness o and the grasping score s for each point in the point cloud, and the cross-entropy loss L o Classify the object points and use the smooth L1 loss L g Perform regression on the grasping score: Among them, and are the true labels of the corresponding objectness and graspability scores, respectively. i and i represent the objectness and graspability scores of the predicted point i, respectively. For the grasping candidate points, predict the grasping direction score q for each grasping candidate point i , and perform regression through the smooth L1 loss L e . The specific loss function is as follows: Among them, is the true grasping direction score of the i-th grasping candidate point; For the grasping width, the smooth L1 loss is also used for regression. where s ij and w ij are the score and the grasping width under the j-th predicted combination, while and are the corresponding true labels; The total loss of the PC-Attention backbone and GPD-Net can be expressed as: L CPD-Net = L o + L g + L e + λ·L rdw ; Among them, since L rdw is the cumulative sum of losses in the case of 48 sets of grasping parameters for each grasping candidate point.

9. A general six-degree-of-freedom grasping pose detection method for a robot according to claim 1, characterized in that, In step S2, the processing method of grasping pose optimization based on antipodal grasping constraints is as follows: During the actual grasping process, the antipodal grasping rule that the two points (c1, c2) where the gripper contacts any object need to satisfy can be expressed as: Among them, and respectively represent the unit tangent vectors of the contact points c1 and c2 with the object surface, while and represent the unit normal vectors of the contact point with the object surface; Define a constraint function to quantify and penalize grasping poses that do not satisfy the antipodal grasping rule: Since the antipodal grasping rule is independent of the length, the equivalent constraint loss after simplification is: For and the determination of two vectors, it is necessary to first solve the coordinates of the contact points c1 and c2; Take the contact point c1 as an example; Convert the above vector in the world coordinate system into the form of coordinates as: wherein, and respectively represent the coordinate values of the contact point c1 in the world coordinate system and the gripper coordinate system, O G represents the coordinate value of point G in the world coordinate system, represents the rotation matrix of the gripper coordinate system relative to the world coordinate system.

10. A general six-degree-of-freedom grasping pose detection method for a robot according to claim 9, characterized in that, In step S2, in the grasping pose optimization based on the antipodal grasping constraint, if another representation of the predicted six-degree-of-freedom grasping pose is g = [R, T, w], it can be further transformed into: where R and T respectively represent the rotation and translation of the gripper relative to the world coordinate system, and w represents the width of the grasp; To solve for the normal vector and Construct a signed truncated distance function volume for the point cloud scene; The total loss of the PSAGrasp technology network architecture is: L total = L GPD-Net + βL antipodal ; Among them, the hyperparameter β is used to balance the importance of L GPD-Net and L antipodal between the two losses.

Images