A multi-feature semantic primitive 6d pose estimation and grasping anomaly monitoring method

Abstract

The application provides a multi-feature semantic primitive 6D pose estimation and grasping anomaly monitoring method, first, a brand-new image-point cloud collaborative multi-feature semantic segmentation network is designed, the network extracts object texture information and the category probability of each pixel and point cloud features, then pixel-point collaborative fusion is carried out, so that the segmentation accuracy and speed are effectively improved. Then, a brand-new primitive feature extraction module is provided, the extracted features guarantee the rotation, translation and scale invariant characteristics, compared with the traditional descriptor, the application has higher robustness and stronger feature expression ability, and compared with other deep learning networks, the application is more efficient and light. Finally, the application also combines the multi-feature semantic primitive 6D pose estimation and the robot grasping anomaly monitoring, so that the precision and sensitivity of the robot grasping anomaly detection are effectively improved.

Description

Technical Field

[0001] This invention relates to the field of machine vision technology, specifically a method for 6D pose estimation and grasping anomaly detection using multi-feature semantic primitives. Background Technology

[0002] With the rapid development of intelligent manufacturing, various types of robots are being used in production, not only improving production efficiency but also effectively ensuring product quality. However, as production increases, the standby time and working frequency of robots inevitably cause wear and tear on various components, which can affect product quality and even damage the robot itself. Modern robots are equipped with vision systems, which help them identify and detect objects and estimate the pose information of targets in space, assisting them in completing grasping tasks. When the deviation between the robot's planned target pose and the camera's estimated target pose exceeds a threshold, the robot's operating state can be judged as unstable, and anomaly checks can be performed. Currently, many 6D pose estimation algorithms exist and are applied in different scenarios and tasks. However, they often rely on CAD models, resulting in poor generalization; and they are easily affected by external conditions such as occlusion, lighting changes, and cluttered backgrounds. Currently, there are two data formats: RGB and RGBD as input. Before depth sensor technology matured, the RGB data format was mainly used, but it is sensitive to texture and lighting changes; with the development of depth sensor technology, the RGBD format is becoming increasingly popular, as it can resist texture and lighting changes. Researchers are also exploring pose estimation methods that do not rely on CAD models, but the aforementioned challenges remain. Furthermore, the accuracy of pose estimation remains a significant challenge, and methods for identifying anomalies in robot grasping are cumbersome and cannot obtain anomaly information in real time. Summary of the Invention

[0003] To address the challenges of existing technologies, this application proposes a multi-feature semantic primitive 6D pose estimation and grasping anomaly detection method. This method combines 6D pose estimation with the detection of robot grasping anomalies, monitoring the robot's grasping state by judging the difference between the 6D pose from the intelligent vision system and the pre-set robot grasping pose. The 6D pose estimation network based on multi-feature semantic primitives achieves higher pose estimation accuracy, ensuring accurate detection of robot grasping anomalies.

[0004] This invention provides a method for 6D pose estimation and anomaly detection using multi-feature semantic primitives, comprising the following steps:

[0005] A method for 6D pose estimation and anomaly detection in grasping using multi-feature semantic primitives includes the following steps:

[0006] Step 1: Construct a multi-feature semantic segmentation network that combines images and point clouds to output the semantic primitives of the target object;

[0007] Step 2: Construct a primitive feature extraction module. Input the semantic primitive into the primitive feature extraction module, and then input the features into the DualSDF generator network to generate an initial coarse primitive shape. Extract the observed semantic primitive and the initial coarse primitive features through the primitive feature extraction module. Optimize the feature by iteratively updating to make the coarse primitive features closer to the semantic primitive of the observed instance.

[0008] Step 3: Calculate the 6D pose information of the object grasped by the robot, compare the pose calculated by the vision system with the preset robot end-effector pose to determine if there is a significant difference and whether the robot grasping is abnormal.

[0009] Furthermore, in step 1, Mask R-CNN is used to segment the target object in the scene into the enclosing region and the corresponding depth map, establishing the 3D coordinate information of the object in the camera coordinate system, i.e., the 3D point cloud. The 3D point cloud and the corresponding RGB image are input into the image-point cloud collaborative multi-feature semantic segmentation network, which outputs the semantic primitives of the target object. A set of parameterized spheres is used as semantic primitives to fuse the geometric and image features of the object to represent the object instance. Compared with traditional point clouds, it can effectively improve the speed of 6D pose estimation while reducing the number of estimation parameters.

[0010] Preferably, step 1 uses RGBD data as network input, and uses Mask R-CNN to crop out the target object and corresponding depth map in the scene. The network fusion uses ResNet101 as the backbone network and uses a feature pyramid network to construct a multi-scale feature pyramid. The newly constructed image-point cloud collaborative multi-feature semantic segmentation network includes two branches: one branch is composed of a 3D-GCN network, which learns and processes the geometric features and semantic information of the point cloud; the other branch is composed of a residual network and multiple Conv2d layers. This branch performs feature extraction on the cropped target object image, extracts information such as object texture and the class probability of each pixel, and then uses the class probability and image features as additional features to perform pixel-to-point collaborative fusion with the segmented point cloud features, and then outputs the semantic primitives of the object.

[0011] Furthermore, step 2, the primitive feature extraction module, includes a point cloud transformation matrix prediction layer and four point set abstraction layers. The primitive transformation matrix prediction layer predicts the transformation matrix of the input data to eliminate the rotation effect. Each point set abstraction layer has a multilayer perceptron and four feature propagation layers. The multilayer perceptron extracts features of local regions to obtain high-dimensional features, and the feature propagation layers output low-dimensional features. After the semantic primitive is input into the feature extraction module, the transformation matrix of the input data is first predicted and multiplied by the transformation matrix to obtain rotation invariance. Then, the primitive feature extraction module (FPIM) calculates the primitive features of the output observation instance.

[0012] Preferably, step 2 uses a multi-scale grouping method, which adaptively selects different radii and sampling numbers to group points according to the point density, thereby extracting multiple local patterns in the point set abstraction layer.

[0013] Furthermore, step 2 involves optimizing the coarse primitive to more closely resemble the semantic primitive of the object. The coarse primitive is input into the primitive feature extraction module, which outputs coarse primitive feature I. prim-freature-coarse Calculate the difference I between the two features. loss The formula is as follows:

[0014] I loss =||I prim-freature-coarse -I prim-freature-obj ||2+η,

[0015] The fine-tuning parameter η is a fixed value, which is continuously optimized to reduce I. loss This makes the generated rough primitives closer to the observed semantic primitives, ultimately yielding the optimized rough primitives.

[0016] Furthermore, in step 1, RGBD data is used as network input. The target object and its corresponding depth map in the scene are cropped out using Mask R-CNN. After cropping the depth map of the target object, the 2D-3D inverse projection method is used to calculate the 3D point cloud of the target object in the camera coordinate system. Then, the target object image is cropped to a uniform size, and data augmentation is performed on the image. This can increase the diversity and robustness of the data, thereby improving the model's ability to recognize images under different scenes and conditions, and improving the model's generalization ability.

[0017] Furthermore, the 2D-3D inverse projection method calculates the 3D point cloud of the target object in the camera coordinate system through the following steps:

[0018] 1) Calculate the pixel coordinates in the camera coordinate system based on the camera intrinsic parameter matrix K and the pixel coordinates (i, j) of the RGB image.

[0019] (X c Y c Z c ):

[0020]

[0021] 2) Obtain the depth value Z of each pixel based on the depth image D.

[0022] Z = D i，j

[0023] Among them, D i，j This means extracting the depth value of the corresponding position in the depth map based on the pixel coordinates (i, j) in the RGB image;

[0024] 3) Based on coordinates (X) c Y c Z c Using the depth value z, calculate the 3D coordinates (X, Y, Z) of the pixel in the camera coordinate system.

[0025]

[0026] 4) Finally, the three-dimensional coordinates P(X, -Y, -Z) are used as point cloud data.

[0027] Furthermore, in step 3, the center points of the optimized rough primitive shape and the semantic primitive of the observed instance are calculated respectively, the transformation between the two sets of center points is obtained, and the 6D pose information of the robot grasping the object is obtained. Then, the pose calculated by the vision system is compared with the preset robot end pose to determine whether there is a large difference and whether the difference exceeds the threshold, thereby monitoring abnormal information of robot grasping.

[0028] Preferably, determining the transformation between the two sets of center points specifically includes the following steps:

[0029] 1) Based on the observed set of semantic primitive centers: P = p i (i = 1, 2, ..., n), where n represents the number of points in the point set. The set of center points of the rough semantic primitive shape is: Q = q i (i = 1, 2, ..., n), calculate the centroid of each point set:

[0030]

[0031] 2) Then calculate the coordinates of the point set after removing the center:

[0032]

[0033] 3) Calculate the covariance matrix based on the coordinates of the two uncentered point sets:

[0034]

[0035] 4) Use SVD to decompose the covariance matrix:

[0036] H = USV T

[0037] Where U represents that the left singular vector matrix is ​​an orthogonal matrix, S represents that the singular value matrix is ​​a diagonal matrix, and V represents that the right singular vector matrix is ​​an orthogonal matrix.

[0038] 5) Finally, calculate the rotation matrix R, scale s, and translation vector t respectively:

[0039]

[0040] 6) The transformation between the last two sets of points can be represented by (R, t, s).

[0041] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0042] This application proposes a multi-feature semantic primitive 6D pose estimation and grasping anomaly detection method, which combines 6D pose estimation with the detection of robot grasping anomalies. By judging the difference between the 6D pose under the intelligent vision system and the preset robot grasping pose, the robot's grasping state is monitored. Among them, the 6D pose estimation network based on multi-feature semantic primitives has higher pose estimation accuracy, which provides a guarantee for accurately identifying robot grasping anomalies.

[0043] 1. This invention utilizes a set of parameterized spheres as semantic primitives, fusing object geometry and image features to represent object instances. Compared to dense point clouds, this method effectively improves the speed of 6D pose estimation while reducing the number of estimation parameters.

[0044] 2. This invention novelly constructs an image-point cloud collaborative multi-feature semantic segmentation network. This network extracts information such as object texture, the category probability of each pixel, and point cloud features, and then effectively improves the segmentation accuracy and speed through pixel-point collaborative fusion. Specifically, this invention uses Mask R-CNN to segment the target object in the scene into its enclosing region and corresponding depth map. Based on 2D-3D inverse projection, the 3D coordinate information of the object in the camera coordinate system is established. First, based on the camera intrinsic parameter matrix K and the pixel coordinates (i, j) of the RGB image, the pixel coordinates (X, J) in the camera coordinate system are calculated. c Y c Z c Then, based on the depth image D, obtain the depth value Z of each pixel, and then based on the coordinates (X... c Y c Z c The image and depth value Z are used to calculate the 3D coordinates (X, Y, Z) of a pixel in the camera coordinate system. Finally, the 3D coordinates P(X, -Y, -Z) are used as the point cloud data, i.e., the 3D point cloud. The 3D point cloud and the corresponding RGB image are input into the image-point cloud collaborative multi-feature semantic segmentation network, which outputs the semantic primitives of the target object.

[0045] 3. This invention novelly constructs a primitives feature-invariant module, whose extracted features guarantee invariance to rotation, translation, and scale. Compared to traditional descriptors, it exhibits higher robustness and stronger feature representation capabilities, and is more efficient and lightweight than other deep learning networks. Specifically, this invention first inputs the semantic primitives into the primitives feature extraction module to calculate primitive features, then inputs the features into a DualSDF generation network to generate an initial coarse primitive shape. This shape is then updated and optimized to more closely approximate the semantic primitives of the observed instance.

[0046] 4. This application is the first to combine 6D pose estimation based on multi-feature semantic primitives with robot grasping anomaly detection, effectively improving the accuracy and sensitivity of robot grasping anomaly detection. Specifically, this invention calculates the center points of the optimized coarse primitive shape and the semantic primitive of the observed instance, respectively, and determines the transformation between the two sets of center points to obtain the 6D pose information of the object grasped by the robot. Then, it compares whether there is a significant difference between the pose calculated by the vision system and the preset robot end-effector pose to determine whether the robot grasping is abnormal. Attached Figure Description

[0047] Figure 1 This is a schematic diagram of a 6D pose estimation and grasping anomaly detection method based on multi-feature semantic primitives proposed in this application;

[0048] Figure 2 This is a schematic diagram of a 6D pose estimation network framework based on multi-feature semantic primitives;

[0049] Figure 3 A schematic diagram of a multi-feature semantic segmentation network framework for image-point cloud collaboration;

[0050] Figure 4 This is a schematic diagram of the primitive feature extraction module. Detailed Implementation

[0051] The following description, in conjunction with the accompanying drawings and specific implementation methods, provides a more detailed explanation of the present invention: a 6D pose estimation and grasping anomaly detection method based on multi-feature semantic primitives.

[0052] This invention discloses a multi-feature semantic primitive 6D pose estimation and grasping anomaly detection method. This method can detect abnormal states in robot grasping by comparing the actual pose and expected pose of the object being grasped. The key step in accurately detecting abnormal states in robot grasping is the multi-feature semantic primitive 6D pose estimation network. This network uses RGBD input to extract image features, image category probabilities, and point cloud features, and then fuses these multiple features to output the semantic primitives of the object. Then, a primitive feature extraction module calculates the feature difference between the initial coarse primitives generated by the DualSDF network and the observed object semantic primitives. Optimization of these feature differences yields a precise generated shape, ultimately calculating the precise 6D pose of the object. By comparing and analyzing the preset values ​​at the robot's grasping end effector with the pose calculated by the intelligent vision system, it is determined whether the robot grasping is abnormal. The entire method framework is as follows: Figure 1 As shown, a 6D pose estimation network based on multi-feature semantic primitives is as follows: Figure 2 As shown, the image-point cloud collaborative multi-feature semantic segmentation network is as follows: Figure 3 As shown, the Primitive Feature Extraction Module (PFIM) structure is as follows: Figure 4 As shown, the specific steps of a multi-feature semantic primitive 6D pose estimation and grasping anomaly detection method are as follows:

[0053] Step 1: Using RGBD data as network input, the target object and its corresponding depth map are cropped from the scene using Mask R-CNN. The extraction network fusion uses ResNet101 as the backbone network and a feature pyramid network to construct a multi-scale feature pyramid. After cropping the depth map of the target object, a 2D-3D inverse projection method is used to calculate the 3D point cloud of the target object in the camera coordinate system. Then, the target object image is cropped to a uniform 192×192 size, and data augmentation is performed on the image. This increases the diversity and robustness of the data, thereby improving the model's ability to recognize images under different scenes and conditions, and enhancing the model's generalization ability.

[0054] The observed 3D point cloud and images cropped to 192×192 pixels are input into an image-point cloud collaborative multi-feature semantic segmentation network, which outputs extracted object semantic primitives. The newly constructed image-point cloud collaborative multi-feature semantic segmentation network consists of two branches: one branch, composed of a 3D-GCN network, learns and processes the geometric features and semantic information of the point cloud; the other branch, composed of a ResNet as a residual network connected to multiple Conv2d layers, extracts features from the cropped target object image, extracting information such as object texture and the class probability of each pixel. Then, the class probability and image features are used as additional features and fused pixel-by-pixel with the segmented point cloud features. The semantic point cloud features extracted by the 3D-GCN network are I.point-cloud The probability of the image category is P. image The extracted image features are I image And the fused semantic point cloud features are I image-cloud The specific relationship can be expressed by formula I. image-cloud =F(I point-cloud P image I image The F method uses the concat concatenation method. After obtaining the fused semantic features, it is mapped to a low-dimensional space, and then the complete semantic primitives are composed by selecting key primitives. The semantic primitive (r, x, y, z) representing the geometric shape of an object is composed of a set of parameterized spheres, each defined by its radius r and center coordinates (x, y, z). This allows for the compression and simplification of the geometry and enhances the expression of geometric features.

[0055] Step 2: Extract the semantic primitives of the objects into the primitives feature-invariant module, and then extract the features I. prim-freature-obj The input is fed into a DualSDF network to initially generate coarse primitives. The primitive feature extraction module includes a primitive transformation matrix prediction layer to predict the transformation matrix of the input data to eliminate the effects of rotation; four point set abstraction layers, each using a multi-scale grouping method; a multilayer perceptron in each layer for feature extraction of each local region; and four feature propagation layers, each using upsampling and concatenation to propagate high-level features to lower levels, which are then fused using an MLP. Therefore, after inputting primitive data, the transformation matrix is ​​first predicted, then local and global features are obtained through the four point set abstraction layers and concatenated into high-level features. Finally, the four feature propagation layers yield the final invariant features. Simultaneously, to optimize the coarse primitives to more closely resemble object semantic primitives, the coarse primitives are input into the primitive feature extraction module, which outputs coarse primitive feature I. prim-freature-coarse Calculate the difference I between the two features. loss The formula is as follows:

[0056] I loss =||I prim-freature-coarse -I prim-freature-obj ||2+η,

[0057] Wherein, the fine-tuning parameter η is a fixed value that is continuously optimized to reduce I. loss This makes the generated rough primitives closer to the observed semantic primitives, ultimately yielding the optimized rough primitives.

[0058] Step 3: Calculate the center points of the optimized coarse primitive and the extracted semantic primitive respectively, and determine the transformation between the two sets of center points to obtain the 6D pose information of the robot grasping the object. Specifically, the steps are as follows: First, based on the extracted semantic primitive center point set: P = p i (i = 1, 2, ..., n), the set of coarse primitive shape centers: Q = q i For each set of points (i = 1, 2, ..., n), calculate the centroid of each point set. Then, calculate the centerless coordinates of the point sets. Next, calculate the covariance matrix based on the two centerless coordinates and use SVD to decompose the covariance matrix. Finally, calculate the rotation matrix R, scale s, and translation vector t to obtain the corresponding 6D pose information of the object. Then, compare the calculated 6D pose information with the pre-set target position and pose, and calculate the difference between the two. If the difference exceeds a certain threshold, it indicates that the robot's grasping has a deviation or error, and an abnormal signal can be fed back to the system to indicate an abnormal grasping behavior.

[0059] This invention proposes a 6D pose estimation and anomaly detection method using multi-feature semantic primitives. The above embodiments are only for illustrating the technical concept and features of this invention, and are intended to enable those skilled in the art to understand the content of this invention and implement it accordingly. They should not be construed as limiting the scope of protection of this invention. All equivalent changes or modifications made in accordance with the spirit and essence of this invention should be covered within the scope of protection of this invention.

Claims

1. A method for 6D pose estimation and grasping anomaly detection using multi-feature semantic primitives, characterized in that, Includes the following steps: Step 1: Construct a multi-feature semantic segmentation network that combines images and point clouds to output the semantic primitives of the target object; Step 2: Construct a primitive feature extraction module. Input the semantic primitive into the primitive feature extraction module, and then input the features into the DualSDF generator network to generate an initial coarse primitive shape. Extract the observed semantic primitive and the initial coarse primitive features through the primitive feature extraction module. Optimize the feature by iteratively updating to make the coarse primitive features closer to the semantic primitive of the observed instance. Step 3: Calculate the 6D pose information of the object grasped by the robot, compare the pose calculated by the vision system with the preset robot end pose to determine whether the robot grasping is abnormal. The specific steps for determining the transformation between the two sets of center points are as follows: 1) According to point cloud 1: P = p i (i = 1, 2, ..., n), Point cloud 2: Q = q i (i = 1, 2, ..., n), where n represents the total number of points in the point set. Calculate the centroid of each point cloud. and 2) Then calculate the point cloud coordinates after removing the center from the point set: 3) Calculate the covariance matrix based on the two decentered point cloud coordinates: 4) Use SVD to decompose the covariance matrix: H=USV T Where U represents that the left singular vector matrix is ​​an orthogonal matrix, S represents that the singular value matrix is ​​a diagonal matrix, and V represents that the right singular vector matrix is ​​an orthogonal matrix. 5) Finally, calculate the rotation matrix R, scale s, and translation vector t respectively: The transformation between the last two sets of point clouds can be represented by (R, t, s).

2. The method for 6D pose estimation and grasping anomaly detection using multi-feature semantic primitives according to claim 1, characterized in that: Step 1 uses Mask R-CNN to segment the target object in the scene into the bounding region and the corresponding depth map, establishing the 3D coordinate information of the object in the camera coordinate system, i.e., the 3D point cloud. The 3D point cloud and the corresponding RGB image are input into the image-point cloud collaborative multi-feature semantic segmentation network, which outputs the semantic primitives of the target object. Through a set of parameterized spheres as semantic primitives, the geometric and image features of the object are fused to represent the object instance. Compared with traditional point clouds, it can effectively improve the speed of 6D pose estimation while reducing the number of estimation parameters.

3. The method for 6D pose estimation and grasping anomaly detection using multi-feature semantic primitives according to claim 2, characterized in that: Step 1 uses RGBD data as network input, and uses Mask R-CNN to crop out the target object and its corresponding depth map in the scene. The network fusion uses ResNet101 as the backbone network and uses a feature pyramid network to construct a multi-scale feature pyramid. The newly constructed image-point cloud collaborative multi-feature semantic segmentation network includes two branches: one branch consists of a 3D-GCN network, which learns and processes the geometric features and semantic information of the point cloud; the other branch consists of a residual network and multiple Conv2d layers. This branch performs feature extraction on the cropped target object image, extracts the object texture information and the class probability of each pixel, and then uses the class probability and image features as additional features to perform pixel-to-point collaborative fusion with the segmented point cloud features, and then outputs the semantic primitives of the object.

4. The method for 6D pose estimation and grasping anomaly detection using multi-feature semantic primitives according to claim 1, characterized in that: Step 2, the primitive feature extraction module, includes a point cloud transformation matrix prediction layer and four point set abstraction layers. The primitive transformation matrix prediction layer predicts the transformation matrix of the input data to eliminate the rotation effect. Each point set abstraction layer has a multilayer perceptron and four feature propagation layers. The multilayer perceptron extracts features of local regions to obtain high-dimensional features, and the feature propagation layers output low-dimensional features. After the semantic primitives are input into the feature extraction module, the transformation matrix of the input data is first predicted and multiplied by the transformation matrix to obtain rotation invariance. Then, the primitive feature extraction module calculates and outputs the primitive features of the observed instances.

5. The method for 6D pose estimation and grasping anomaly detection using multi-feature semantic primitives according to claim 4, characterized in that: Step 2 uses a multi-scale grouping method to adaptively select different radii and sampling numbers to group points based on point density, thereby extracting multiple local patterns in the point set abstraction layer.

6. The method for 6D pose estimation and grasping anomaly detection using multi-feature semantic primitives according to claim 4, characterized in that: Step 2 involves optimizing the coarse primitives to better approximate the semantic primitives of the objects. The coarse primitives are input into the primitive feature extraction module, which outputs coarse primitive feature I. prim-freature-coarse Calculate the difference I between the two features. loss The formula is as follows: I loss =||I prim-freature-coarse -I prim-feature-obj ||2+η, The fine-tuning parameter η is a fixed value, which is continuously optimized to reduce I. loss This makes the generated rough primitives closer to the observed semantic primitives, ultimately yielding the optimized rough primitives.

7. The method for 6D pose estimation and grasping anomaly detection using multi-feature semantic primitives according to claim 1, characterized in that: Step 1 uses RGBD data as network input. Mask R-CNN is used to crop out the target object and its corresponding depth map in the scene. After cropping the depth map of the target object, the 2D-3D inverse projection method is used to calculate the 3D point cloud of the target object in the camera coordinate system. Then, the target object image is cropped to a uniform size and data augmentation is performed on the image. This can increase the diversity and robustness of the data, thereby improving the model's ability to recognize images under different scenes and conditions and improving the model's generalization ability.

8. The method for 6D pose estimation and grasping anomaly detection using multi-feature semantic primitives according to claim 1, characterized in that, The specific process of calculating the 3D point cloud of the target object in the camera coordinate system using the 2D-3D inverse projection method includes the following steps: 1) Calculate the pixel coordinates (X, J) in the camera coordinate system based on the camera intrinsic parameter matrix K and the pixel coordinates (i, j) of the RGB image. c ,Y c Z c ): 2) Based on the depth image D, obtain the depth value Z of each pixel: Z=D i,j Among them, D i,j This means extracting the depth value of the corresponding position in the depth map based on the pixel coordinates (i,j) in the RGB image; 3) Based on coordinates (X) c ,Y c Z c Given the depth value Z, calculate the 3D coordinates (X, Y, Z) of the pixel in the camera coordinate system: 4) Finally, the three-dimensional coordinates P(X,-Y,-Z) are used as the point cloud data.

9. The method for 6D pose estimation and grasping anomaly detection using multi-feature semantic primitives according to claim 1, characterized in that: Step 3 calculates the center point of the optimized rough primitive shape and the semantic primitive of the observed instance, and finds the transformation between the two sets of center points to obtain the 6D pose information of the robot grasping the object. Then, it compares whether there is a large difference between the pose calculated by the vision system and the preset robot end pose, and judges whether the difference exceeds the threshold to monitor abnormal information of robot grasping.

Images