A pose estimation method for robot depalletizing and grasping based on image segmentation model

Abstract

The present invention discloses a robot depalletizing grasping posture estimation method based on an image segmentation model, which relates to the field of image processing technology. The method mainly comprises the following steps: using a two-dimensional detection frame as a positioning basis to input an image segmentation model, generating a pixel-level mask of the grasped target and mapping it to a three-dimensional point cloud aligned with the mask, and extracting a point cloud subset of the grasped target based on the mapping result; performing normal vector estimation and directional consistency processing on the point cloud subset, and dividing the point cloud clusters corresponding to each independent surface of the grasped target based on directional consistency as the clustering segmentation basis; calculating the center of mass position of each independent surface of the grasped target based on the divided point cloud clusters, and extracting a local point cloud within a preset radius of the center of mass; using the average point of the local point cloud as the grasping point, and the average normal vector of the local point cloud as the grasping direction, to generate the robot grasping posture. According to the present invention, when the object is tilted, stacked tightly, or has an irregular shape, the point cloud subset mapped by the mask can still accurately exclude interference from the background and adjacent objects.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a robot depalletizing and grasping posture estimation method based on an image segmentation model. Background Art

[0002] In the field of robotic automated depalletizing, existing methods typically rely on 3D sensors to acquire scene point cloud data. Object detection models (such as YOLO and Faster R-CNN) are then used to locate the object to be grasped within the RGB image. The corresponding 3D point cloud region is then captured based on the detection bounding box to calculate the grasping pose. However, this technical approach has significant drawbacks.

[0003] First, the limited accuracy of the detection box leads to a significant problem of point cloud contamination. When using a standard axis-aligned detection box, the extracted point cloud often contains a large number of background points or adjacent object points because it cannot fit the actual contours of tilted, irregularly shaped, or tightly stacked objects. This contaminated point cloud directly interferes with subsequent center of mass calculation and pose estimation. For example, when calculating the center of mass of a box, the presence of contaminated side or pallet point clouds can cause it to deviate significantly from the true center of the target grasping surface (such as the top surface).

[0004] Secondly, the introduction of rotating detection frames does not fundamentally solve the problem. Although rotating frames can partially improve object framing accuracy, they significantly increase the complexity and training difficulty of the detection model. For non-rigid objects such as soft packages and deformable bags, rotating frames still have difficulty accurately describing the contours, and the prediction error of the rotation angle further amplifies the deviation of point cloud extraction. More importantly, the algorithm framework that relies on high-precision detection frames cannot adapt to the complex conditions such as random stacking and partial occlusion common in logistics scenarios.

[0005] Finally, existing methods lack targeted processing for multi-planar targets. When grasping box-like objects, directly calculating a single center of mass for a mixed surface point cloud (e.g., one that includes both the top and side surfaces) can cause the grasping point to deviate from the intended area due to interference from point clouds of non-target surfaces. For example, the inclusion of side point clouds may cause the grasping point to deviate toward the edge of the box rather than the center of the top surface, leading to unstable grasping or collision risks. This problem is particularly prominent in densely stacked scenarios, and existing technologies have not yet provided an effective surface separation mechanism. Summary of the Invention

[0006] In order to improve the accuracy of robot depalletizing and reduce recognition bias and grasping point offset, the present invention proposes a robot depalletizing grasping pose estimation method based on an image segmentation model, comprising the following steps:

[0007] S1: Obtain the scene RGB image and the 3D point cloud data registered with it, and use the object detection model to extract the axis-aligned 2D detection box of the grasped object in the RGB image;

[0008] S2: Using the 2D detection box as the positioning basis, the image segmentation model is input to generate a pixel-level mask of the grasped object and map it to the 3D point cloud aligned with it. Based on the mapping result, a point cloud subset of the grasped object is extracted;

[0009] S3: Perform normal vector estimation and directional consistency processing on the point cloud subset, and use directional consistency as the clustering segmentation basis to divide the point cloud clusters corresponding to each independent surface of the grasped target;

[0010] S4: Calculate the centroid position of each independent surface of the grasped target based on the divided point cloud clusters, and extract the local point cloud within the preset radius of the centroid;

[0011] S5: The average point of the local point cloud is used as the grasping point, and the average normal vector of the local point cloud is used as the grasping direction to generate the robot grasping pose.

[0012] Furthermore, in the step S3, the direction consistency processing is specifically as follows:

[0013] The normal vector direction is unified through the tangent plane normal consistency orientation algorithm, so that the normal vector direction in the point cloud area belonging to the same continuous surface is consistent.

[0014] Furthermore, in the step S3, the cluster segmentation adopts a density-based noise space clustering algorithm, and divides the independent surface corresponding point cloud clusters using the normalized normal vector as a feature.

[0015] Furthermore, the step S3 further includes the following steps:

[0016] S31: Denoising the extracted point cloud subset using a statistical outlier removal algorithm.

[0017] Furthermore, in step S4, the centroid position is obtained by the following formula:

[0018]

[0019] Where, is the spatial three-dimensional coordinate of the L-th independent surface centroid position, is the number of points in the point cloud cluster corresponding to the Lth independent surface, is the spatial three-dimensional coordinate of the i-th point in the point cloud cluster corresponding to the L-th independent surface.

[0020] Furthermore, in the step S4, the preset radius range is adaptively adjusted according to the size of the grasped target, and the formula is expressed as:

[0021]

[0022] Where, is the preset radius range, is the minimum side length of the 3D bounding box of the current point cloud cluster, is the preset scale factor.

[0023] Furthermore, in the step S5, the grasping point and the grasping direction are obtained by the following formula:

[0024]

[0025] Where G is the three-dimensional coordinate of the grasping point, is the number of points in the local point cloud, is the spatial three-dimensional coordinate of the j-th point in the local point cloud, is the grasping direction, for The corresponding unit normal vector.

[0026] Furthermore, in the step S4, after obtaining the centroid position, the step of performing validity verification on each divided point cloud cluster is further included, and the formula is expressed as:

[0027]

[0028] Where, is the plane fitting residual of the Lth independent surface, is the average normal vector of the Lth independent surface, is the spatial three-dimensional coordinate of the L-th independent surface centroid position, is the spatial three-dimensional coordinate of the i-th point in the point cloud cluster corresponding to the L-th independent surface.

[0029] Furthermore, in step S5, for each grasping point of the independent surface local point cloud, the method further includes the steps of calculating the stability score of each grasping point and selecting the grasping point with the highest stability score. The stability score calculation formula is:

[0030]

[0031] Where, For stability score, is the weight coefficient, is the grasping direction, is the unit vector of gravity direction, is the density of the local point cloud.

[0032] Furthermore, in the step S2, the image segmentation model adopts SAM, and the detection box is used as a prompt input into its mask decoder to generate a pixel-level mask.

[0033] Compared with the prior art, the present invention has at least the following beneficial effects:

[0034] (1) This paper proposes a method for estimating the pose of a robot depalletizing and grasping object based on an image segmentation model. This method uses a common axis-aligned detection frame to guide the image segmentation model to generate a pixel-level mask, completely eliminating the reliance on a high-precision rotation frame. This design makes the algorithm compatible with any general detector. When objects are tilted, stacked tightly, or irregularly shaped, the point cloud subset mapped by the mask can still accurately exclude interference from background and adjacent objects.

[0035] (2) To address the industry challenge of multi-plane target capture, a three-level guarantee mechanism has been constructed, including normal vector consistency processing, normal vector clustering and segmentation, and surface verification, to achieve precise separation of the top and side surfaces of the box. The direction consistency algorithm eliminates the confusion of normal vectors within a single plane, and combines clustering with dynamic verification of plane fitting residuals to effectively solve the problem of surface fragmentation caused by occlusion or deformation.

[0036] (3) In terms of anti-interference and adaptive capabilities, multiple technical barriers are adopted: the statistical outlier removal algorithm significantly suppresses sensor noise and improves the reliability of normal vector estimation; the adaptive radius mechanism based on the minimum side length of the point cloud cluster bounding box ensures that effective local point clouds can be extracted on horizontal, vertical and inclined surfaces; the pose calculation strategy of focusing on the local point cloud near the center of mass successfully avoids edge noise interference, and the measured grasping stability is improved;

[0037] (4) When there are multiple visible surfaces for the same grasping target, the feasibility of grasping each surface is quantitatively evaluated through a stability scoring model, and the surface with the best perpendicularity between the normal vector and the direction of gravity and the highest point cloud density is given priority (for example, when stacking boxes, the obscured top surface is automatically avoided and the complete side surface is selected), so as to improve the adaptability of single target grasping in complex stacking scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 This is a step diagram of a robot depalletizing grasping pose estimation method based on an image segmentation model;

[0039] Figure 2 Schematic diagram of the original point cloud;

[0040] Figure 3 is a schematic diagram of the target point cloud;

[0041] Figure 4 Schematic diagram of multi-plane target point cloud;

[0042] Figure 5 Schematic diagram of cluster segmentation based on normal vector. DETAILED DESCRIPTION

[0043] The following are specific embodiments of the present invention and the accompanying drawings to further describe the technical solutions of the present invention, but the present invention is not limited to these embodiments.

[0044] In automated warehousing and logistics depalletizing systems, mainstream technical solutions typically rely on the collaborative processing of 3D point cloud data and object detection models. First, detectors such as YOLO and Faster R-CNN are used to generate a 2D detection bounding box of the target object in the RGB image. The corresponding 3D point cloud region is then captured based on the bounding box to calculate the grasping pose. However, this technical approach has significant limitations in real-world industrial scenarios. First, the accuracy of the bounding box's contour severely restricts point cloud quality. When processing tilted boxes or deformed soft packages, standard axis-aligned bounding boxes introduce a large amount of background and adjacent object point clouds. While rotating bounding boxes partially improves alignment, the increased model complexity and angular prediction errors make implementation difficult. Second, mixed point clouds of multi-planar objects can cause grasp point offsets. Point clouds of box-like objects often include both top and side surfaces. Directly calculating the center of mass can cause the grasp point to drift toward non-target surfaces (such as side edges). Especially when boxes are stacked tightly together, the grasp point offset can reach centimeters, significantly increasing the risk of grasp failure and stack collapse. Third, existing methods lack an adaptive decision-making mechanism. They are unable to dynamically select the optimal grasping surface based on surface integrity (e.g., automatically switching to the side when the top surface is blocked). They also have difficulty coordinating the grasping order of multiple targets in dense stacks, resulting in insufficient system robustness. To address the above industry pain points, the present invention proposes a robot depalletizing grasping pose estimation method based on an image segmentation model, which includes the following steps:

[0045] S1: Obtain the scene RGB image and the 3D point cloud data registered with it, and use the object detection model to extract the axis-aligned 2D detection box of the grasped object in the RGB image;

[0046] S2: Using the 2D detection box as the positioning basis, the image segmentation model is input to generate a pixel-level mask of the grasped object and map it to the 3D point cloud aligned with it. Based on the mapping result, a point cloud subset of the grasped object is extracted;

[0047] S3: Perform normal vector estimation and directional consistency processing on the point cloud subset, and use directional consistency as the clustering segmentation basis to divide the point cloud clusters corresponding to each independent surface of the grasped target;

[0048] S4: Calculate the centroid position of each independent surface of the grasped target based on the divided point cloud clusters, and extract the local point cloud within the preset radius of the centroid;

[0049] S5: The average point of the local point cloud is used as the grasping point, and the average normal vector of the local point cloud is used as the grasping direction to generate the robot grasping pose.

[0050] In the present invention, an RGBD camera (such as Intel RealSense D435) fixed at the end of the robot or above the scene is used to synchronously capture the RGB image and depth map of the scene, and the camera intrinsic parameter matrix and registration algorithm are used to convert the depth map into three-dimensional point cloud data (such as RGB point cloud data) aligned with the RGB image space. Figure 2 As shown), ensure that each pixel The coordinate transformation relationship for representing the same physical location in the RGB image and point cloud is:

[0051]

[0052] in, is the point cloud coordinate, is the depth value, is the camera intrinsic parameter matrix. The RGB image is then fed into a pre-trained object detection model (e.g., YOLOv7, YOLO-World, or Faster R-CNN). The model outputs one or more axis-aligned 2D detection bounding boxes based on the number of grasped objects as preliminary positioning information. It is important to emphasize that only normal detection bounding boxes without rotation angles need to be generated at this stage. For example, for boxes stacked at a 30° tilt, the detection bounding box can include the background area, thus reducing the complexity of the detection model.

[0053] The output axis-aligned detection box is used as a spatial prompt and input into the mask decoder of the pre-trained SegmentAnything Model (SAM). SAM generates a high-precision binary mask of the target object within the detection box area based on the global features extracted by the ViT-H image encoder. , where the pixels with mask value 1 correspond to the target main area, and 0 marks the background. This process completely breaks through the geometric limitations of the detection box. Then, using the RGB-point cloud registration relationship established by S1, the mask is mapped to the three-dimensional space: traversing the mask All pixel points, index their corresponding three-dimensional point cloud coordinates , thereby extracting a pure target point cloud subset (like Figure 3 (as shown). This subset reduces noise points compared to traditional detection box extraction methods and reduces the root mean square error of contour edges. Here, the present invention uses the detection box as a "guiding cue" rather than a "geometric boundary," enabling the algorithm to achieve segmentation accuracy comparable to manual annotation while retaining the efficiency of conventional detectors.

[0054] Furthermore, considering the multi-plane targets such as boxes and boxes, the target point cloud obtained May contain multiple surfaces of an object at the same time, such as the top and side surfaces (e.g. Figure 4If the centroid is calculated directly for a mixed point cloud containing multiple surfaces, the points on non-target surfaces (such as the side) will significantly affect the position of the centroid, causing it to deviate from the actual center of the target grasping surface (such as the top surface), and thus the calculated grasping point G is not located in the center area of ​​the desired surface. To overcome this problem, the present invention first calculates the target point cloud subset extracted by S2. Normal vector estimation - construct local tangent plane based on KD tree accelerated neighbor search, for each point Calculate its covariance matrix:

[0055]

[0056] in, is the centroid of the k nearest neighbors, for point The jth neighbor of .

[0057] The eigenvalue decomposition is used to obtain the eigenvector corresponding to the minimum eigenvalue as the initial normal vector At this time, the normal vector direction is ambiguous (may point to the inside or outside of the object), and the direction needs to be unified through the tangent plane normal consistency orientation algorithm: a global tangent plane is established with the point cloud centroid as the origin, forcing all normal vectors to satisfy , ensuring that the normal vectors of the point clouds on the same continuous surface (such as the top surface of the box) are in the same direction.

[0058] Then, the normalized normal vector As a feature, the density-based DBSCAN clustering algorithm is used to separate independent surfaces (such as Figure 5 Set the neighborhood distance threshold (radians) and the minimum number of points that will satisfy This design can accurately distinguish surfaces with angles greater than a threshold, such as the top of a box (where the normal vector is pointing vertically upward). ) and side (horizontal direction ) are divided into different clusters because the normal vector angle is greater than 90°. The final output point cloud cluster set , M is the total number of faces of the grasped target, and each cluster corresponds to an independent physical surface.

[0059] To improve robustness, statistical outlier removal (SOR) can be optionally performed before normal vector estimation to filter out outliers caused by sensor noise or occlusion, thus obtaining a cleaner target point cloud. Each point in , and its k nearest neighbors ,calculate The average distance to its k nearest neighbors , and the mean of all average distances and distance standard deviation , eliminating the satisfied The noise point ( is the control coefficient), and the filtered point cloud is obtained .

[0060] According to the segmented point cloud cluster , the corresponding center of mass position is calculated by arithmetic averaging of spatial coordinates to avoid mixed surface interference:

[0061]

[0062] in, is the spatial three-dimensional coordinate of the L-th independent surface centroid position, is the number of points in the point cloud cluster corresponding to the Lth independent surface, is the spatial three-dimensional coordinate of the i-th point in the point cloud cluster corresponding to the L-th independent surface.

[0063] Subsequently, the present invention adopts an adaptive radius mechanism to set the radius of the local point cloud subset, and calculates the point cloud cluster The three-dimensional axial bounding box (AABB) with the minimum side length ,according to Set a preset radius , and the centroid Extract local point cloud subset for the center .

[0064] However, in the automated destacking scenario, the point cloud clusters obtained by segmentation The surface integrity may be lost due to object deformation, partial occlusion or sensor noise. Directly calculating the grasping posture will cause serious engineering risks. First, the grasping point offset risk: When the surface is partially missing (such as the top surface of the box is blocked by adjacent goods for 30% of the area), the center of mass The point cloud will be biased to one side of the residual point cloud, causing the robot's end effector to be misaligned with the target center, resulting in unstable grasping or object slipping. Second, the risk of grasping posture inaccuracy: the direction of the point cloud normal vector of the deformed surface (such as a bulging soft bag) is disordered, and the average normal vector calculated at this time is The inability to represent the true surface orientation (orientation error of more than 25°) can cause the suction cup or gripper to tilt and collide with adjacent objects. Third, wasteful grasping actions: Forced grasping of broken surfaces (such as torn packaging) not only has a near-zero success rate but also risks contaminating the actuator and disrupting the workflow.

[0065] To address the above defects, the present invention introduces a plane fitting residual verification mechanism to perform validity verification on each divided point cloud cluster after obtaining the centroid position. The quantitative evaluation formula is:

[0066]

[0067] in, is the plane fitting residual of the Lth independent surface. Greater than the preset residual threshold When a surface is broken, it is determined to be invalid and automatically switches to an adjacent valid surface (such as abandoning the damaged top surface and grabbing the intact side surface instead) to avoid process interruption. This mechanism increases the success rate of grasping blocked scenes and reduces invalid grasping actions.

[0068] For surfaces that pass the validity check , with its local point cloud subset As input, the coordinates of the grab points are calculated by geometric averaging:

[0069]

[0070] Among them, G is the three-dimensional coordinate of the grasping point, is the number of points in the local point cloud, is the spatial 3D coordinate of the jth point in the local point cloud. The design focuses on the centroid neighborhood (radius The average normal vector is calculated as the grasping direction:

[0071]

[0072] in, is the grasping direction, for This process achieves dual stability of the grasping posture: the geometric average ensures that G is located in a relatively flat area of ​​the local surface, while the normal vector average suppresses the posture jitter caused by outliers.

[0073] When the target has multiple valid surfaces (such as a box with exposed top and side surfaces at the same time), a stability scoring decision mechanism is introduced:

[0074]

[0075] in, For stability score, is the weight coefficient, is the grasping direction, is the unit vector of gravity direction, is the density of the local point cloud. The larger the value, the more perpendicular the grasping direction is to the ground (when the end tool for depalletizing soft bags and boxes is a suction cup, the grasping direction should be as parallel to the direction of gravity as possible). Characterize local surface integrity, weight configuration Prioritize anti-slip safety and calculate all surfaces of the same target , select the surface with the highest score to perform grasping, so that the success rate of single-target grasping in complex stacking scenes is improved. Select the surface with the highest score to perform grasping, so that the success rate of single-target grasping in complex stacking scenes is improved.

[0076] In summary, the proposed method for robotic depalletizing and grasping pose estimation based on an image segmentation model uses a common axis-aligned detection frame to guide the image segmentation model in generating pixel-level masks, completely eliminating the need for high-precision rotation frames. This design makes the algorithm compatible with any general-purpose detector. Even when objects are tilted, tightly stacked, or irregularly shaped, the masked point cloud subset can still accurately exclude interference from background and adjacent objects.

[0077] To address the industry challenge of multi-plane target capture, a three-level guarantee mechanism has been constructed, including normal vector consistency processing, normal vector clustering and segmentation, and surface verification. This mechanism achieves precise separation of the top / side surfaces of the box, eliminates normal vector confusion within a single plane through a directional consistency algorithm, and combines clustering with dynamic verification of plane fitting residuals to effectively solve surface fragmentation problems caused by occlusion or deformation.

[0078] In order to enhance anti-interference and adaptive capabilities, multiple technical barriers are adopted: the statistical outlier removal algorithm significantly suppresses sensor noise and improves the reliability of normal vector estimation; the adaptive radius mechanism based on the minimum side length of the point cloud cluster bounding box ensures that effective local point clouds can be extracted from horizontal, vertical and inclined surfaces; the pose calculation strategy focusing on the local point cloud near the center of mass successfully avoids edge noise interference, and the measured grasping stability is improved.

[0079] When there are multiple visible surfaces for the same grasping target, the feasibility of grasping each surface is quantitatively evaluated through a stability scoring model, and surfaces with the best perpendicularity between the normal vector and the direction of gravity and the highest point cloud density are prioritized (for example, when stacking boxes, the system automatically avoids obscured top surfaces and selects intact side surfaces), thereby improving the adaptability of single-target grasping in complex stacking scenarios.

[0080] It should be noted that all directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present invention are only used to explain the relative position relationship, movement status, etc. between the various components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0081] In addition, in the present invention, descriptions such as "first," "second," and "one" are for descriptive purposes only and should not be understood to indicate or imply their relative importance or implicitly specify the number of the technical features indicated. Therefore, a feature specified as "first" or "second" may explicitly or implicitly include at least one of such features. In the description of the present invention, "plurality" means at least two, such as two, three, etc., unless otherwise specifically defined.

[0082] In the present invention, unless otherwise specified or limited, the terms "connection" and "fixation" should be understood in a broad sense. For example, "fixation" can mean fixed connection, detachable connection, or integration; mechanical connection or electrical connection; direct connection or indirect connection through an intermediate medium; internal communication between two elements or interaction between two elements, unless otherwise specified. Those skilled in the art will be able to understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0083] In addition, the technical solutions between the various embodiments of the present invention can be combined with each other, but it must be based on the fact that ordinary technicians in this field can implement it. When the combination of technical solutions is mutually contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

Claims

1. A robot depalletizing grasping pose estimation method based on image segmentation model, characterized in that: Including steps: S1: Obtain the scene RGB image and the 3D point cloud data registered with it, and use the object detection model to extract the axis-aligned 2D detection box of the grasped object in the RGB image; S2: Using the 2D detection box as the positioning basis, the image segmentation model is input to generate a pixel-level mask of the grasped object and map it to the 3D point cloud aligned with it. Based on the mapping result, a point cloud subset of the grasped object is extracted; S3: Perform normal vector estimation and directional consistency processing on the point cloud subset, and use directional consistency as the clustering segmentation basis to divide the point cloud clusters corresponding to each independent surface of the grasped target; S4: Calculate the centroid position of each independent surface of the grasped target based on the divided point cloud clusters, and extract the local point cloud within the preset radius of the centroid; S5: The average point of the local point cloud is used as the grasping point, and the average normal vector of the local point cloud is used as the grasping direction to generate the robot grasping pose.

2. A robot depalletizing grasping pose estimation method based on an image segmentation model as claimed in claim 1, characterized in that: In the step S3, the directional consistency processing is specifically as follows: The normal vector direction is unified through the tangent plane normal consistency orientation algorithm, so that the normal vector direction in the point cloud area belonging to the same continuous surface is consistent.

3. The robot depalletizing grasping pose estimation method based on image segmentation model according to claim 1, characterized in that: In the step S3, the cluster segmentation adopts a density-based noise space clustering algorithm, and divides the independent surface corresponding point cloud clusters using the normalized normal vector as a feature.

4. The robot depalletizing grasping posture estimation method based on image segmentation model according to claim 1, characterized in that: The S3 step further includes the following steps: S31: Denoising the extracted point cloud subset using a statistical outlier removal algorithm.

5. The robot depalletizing grasping posture estimation method based on image segmentation model according to claim 1, characterized in that: In step S4, the preset radius range is adaptively adjusted according to the size of the grasped target, and the formula is expressed as: Where, is the preset radius range, is the minimum side length of the 3D bounding box of the current point cloud cluster, is the preset scale factor.

6. The robot depalletizing grasping posture estimation method based on image segmentation model according to claim 1, characterized in that: In the step S5, the grasping point and the grasping direction are obtained by the following formula: Where G is the three-dimensional coordinate of the grasping point, is the number of points in the local point cloud, is the spatial three-dimensional coordinate of the j-th point in the local point cloud, is the grasping direction, for The corresponding unit normal vector.

7. The robot depalletizing grasping posture estimation method based on image segmentation model according to claim 1, characterized in that: In the step S4, after obtaining the centroid position, the step of performing validity check on each divided point cloud cluster is also included. The formula is expressed as: Where, is the plane fitting residual of the Lth independent surface, is the number of points in the point cloud cluster corresponding to the Lth independent surface, is the average normal vector of the Lth independent surface, is the spatial three-dimensional coordinate of the L-th independent surface centroid position, is the spatial three-dimensional coordinate of the i-th point in the point cloud cluster corresponding to the L-th independent surface.

8. The robot depalletizing grasping posture estimation method based on image segmentation model according to claim 1, characterized in that: In the step S5, for each grasping point of the independent surface local point cloud, the step further includes: calculating the stability score of each grasping point, and selecting the grasping point with the highest stability score. The stability score calculation formula is: Where, For stability score, is the weight coefficient, is the grasping direction, is the unit vector of gravity direction, is the density of the local point cloud.

9. The robot depalletizing grasping posture estimation method based on image segmentation model according to claim 1, characterized in that: In step S2, the image segmentation model uses SAM, and the detection box is used as a prompt to input its mask decoder to generate a pixel-level mask.

Images