Robot unstacking grabbing pose estimation method based on image segmentation model

Abstract

The invention discloses a robot unstacking grabbing pose estimation method based on an image segmentation model, and relates to the technical field of image processing, and the method mainly comprises the steps: taking a two-dimensional detection frame as a positioning basis, inputting an image segmentation model, generating a pixel-level mask of a grabbing target, mapping the pixel-level mask to a three-dimensional point cloud which is registered with the pixel-level mask, and carrying out the positioning of the three-dimensional point cloud; extracting a point cloud subset of the captured target according to a mapping result; normal vector estimation and direction consistency processing are carried out on the point cloud subsets, and division of point cloud clusters corresponding to each independent surface of the grabbing target is carried out by taking direction consistency as a clustering segmentation basis; calculating the mass center position of each independent surface of the grabbed target according to the divided point cloud clusters, and extracting a local point cloud within a preset radius range of the mass center; and the average point of the local point cloud serves as a grabbing point, the average normal vector of the local point cloud serves as the grabbing direction, and the grabbing pose of the robot is generated. According to the invention, when the object is inclined, stacked tightly or shaped irregularly, the point cloud subset mapped by the mask can still accurately eliminate the interference of the background and the adjacent object.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and particularly relates to a method for estimating the grasping pose of a robot palletizing and depalletizing based on an image segmentation model. Background Art

[0002] In the field of robot automatic depalletizing, existing methods usually rely on 3D sensors to obtain scene point cloud data, and use object detection models (such as YOLO, Faster R-CNN, etc.) to locate the objects to be grasped in RGB images, and then intercept the corresponding 3D point cloud regions based on the detection boxes to calculate the grasping pose. However, this technical route has significant defects.

[0003] First, the accuracy limitation of the detection box leads to a prominent problem of point cloud contamination. When using a standard axis-aligned detection box, since it cannot fit the actual contours of inclined, irregularly shaped or tightly stacked objects, the extracted point cloud often contains a large number of background points or points of adjacent objects. Such contaminated point clouds will directly interfere with subsequent centroid calculation and pose estimation. For example, when calculating the centroid of a box, the mixed side or pallet point clouds will cause it to deviate seriously from the true center of the target grasping surface (such as the top surface).

[0004] Second, the introduction of a rotated detection box does not fundamentally solve the problem. Although the rotated box can partially improve the target bounding accuracy, it significantly increases the complexity and training difficulty of the detection model. For non-rigid objects such as soft packages and deformed bags, it is still difficult for the rotated box to accurately describe the contour, and the prediction error of the rotation angle will further amplify the point cloud extraction deviation. More critically, the algorithm framework relying on high-precision detection boxes cannot adapt to complex conditions such as random stacking and partial occlusion commonly found in logistics scenarios.

[0005] Finally, existing methods lack targeted processing of multi-plane targets. When grasping box-shaped objects, directly calculating a single centroid for the mixed surface point cloud (such as containing both the top surface and the side surface) will cause the grasping point to deviate from the expected area due to the interference of the point cloud of non-target surfaces. For example, the participation of the side surface point cloud may cause the grasping point to deviate towards the edge of the box rather than the center of the top surface, thereby leading to risks of unstable grasping or collision. This problem is particularly prominent in dense stacking scenarios, and the existing technology has not provided an effective surface separation mechanism. Summary of the Invention

[0006] In order to improve the accuracy of robot depalletizing, reduce recognition deviation and grasping point offset, the present invention proposes a method for estimating the grasping pose of a robot palletizing and depalletizing based on an image segmentation model, including the steps of: S1: Obtain the scene RGB image and the registered 3D point cloud data, and extract the axis-aligned two-dimensional detection box of the grasping target in the RGB image through an object detection model; S2: Use the 2D detection box as the positioning basis to input the image segmentation model, generate the pixel-level mask of the grasping target and map it to the registered 3D point cloud, and extract the point cloud subset of the grasping target according to the mapping result; S3: Perform normal vector estimation and direction consistency processing on the point cloud subset, and divide the point cloud clusters corresponding to the independent surfaces of the grasping target based on direction consistency as the clustering segmentation basis; S4: Calculate the centroid positions of the independent surfaces of the grasping target according to the divided point cloud clusters, and extract the local point cloud within the preset radius range of the centroid; S5: Use the average point of the local point cloud as the grasping point and the average normal vector of the local point cloud as the grasping direction to generate the robot grasping pose.

[0007] Further, in the step S3, the direction consistency processing is specifically: Unify the normal vector directions through the tangent plane normal consistency orientation algorithm to make the normal vectors in the point cloud region belonging to the same continuous surface point in the same direction.

[0008] Further, in the step S3, the clustering segmentation uses the density-based spatial clustering of applications with noise (DBSCAN) algorithm, and divides the point cloud clusters corresponding to the independent surfaces with the normalized normal vector as the feature.

[0009] Further, after the step S3, there is also a step: S31: Denoise the extracted point cloud subset through the statistical outlier removal algorithm.

[0010] Further, in the step S4, the centroid position is obtained through the following formula: In the formula, is the three-dimensional spatial coordinate of the centroid position of the L-th independent surface, is the number of points in the point cloud cluster corresponding to the L-th independent surface, is the three-dimensional spatial coordinate of the i-th point in the point cloud cluster corresponding to the L-th independent surface.

[0011] Further, in the step S4, the preset radius range is adaptively adjusted according to the size of the grasping target, and the formula is expressed as: In the formula, is the preset radius range, is the minimum side length of the three-dimensional bounding box of the current point cloud cluster, is the preset proportionality coefficient.

[0012] Further, in the step S5, the grasping point and the grasping direction are obtained through the following formula: Wherein, G is the three-dimensional spatial coordinate of the grasping point, is the number of point clouds of the local point cloud, is the three-dimensional spatial coordinate of the j-th point in the local point cloud, is the grasping direction, is the corresponding unit normal vector.

[0013] Furthermore, in the step S4, after obtaining the centroid position, the following steps are further included: performing validity verification on each divided point cloud cluster, and the formula is expressed as: Wherein, is the plane fitting residual of the L-th independent surface, is the average normal vector of the L-th independent surface, is the three-dimensional spatial coordinate of the centroid position of the L-th independent surface, is the three-dimensional spatial coordinate of the i-th point in the point cloud cluster corresponding to the L-th independent surface.

[0014] Furthermore, in the step S5, for the grasping points of the local point cloud of each independent surface, the following steps are further included: calculating the stability score of each grasping point, and selecting the grasping point with the highest stability score. The stability score calculation formula is: Wherein, is the stability score, is the weight coefficient, is the grasping direction, is the unit vector of the gravity direction, is the density of the local point cloud.

[0015] Furthermore, in the step S2, the image segmentation model adopts SAM, and the detection box is used as a prompt to input its mask decoder to generate a pixel-level mask.

[0016] Compared with the prior art, the present invention has at least the following beneficial effects: (1) A robot palletizing and grasping pose estimation method based on an image segmentation model proposed by the present invention, which generates a pixel-level mask based on a general axis-aligned detection box to guide the image segmentation model, completely getting rid of the dependence on high-precision rotated boxes. This design makes the algorithm compatible with any general detector. When the object is tilted, stacked tightly or has an irregular shape, the point cloud subset mapped by the mask can still accurately exclude the interference of the background and adjacent objects; (2)A three-level guarantee mechanism of normal vector consistency processing, normal vector clustering segmentation, and surface verification is constructed to address the industry problem of multi-plane target grasping, achieving precise separation of the top and side surfaces of the box. The normal vector chaos within a single plane is eliminated through the direction consistency algorithm, and combined with clustering and dynamic verification of the plane fitting residuals, effectively solving the problem of surface fragmentation caused by occlusion or deformation. (3)Regarding anti-interference and adaptability, multiple technical barriers are adopted: the statistical outlier removal algorithm significantly suppresses sensor noise and improves the reliability of normal vector estimation; the adaptive radius mechanism based on the minimum side length of the bounding box of the point cloud cluster ensures that effective local point clouds can be extracted from horizontal, vertical, and inclined surfaces; the pose calculation strategy focusing on the local point cloud near the centroid successfully avoids edge noise interference, and the measured grasping stability is improved. (4)When there are multiple visible surfaces for the same grasping target, the grasping feasibility of each surface is quantitatively evaluated through a stability scoring model, and the surface with the best perpendicularity between the normal vector and the gravity direction and the highest point cloud density is preferentially selected (for example, when stacking boxes, the occluded top surface is automatically avoided and the complete side surface is selected), improving the single-target grasping adaptability in complex stacking scenarios. Description of the Drawings

[0017] Figure 1 It is a step diagram of a robot unstacking and grasping pose estimation method based on an image segmentation model; Figure 2 It is a schematic diagram of the original point cloud; Figure 3 It is a schematic diagram of the target point cloud; Figure 4 It is a schematic diagram of the multi-plane target point cloud; Figure 5 It is a schematic diagram of clustering segmentation according to the normal vector. Detailed Implementation Manner

[0018] The following are specific embodiments of the present invention in combination with the drawings, further describing the technical solutions of the present invention, but the present invention is not limited to these embodiments.

[0019] In an automated warehousing and logistics palletizing and depalletizing system, the mainstream technical solutions usually rely on the collaborative processing of 3D point cloud data and object detection models: First, detectors such as YOLO and Faster R-CNN are used to generate 2D detection boxes of target objects in RGB images, and then the corresponding 3D point cloud regions are intercepted according to the detection boxes to calculate the grasping poses. However, this technical route exposes significant limitations in actual industrial scenarios. First, the contour fitting degree of the detection box severely restricts the point cloud quality - when dealing with boxes placed obliquely or deformed soft packages, the standard axis-aligned detection box will introduce a large amount of background and adjacent object point clouds. Although the rotated detection box partially improves the fitting degree, it is difficult to implement in engineering due to the sharp increase in model complexity and angle prediction deviation. Second, the mixed point cloud of multi-plane targets causes the grasping point to deviate: The point cloud of box-like objects often contains both the top surface and the side surface at the same time. Directly calculating the overall centroid will cause the grasping point to deviate to non-target surfaces (such as the side edge). Especially when the boxes are stacked tightly, the deviation of the grasping point can reach the centimeter level, significantly increasing the risk of grasping failure and stack collapse. Third, the existing methods lack an adaptive decision-making mechanism, which can neither dynamically select the optimal grasping surface according to the surface integrity (such as automatically switching to the side surface when the top surface is blocked), nor coordinate the grasping order of multiple targets in dense stacking, resulting in insufficient system robustness. To address the above industry pain points, the present invention proposes a method for estimating the grasping pose of a robot for palletizing and depalletizing based on an image segmentation model, including the steps: S1: Obtain the scene RGB image and the registered 3D point cloud data, and extract the axis-aligned 2D detection box of the grasping target in the RGB image through the object detection model; S2: Use the 2D detection box as the positioning basis to input the image segmentation model, generate the pixel-level mask of the grasping target and map it to the registered 3D point cloud, and extract the point cloud subset of the grasping target according to the mapping result; S3: Perform normal vector estimation and direction consistency processing on the point cloud subset, and divide the point cloud clusters corresponding to each independent surface of the grasping target based on the direction consistency as the clustering segmentation basis; S4: Calculate the centroid positions of each independent surface of the grasping target according to the divided point cloud clusters, and extract the local point cloud within the preset radius of the centroid; S5: Generate the robot grasping pose with the average point of the local point cloud as the grasping point and the average normal vector of the local point cloud as the grasping direction.

[0020] In the present invention, first, an RGBD camera (such as an Intel RealSense D435) fixed at the end of the robot or above the scene is used to synchronously collect the RGB image and the depth map of the scene, and the depth map is converted into 3D point cloud data (such as Figure 2 shown) that is spatially aligned with the RGB image by using the camera intrinsic matrix and the registration algorithm, ensuring that each pixel point Represent the same physical location in the RGB image and the point cloud, and the coordinate conversion relationship is as follows: Wherein, is the point cloud coordinate, is the depth value, is the camera intrinsic matrix. Subsequently, the RGB image is input into a pre-trained object detection model (optional architectures such as YOLOv7, YOLO-World, or Faster R-CNN, etc.). This model outputs one or more axis-aligned 2D detection boxes as the preliminary localization information of the grasping target according to the number of grasping targets. It should be emphasized that at this stage, only ordinary detection boxes without rotation angles need to be generated. For example, for a box stacked at an inclination of 30°, the detection box is allowed to contain background areas, thereby reducing the complexity required by the detection model.

[0021] Taking the output axis-aligned detection box as a spatial prompt, input it into the mask decoder of the pre-trained SegmentAnything Model (SAM). SAM generates a high-precision binary mask of the target object within the detection box area based on the global features extracted by the ViT-H image encoder , where the pixels with a mask value of 1 correspond to the target main body area, and the value of 0 marks the background. This process completely breaks through the geometric limitations of the detection box. Subsequently, using the RGB-point cloud registration relationship established in S1, the mask is mapped to the three-dimensional space: traverse all pixel points of the mask , and index its corresponding three-dimensional point cloud coordinate , so as to extract a pure target point cloud subset (as shown in Figure 3 ). This subset reduces the noise points compared with the traditional detection box interception method, and the root mean square error of the contour edge is reduced. Here, the present invention uses the detection box as a "guidance clue" rather than a "geometric boundary", enabling the algorithm to obtain segmentation accuracy comparable to manual annotation while retaining the high efficiency of ordinary detectors.

[0022] Furthermore, considering that when grasping multi-plane targets such as boxes and boxes, the obtained target point cloud may simultaneously contain multiple surfaces of the object, such as the top surface and the side surface (as shown in Figure 4 ). If the centroid is directly calculated for the mixed point cloud containing multiple surfaces, the points on the non-target surface (such as the side surface) will significantly affect the position of the centroid, resulting in its deviation from the actual center of the target grasping surface (such as the top surface), and further causing the calculated grasping point G not to be located in the central area of the expected surface. To overcome this problem, the present invention first estimates the normal vector of the target point cloud subset extracted in S2 - construct a local tangent plane based on the nearest neighbor search accelerated by the KD tree, for each point Calculate its covariance matrix: where, is the centroid of the k nearest neighbor points, is the th neighbor point of point

[0023] The eigenvector corresponding to the minimum eigenvalue is obtained through eigenvalue decomposition as the initial normal vector . At this time, there is ambiguity in the direction of the normal vector (it may point to the inside or outside of the object), and the direction needs to be unified through the tangent plane normal consistency orientation algorithm: a global tangent plane is established with the centroid of the point cloud as the origin, and all normal vectors are forced to satisfy , ensuring that the normal vectors of the point clouds on the same continuous surface (such as the top surface of the box) are in the same direction.

[0024] Subsequently, using the normalized normal vector as the feature, the density-based DBSCAN clustering algorithm is used to separate independent surfaces (as shown in Figure 5 ): Set the neighborhood distance threshold (in radians) and the minimum number of points. Points that satisfy are grouped into the same cluster. This design can accurately distinguish surfaces with an included angle greater than the threshold - for example, the top surface of the box (the normal vector is vertically upward ) and the side surface (horizontal direction ) are divided into different clusters because the included angle between the normal vectors is greater than 90°. Finally, the point cloud cluster set is output, where M is the total number of surfaces of the grasping target, and each cluster corresponds to an independent physical surface.

[0025] To improve robustness, statistical outlier removal (SOR) can be optionally performed before normal vector estimation to filter out outliers caused by sensor noise or occlusion, obtaining a cleaner target point cloud. Among them, for each point in the point cloud , and its k nearest neighbor points , calculate the average distance from to its k nearest neighbor points, as well as the mean of all average distances and the distance standard deviation . Remove the noise points that satisfy ( is the control coefficient), obtaining the filtered point cloud .

[0026] According to the segmented point cloud clusters , the corresponding centroid positions are calculated through arithmetic averaging of spatial coordinates, and this design is used to avoid interference from mixed surfaces: Among them, is the three-dimensional spatial coordinates of the centroid position of the L-th independent surface, is the number of points in the point cloud cluster corresponding to the L-th independent surface, is the three-dimensional spatial coordinates of the i-th point in the point cloud cluster corresponding to the L-th independent surface.

[0027] Subsequently, the present invention adopts an adaptive radius mechanism to set the radius of the local point cloud subset. By calculating the three-dimensional axis-aligned bounding box (AABB) of the point cloud cluster , taking its minimum side length , according to set the preset radius range , and extract the local point cloud subset with the centroid as the center.

[0028] However, in the automated palletizing scenario, the point cloud clusters obtained by segmentation may lose surface integrity due to object deformation, local occlusion, or sensor noise. Directly calculating the grasping pose for them will cause serious engineering risks. First, the risk of grasping point deviation: When part of the surface is missing (for example, 30% of the top surface of the box is blocked by adjacent goods), the centroid will deviate towards the side of the remaining point cloud, resulting in the end effector of the robot not being aligned with the target center, causing unstable grasping or the item slipping off. Second, the risk of inaccurate grasping posture: The normal vector directions of the point cloud on the deformed surface (such as a bulging soft package) are disordered. At this time, the calculated average normal vector cannot represent the true surface orientation (the direction error is more than 25°), resulting in the suction cup or fixture tilting and colliding with adjacent objects. Third, the waste of ineffective grasping actions: Forcefully grasping a broken surface (such as a torn packaging bag) not only has a success rate approaching zero, but is more likely to contaminate the actuator and interrupt the operation process.

[0029] To solve the above defects, the present invention introduces a plane fitting residual verification mechanism to perform validity verification on each divided point cloud cluster after obtaining the centroid position. The quantization evaluation formula is: Among them, is the plane fitting residual of the L-th independent surface. When is greater than the preset residual threshold , it is determined that the surface is invalid, and it will automatically switch to the adjacent valid surface (such as abandoning the damaged top surface and grasping the intact side surface) to avoid process interruption. Through this mechanism, the grasping success rate in the occlusion scenario is increased, and ineffective grasping actions are reduced.

[0030] For the surface that passes the validity verification, with its local point cloud subset Take the input and calculate the coordinates of the grasping point through geometric mean: Among them, \(G\) is the three-dimensional spatial coordinate of the grasping point, is the number of points in the local point cloud, is the three-dimensional spatial coordinate of the \(j\)-th point in the local point cloud. This design focuses on the centroid neighborhood area (within the radius range), avoiding the interference of the object edge or deformed part. Synchronously calculate the average normal vector as the grasping direction: Among them, is the grasping direction, is the corresponding unit normal vector. This process realizes the dual stability guarantee of the grasping pose: geometric mean makes \(G\) located in the relatively flat area of the local surface, while the average of the normal vectors suppresses the pose jitter caused by outliers.

[0031] When there are multiple effective surfaces for the target (such as when the top surface and the side surface of the box are exposed simultaneously), introduce a stability scoring decision mechanism: Among them, is the stability score, is the weight coefficient, is the grasping direction, is the unit vector of the gravity direction, is the density of the local point cloud. When the value of the normal vector perpendicularity term is larger, it indicates that the grasping direction is more perpendicular to the ground (in the case where the end effector for unpacking soft packages and boxes is a suction cup, the grasping direction should be as parallel as possible to the gravity direction), and the point cloud density term characterizes the integrity of the local surface. The weight configuration gives priority to ensuring anti-slip safety. Calculate for all surfaces of the same target, select the surface with the highest score to perform grasping, which improves the single-target grasping success rate in complex stacking scenarios. Select the surface with the highest score to perform grasping, which improves the single-target grasping success rate in complex stacking scenarios.

[0032] To sum up, a method for estimating the grasping pose of a robot for unpacking based on an image segmentation model proposed by the present invention generates a pixel-level mask based on a general axis-aligned detection box to guide the image segmentation model, completely getting rid of the dependence on high-precision rotated boxes. This design makes the algorithm compatible with any general detector. When the object is tilted, stacked closely or has an irregular shape, the point cloud subset mapped by the mask can still accurately exclude the interference of the background and adjacent objects.

[0033] A three - level guarantee mechanism for normal vector consistency processing, normal vector clustering segmentation, and surface verification is constructed to address the industry problem of multi - plane target grasping, achieving precise separation of the top and side surfaces of the box. The normal vector chaos within a single plane is eliminated through the direction consistency algorithm, and combined with clustering and dynamic verification of the plane fitting residuals, the problem of surface fragmentation caused by occlusion or deformation is effectively solved.

[0034] Regarding anti - interference and adaptability, multiple technical barriers are adopted: the statistical outlier removal algorithm significantly suppresses sensor noise and improves the reliability of normal vector estimation; the adaptive radius mechanism based on the minimum side length of the bounding box of the point cloud cluster ensures that effective local point clouds can be extracted from horizontal, vertical, and inclined surfaces; the pose calculation strategy focusing on the local point cloud near the centroid successfully avoids edge noise interference, and the measured grasping stability is improved.

[0035] When there are multiple visible surfaces for the same grasping target, the grasping feasibility of each surface is quantitatively evaluated through a stability scoring model, and the surface with the best perpendicularity between the normal vector and the gravity direction and the highest point cloud density is preferentially selected (for example, when stacking boxes, the occluded top surface is automatically avoided and the complete side surface is selected), improving the single - target grasping adaptability in complex stacking scenarios.

[0036] It should be noted that all directional indications (such as up, down, left, right, front, back...) in the embodiments of the present invention are only used to explain the relative positional relationship and motion conditions between components in a specific posture (as shown in the attached drawings). If the specific posture changes, the directional indication will also change accordingly.

[0037] In addition, in the present invention, descriptions such as "first", "second", "one" are only for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly specifying the quantity of the indicated technical features. Thus, features defined with "first" and "second" may explicitly or implicitly include at least one such feature. In the description of the present invention, the meaning of "multiple" is at least two, such as two, three, etc., unless otherwise specifically defined.

[0038] In the present invention, unless otherwise clearly specified and limited, terms such as "connection" and "fixation" should be understood in a broad sense. For example, "fixation" can be a fixed connection, a detachable connection, or integrated; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium, and can be the internal connection of two components or the interaction relationship between two components, unless otherwise clearly limited. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0039] In addition, the technical solutions between the various embodiments of the present invention can be combined with each other, but it must be based on the fact that those of ordinary skill in the art can implement it. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

Claims

1. A method for estimating the grasping pose of a robot for palletizing and depalletizing based on an image segmentation model, characterized in that, Including the steps: S1: Obtain the scene RGB image and the registered 3D point cloud data, and extract the axis-aligned 2D detection box of the grasping target in the RGB image through the object detection model; S2: Take the 2D detection box as the positioning basis and input it into the image segmentation model to generate the pixel-level mask of the grasping target and map it to the registered 3D point cloud. Extract the point cloud subset of the grasping target according to the mapping result; S3: Perform normal vector estimation and direction consistency processing on the point cloud subset, and divide the point cloud clusters corresponding to each independent surface of the grasping target based on direction consistency as the clustering segmentation basis; S4: Calculate the centroid positions of each independent surface of the grasping target according to the divided point cloud clusters, and extract the local point cloud within the preset radius range of the centroid; S5: Take the average point of the local point cloud as the grasping point and the average normal vector of the local point cloud as the grasping direction to generate the robot grasping pose.

2. The method for estimating the grasping pose of a robot for palletizing and depalletizing based on an image segmentation model according to claim 1, wherein In the step S3, the direction consistency processing is specifically: Unify the normal vector direction through the tangent plane normal consistency orientation algorithm, so that the normal vectors in the point cloud region belonging to the same continuous surface face the same direction.

3. The robot palletizing and grasping pose estimation method based on an image segmentation model according to claim 1, wherein In the step S3, the clustering segmentation adopts the density-based spatial clustering of applications with noise (DBSCAN) algorithm, and divides the point cloud clusters corresponding to independent surfaces with the normalized normal vector as the feature.

4. The robot palletizing and grasping pose estimation method based on an image segmentation model according to claim 1, characterized in that, After the step S3, there is also a step: S31: Denoise the extracted point cloud subset through the statistical outlier removal algorithm.

5. The robot palletizing and grasping pose estimation method based on an image segmentation model according to claim 1, characterized in that, In the step S4, the preset radius range is adaptively adjusted according to the size of the grasping target, and the formula is expressed as: Wherein, is the preset radius range, is the minimum side length of the three-dimensional bounding box of the current point cloud cluster, is the preset proportionality coefficient.

6. The robot palletizing and grasping pose estimation method based on an image segmentation model according to claim 1, wherein In the step S5, the grasping point and the grasping direction are obtained through the following formula: Where G is the three-dimensional spatial coordinate of the grasping point, is the number of points in the local point cloud, is the three-dimensional spatial coordinate of the j-th point in the local point cloud, is the grasping direction, is the corresponding unit normal vector.

7. The robot palletizing and grasping pose estimation method based on an image segmentation model according to claim 1, wherein In the step S4, after obtaining the centroid position, there is also a step: Perform validity verification on each divided point cloud cluster, and the formula is expressed as: In the formula, is the plane fitting residual of the L-th independent surface, is the number of points in the point cloud cluster corresponding to the L-th independent surface, is the average normal vector of the L-th independent surface, is the three-dimensional spatial coordinates of the centroid position of the L-th independent surface, is the three-dimensional spatial coordinates of the i-th point in the point cloud cluster corresponding to the L-th independent surface.

8. The method for estimating the grasping pose of a robot for palletizing and depalletizing based on an image segmentation model according to claim 1, wherein, In the step S5, for the grasping points of the local point cloud of each independent surface, there is also a step: Calculate the stability score of each grasping point, and select the grasping point with the highest stability score. The stability score calculation formula is: Wherein, is the stability score, is the weight coefficient, is the grasping direction, is the unit vector of the gravity direction, is the density of the local point cloud.

9. The method for estimating the grasping pose of a robot for palletizing and depalletizing based on an image segmentation model according to claim 1, wherein, In the step S2, the image segmentation model adopts SAM, and the detection box is used as a prompt to input its mask decoder to generate the pixel-level mask.

Images