Unknown object six-degree-of-freedom grabbing method considering point cloud skeleton features

Abstract

The application discloses a six-degree-of-freedom grasping method for unknown objects considering point cloud skeleton features, adopts an iterative moving sampling algorithm based on L1 median skeleton extraction, ensures that grasping points are finally uniformly arranged on the skeleton of the object, and avoids generating a large number of invalid grasps; candidate grasping poses are generated by using the distribution information of the skeleton points and the geometric information of the point cloud around the skeleton points, and the quality of the candidate grasping poses is effectively improved; according to the positional relationship between the grasping device and the object, a heuristic evaluation function is used to evaluate the grasping pose, so that the optimization sampling of the pose is ensured; the method is suitable for unknown objects with different shapes, and can generate stable grasping poses in an unstructured environment.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of mechanical arm vision grasping, and more particularly relates to a six-degree-of-freedom grasping method for unknown objects considering point cloud skeleton features. BACKGROUND

[0002] Intelligent robots have the advantages of high automation and strong flexibility, and can greatly improve productivity, and have penetrated into the fields of industrial production and logistics sorting. Grasping is one of the most basic functions thereof, and with the diversification of object types and the complication of working environments, the difficulty of grasping tasks is also increasing. Therefore, how to use visual information to quickly and robustly grasp unknown objects in unstructured environments has become a research hotspot at present.

[0003] Existing grasping pose generation methods are mainly divided into model-based methods and learning-based methods. The model-based method usually needs to establish a model library, matches the target object with the model through local features or global features, realizes pose estimation, and then determines the grasping pose, and therefore cannot be extended to other unknown objects. The learning-based method mainly uses two-dimensional images or additional depth information as input, and does not consider the three-dimensional geometric information of the object, and therefore can only generate a three-degree-of-freedom grasping frame, and adopts a top-down grasping strategy perpendicular to the image plane, which limits the grasping in unstructured environments. In recent years, some methods for realizing six-degree-of-freedom grasping by analyzing the geometric information of object point clouds have appeared, which do not need to establish a model and can be applied to unknown objects, but do not consider the features of the points suitable for grasping in the object, resulting in a large number of invalid grasps. SUMMARY

[0004] In view of the above defects or improvement needs of the prior art, the present application provides a six-degree-of-freedom grasping method for unknown objects considering point cloud skeleton features, which does not need to establish a model of the object in advance, and is not limited to two-dimensional plane grasping, and is suitable for more unknown objects with complex shapes.

[0005] To achieve the above-mentioned purpose, according to a first aspect of the present application, a six-degree-of-freedom grasping method for unknown objects considering point cloud skeleton features is provided, comprising:

[0006] S1, a depth camera is used to collect scene point clouds including a target object, and the target object point clouds J are obtained by preprocessing; wherein the depth camera is installed at the end of the mechanical arm in an eye-in-hand manner;

[0007] S2, the target object point clouds are down-sampled to obtain an initial sampling point set I, according to the formula The position of each initial sampling point is iteratively adjusted to obtain each skeleton point;

[0008] wherein, The initial sampling point x i is adjusted for the k+1th iteration, and the position J r is the set of points in J centered on r and with a radius r, and I is the set of points in I j centered on i′ and with a radius r, and q r is a point in J r and I

[0009]

[0010] S3, determine the grasping pose corresponding to each skeleton point;

[0011] S4, evaluate each grasping pose to select the optimal grasping pose;

[0012] S5, convert the optimal grasping pose from the camera coordinate system to the robot base coordinate system to perform the grasping operation on the target object.

[0013] Preferably, in step S3, for each skeleton point, a preset operation is performed to obtain the grasping pose corresponding to each skeleton point;

[0014] The preset operation includes: taking a region centered on the skeleton point and with a radius r as a target region; taking the target object point cloud and the points in the initial sampling point set located in the target region as neighborhood point clouds, respectively, and using the PCA method to construct the covariance matrix of each neighborhood point cloud; for the covariance matrix of the target object point cloud, taking the eigenvector corresponding to the smallest eigenvalue as the normal vector of the skeleton point; for the covariance matrix of the initial sampling point set, taking the eigenvector corresponding to the largest eigenvalue as the principal axis vector of the skeleton point; and determining the grasping pose of the skeleton point according to the normal vector and the principal axis vector.

[0015] Preferably, in step S4, each grasping pose is evaluated according to the positional relationship between the gripper and the target object;

[0016] The positional relationship includes: the maximum depth of the target object in the gripper, the distance between the target object and the left and right sides of the gripper, and the fitting degree of the target object boundary and the gripper.

[0017] Preferably, in step S4, each grasping pose is evaluated based on a heuristic evaluation function according to the positional relationship between the gripper and the target object;

[0018] The heuristic evaluation function is: s(g)=k l s l +kw s w +k b s b ;

[0019] in, k l k w k b Here, l and l' represent the weighting coefficients, respectively, the maximum depth of the target object in the gripper, and the gripper's gripping depth. left w right θ represents the distance between the target object and the left and right sides of the gripper, respectively. left θ right The slopes of the left and right contour boundaries of the gripper's gripping area are respectively fitted with straight lines and the acute angle between them and the Y-axis. The gripping area is obtained by projecting the object point cloud within the gripper's closed space onto the YOZ plane. O is the center point of the gripper's bottom, the gripper's closing direction is the Y-axis, and the normal vector of the skeleton point is the X-axis.

[0020] Preferably, in step S1, the preprocessing includes: identifying and segmenting the desktop point cloud, voxel downsampling, and removing noise points.

[0021] Preferably, in step S5, the optimal grasping pose is transformed from the camera coordinate system to the robot arm base coordinate system based on the hand-eye matrix of the depth camera relative to the end of the robotic arm.

[0022] According to a second aspect of the present invention, a six-DOF grasping device for an unknown object considering point cloud skeleton features is provided, comprising:

[0023] The preprocessing module is used to acquire scene point clouds including target objects using a depth camera, and preprocess them to obtain target object point cloud J; wherein, the depth camera is mounted at the end of the robotic arm in an eye-on manner.

[0024] The iterative adjustment module is used to downsample the point cloud of the target object to obtain an initial sampling point set I, according to the formula... Each initial sampling point is iteratively adjusted to obtain each skeleton point;

[0025] in, For the initial sampling point x i The position J after the (k+1)th iteration adjustment r For J in Let I be the set of points in a neighborhood centered at radius r. r For I in Let q be the set of points in a neighborhood centered at a radius of r. j x i′ Jr , I r the point in I,

[0026] a grasping pose module configured to determine grasping poses of the skeleton points;

[0027] an evaluation module configured to evaluate the grasping poses to select an optimal grasping pose;

[0028] a conversion module configured to convert the optimal grasping pose from a camera coordinate system to a robot base coordinate system to perform a grasping operation on the target object.

[0029] According to a third aspect of the present application, there is provided a six-degree-of-freedom grasping system for unknown objects considering skeleton features of point clouds, comprising: a computer readable storage medium and a processor;

[0030] the computer readable storage medium is configured to store executable instructions;

[0031] the processor is configured to read the executable instructions stored in the computer readable storage medium and execute the method according to the first aspect.

[0032] According to a fourth aspect of the present application, there is provided a computer readable storage medium storing computer instructions for causing a processor to execute the method according to the first aspect.

[0033] Overall, the above technical solutions conceived by the present application can achieve the following beneficial effects compared with the prior art:

[0034] 1. The method provided by the present application uses an iterative moving sampling algorithm based on L1 median skeleton extraction to obtain skeleton points, which can ensure that the grasping points are finally uniformly arranged on the skeleton of the object and avoid generating a large number of invalid grasps. Since the skeleton model is equivalent to a simplified description of the topological structure of the object, the simplified skeleton points are extracted from complex objects of different shapes, and the skeleton points are located on the center axis of the object, which is more in line with the habit of manual grasping of objects by humans, and the success rate of grasping is also higher. Therefore, the method provided by the present application is suitable for unknown objects of different shapes and can generate stable grasping poses in unstructured environments.

[0035] 2. The method provided by the present application uses the distribution information of the skeleton points and the geometric information of the point cloud around the skeleton points to generate candidate grasping poses, effectively improving the quality of the candidate grasping poses.

[0036] 3. The method provided by the present application uses a heuristic evaluation function to evaluate the grasping poses according to the positional relationship between the grasping device and the object, thereby ensuring the optimal sampling of the poses. BRIEF DESCRIPTION OF DRAWINGS

[0037] Figure 1 A flowchart of a six-degree-of-freedom grasping method of an unknown object considering point cloud skeleton features is provided for an embodiment of the present application.

[0038] Figure 2 A candidate set of grasping poses generated in an embodiment of the present application is provided.

[0039] Figure 3 A schematic diagram of point cloud and contour point extraction in a gripper in an embodiment of the present application is provided.

[0040] Figure 4 A simplified two-finger parallel gripper configuration diagram in an embodiment of the present application is provided.

[0041] Figure 5 A schematic diagram of extracted point cloud skeleton points in an embodiment of the present application is provided. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application. In addition, the technical features involved in each embodiment of the present application described below can be combined with each other as long as they do not conflict with each other.

[0043] An embodiment of the present application provides a six-degree-of-freedom grasping method of an unknown object considering point cloud skeleton features, as shown in Figure 1 , comprising:

[0044] S1, a depth camera is used to collect scene point cloud including target object, and target object point cloud J is obtained by preprocessing; wherein the depth camera is installed at the end of the mechanical arm in the manner of eye-in-hand.

[0045] Specifically, S1 comprises:

[0046] S11, a depth camera is installed at the end of the mechanical arm in the manner of eye-in-hand, the intrinsic parameters of the camera are obtained by camera calibration, and the hand-eye matrix representing the conversion relationship between the camera coordinate system and the mechanical arm end coordinate system is obtained by hand-eye calibration.

[0047] S12, the mechanical arm is moved to a set photographing position, scene single-view point cloud containing the target object is collected by the depth camera, and target object point cloud is obtained by preprocessing the scene single-view point cloud.

[0048] Preferably, the preprocessing comprises: identifying and segmenting the tabletop point cloud, voxel down-sampling processing, and removing noise points.

[0049] For example, a random sampling consistent algorithm is used to identify and segment the desktop point cloud, a voxel downsampling method is used to simplify the point cloud, and a statistical-based method is used to remove outlier noise points.

[0050] S2, down-sampling the target object point cloud to obtain an initial sampling point set I, according to the formula The position of each initial sampling point is adjusted iteratively to obtain each skeleton point;

[0051] wherein, is the position of the initial sampling point x i after the k+1th iteration adjustment (the position is represented in the form of coordinates), J r is a set of points in the neighborhood of x with r as the center and r as the radius, I r is a set of points in the neighborhood of x with r as the center and r as the radius, q j , x i′ are points in J r , I r

[0052]

[0053] Specifically, in order to reduce the amount of calculation, first, the target object point cloud is randomly down-sampled to obtain an initial sampling point set; then, a L1 median skeleton extraction algorithm is used to iteratively move the sampling points, and local adjustment is performed according to the dispersion degree of the sampling points, to obtain a skeleton model composed of skeleton points.

[0054] wherein, the dispersion degree of the sampling points is described by three eigenvalues of the covariance matrix of the three-dimensional point set.

[0055] The extraction of the skeleton model is performed by selecting the median of the neighborhood of the sampling point for contraction, and iteratively updating to the center of the neighborhood, thereby generating a new skeleton point. It can be understood that the number of iterations can be set according to actual conditions, for example: 15-20 times.

[0056] For each sampling point in the set I, the local L1 median within a specified neighborhood radius r is iteratively solved according to the following formula:

[0057]

[0058]

[0059] wherein, is the new sampling point obtained after the k+1th iteration movement of x i , q j represents the target object point cloud set J with ​where x is the Gaussian weight function with r as the neighborhood radius, is the Euclidean distance between the sample point and the point in the neighborhood.

[0060] Since the neighboring sample points will eventually converge to a few cluster centers, this will result in the generated grasp poses being repetitive and the optional grasp points being less. Therefore, another constraint is introduced to prevent the sample points from being too clustered. The three eigenvalues of the covariance matrix of a three-dimensional point set represent the three principal directions of the point set. If the three eigenvalues are approximately the same, it means that the point set is distributed relatively dispersedly in the three-dimensional space, and it is likely to be located at the branch part of the skeleton. On the contrary, if the largest eigenvalue is much larger than the other two eigenvalues, it means that the point set is distributed in a certain direction, and it is likely to be located at the main part of the skeleton. The following formula is used to describe the distribution of the sample point set with x i as the center and the specified neighborhood radius:

[0061]

[0062] where, denotes the three eigenvalues obtained by solving the covariance matrix, and is the largest eigenvalue. i The closer to 1, the more points arranged on a branch, and a larger repulsive force is needed to disperse the sample points on the skeleton. The smaller i , the smaller the repulsive force needed when the sample points are located at the branch part of the skeleton to facilitate the sample points to converge together. Therefore, when iteratively solving the local L1 median, the following repulsive force term is added to automatically adjust the local repulsive force between the sample points,

[0063]

[0064]

[0065] where R k (x i ) is the repulsive force received by x i after the kth iteration from the neighborhood sample points, μ is a coefficient, and x i′ denotes the sample point set I with x as the center and the points in the neighborhood with a radius of r, is the Gaussian weight function with r as the neighborhood radius, is the Euclidean distance between the sample point and the point in the neighborhood.

[0066] In summary, the repulsive force term is added to the local L1 median to obtain the iterative formula of the final skeleton point:

[0067]

[0068] S3, determining a grasping pose corresponding to each skeleton point.

[0069] Specifically, the geometry information of the point cloud around the skeleton point and the distribution information of the skeleton model are analyzed to determine a grasping pose, and a candidate grasping pose set is obtained.

[0070] The determination of the grasping pose is performed by approaching the object along the normal direction and grasping along the main direction of the local skeleton model. The surface normal vector at the skeleton point and the main axis vector corresponding to the local skeleton model are calculated by using Principal Component Analysis (PCA).

[0071] Preferably, in step S3, for each skeleton point, a preset operation is performed to obtain a grasping pose corresponding to each skeleton point.

[0072] The preset operation includes: taking a region with the skeleton point as the center and a radius r as the target region; taking the target object point cloud and the points in the initial sampling point set located in the target region as the neighborhood point cloud, respectively, and constructing the covariance matrix of each neighborhood point cloud by using the PCA method; for the covariance matrix of the target object point cloud, taking the eigenvector corresponding to the smallest eigenvalue as the normal vector of the skeleton point; for the covariance matrix of the initial sampling point set, taking the eigenvector corresponding to the largest eigenvalue as the main axis vector of the skeleton point; and determining the grasping pose of the skeleton point according to the normal vector and the main axis vector.

[0073] Specifically, for each skeleton point s obtained in step S2, all neighborhood points in a region with the skeleton point as the center and a radius r are found to form a neighborhood point cloud. The covariance matrix M is constructed based on the neighborhood point cloud by using the PCA method. s

[0074]

[0075] where C r (s) represents the neighborhood point cloud in the region with the skeleton point s as the center and a radius r, p is one of the points, k is the number of points in the neighborhood point cloud, and p c is the center of all points in the neighborhood point cloud.

[0076] Three eigenvalues λ1> λ2> λ3(λ i ​R, i = 1, 2, 3) and the corresponding eigenvectors, wherein the eigenvector v corresponding to the largest eigenvalue λ1 is a local principal axis vector at the skeleton point, and the eigenvector n corresponding to the smallest eigenvalue λ3 is a local normal vector at the skeleton point. The origin of the camera coordinate system is p0, and the direction of the normal vector is selected to be the same as the direction of the camera view, i.e., satisfying n · (s - p0) > 0.

[0077] The surface normal vector of the skeleton point is calculated using the point cloud of the original object as the neighborhood point cloud, and the principal axis vector of the skeleton model is calculated using the point cloud composed of the skeleton points as the neighborhood point cloud. The grasping pose can be determined by the surface normal vector n at the skeleton point and the principal axis vector v of the skeleton model.

[0078] The closing direction of the gripper is perpendicular to the plane in which n and v are located, and three orthogonal unit vectors rx, ry, and rz can be obtained by using vector cross product and normalization. The three orthogonal unit vectors rx, ry, and rz represent the three coordinate axes of the gripper coordinate system F g g c The rotation matrix of the gripper coordinate system F g with respect to the camera coordinate system can be expressed as follows:

[0079]

[0080] The grasping position can be determined by the coordinates of the skeleton point s. The initial coordinate origin s of the gripper coordinate system F g is set. In order to avoid collision between the gripper and the object and the table, and to stably grasp the object with a larger depth, s is moved along the negative direction of the X axis by a distance m each time until the gripper does not produce collision and there is object point cloud in the closed space. The calculation method of the final grasping point O is as follows:

[0081] O = s - n · m · rx

[0082] wherein n is the number of reverse movements.

[0083] S4, evaluating each grasping pose to select the optimal grasping pose;

[0084] Preferably, in step S4, each grasping pose is evaluated according to the positional relationship between the gripper and the target object.

[0085] The positional relationship includes the maximum depth of the target object in the gripper, the distance between the target object and the left and right sides of the gripper, and the fitting degree of the boundary of the target object and the gripper.

[0086] Specifically, the positional relationship between the simplified gripper and the target object is analyzed, and the grasping poses in the candidate set are evaluated to achieve grasping optimization.

[0087] Furthermore, the positional relationship between the gripper and the object includes: the maximum depth of the object in the gripper, the distance between the object and the left and right sides of the gripper, and the degree of fit between the object boundary and the gripper.

[0088] Preferably, in step S4, each grasping pose is evaluated based on a heuristic evaluation function according to the positional relationship between the grasper and the target object.

[0089] The heuristic evaluation function is: s(g) = k l s l +k w s w +k b s b ;

[0090] in, k l k w k b Here, l and l' represent the weighting coefficients, respectively, the maximum depth of the target object in the gripper, and the gripper's gripping depth. left w right θ represents the distance between the target object and the left and right sides of the gripper, respectively. left θ right The slopes of the left and right contour boundaries of the gripper's gripping area are respectively fitted with straight lines and the acute angle between them and the Y-axis. The gripping area is obtained by projecting the object point cloud within the gripper's closed space onto the YOZ plane. O is the center point of the gripper's bottom, the gripper's closing direction is the Y-axis, and the normal vector of the skeleton point is the X-axis.

[0091] Specifically, the two-finger parallel gripper is first simplified as follows: Figure 2 As shown, the gripper has a thickness of h, a depth of l, and a maximum open width of w. The origin is set at the center of the gripper's bottom, the positive X-axis is defined as the direction approaching the object, the positive Y-axis is defined as the gripper closing direction, and the positive Z-axis is determined using the right-hand rule. This establishes the gripper coordinate system F. g To represent a six-DOF grasp pose

[0092] Considering the positional relationship between the gripper and the object, a heuristic evaluation function was designed as follows to evaluate the generated candidate grippers:

[0093] s(g)=k l s l +k w s w +k b s b

[0094] Among them, s lThis reflects the impact of gripping depth on gripping stability. The deeper the gripper grips, the less likely the object is to fall out. It is measured as the ratio of the object's maximum depth within the gripper to the depth of the grippers.

[0095]

[0096] s w This represents the effect of the gripping action on the object's translation; its value is the ratio of the distance between the object and the left and right sides of the gripper.

[0097]

[0098] s b This indicates the effect of gripping on the object's rotation, characterized by the contour of the contact surface between the gripper and the object. Specifically, for example... Figure 3 As shown, firstly, the point cloud of the object within the closed space of the grasper is extracted, and then it is projected onto the YOZ plane to obtain the grasping region. The alpha-shape concave hull extraction algorithm is used to extract the contour points of the grasping region. Then, the least squares method is used to perform line fitting on the contour boundaries formed by the contour boundary points on the left and right sides to obtain the slope k of the boundary lines. left and k right This allows us to obtain the acute angle θ between the angle and the Y-axis. left and θ right The larger θ is, the better the fit between the gripper and the object boundary, the smaller the torque generated, and the higher the stability. Therefore, s b The following can be calculated:

[0099]

[0100] The results of the three components in the evaluation function are all in the range of 0-1, but their influence on the crawling results is slightly different. This can be analyzed experimentally to adjust the three coefficients k. l k w k b Finally, the optimal capture is obtained by sorting the evaluation scores of the candidate captures.

[0101] S5, the optimal grasping pose is transformed from the camera coordinate system to the robot arm base coordinate system to perform a grasping operation on the target object.

[0102] Specifically, based on the hand-eye calibration results, the optimal grasping pose in the camera coordinate system is transferred to the robotic arm base coordinate system to perform the grasping action.

[0103] Preferably, in step S5, based on the hand-eye matrix of the depth camera relative to the end of the robotic arm, the optimal grasping pose is transformed from the camera coordinate system to the robotic arm base coordinate system.

[0104] Specifically, the optimal grasping pose T in the camera coordinate system is obtained pose2camera Then, it is converted to the pose of the robot arm base coordinate system for performing grasping by the following formula.

[0105] T pose2base = T gripper2base T camera2gripper T pose2camera

[0106] wherein T gripper2base is the pose of the robot arm gripper end relative to the robot arm base coordinate system, and T camera2gripper is the hand-eye matrix obtained in step S1.

[0107] The method provided by the present application will be described below with a specific example.

[0108] The experimental environment is Windows 10, the computer configuration is Intel(R) / i5 / 2.30GHz, 16GB RAM, and in the present application example, a Mitsubishi 6DOF robot arm is used, and an Intel RealSense D435i depth camera is used. The camera is installed at the end of the robot arm in the way of eye-in-hand. Common objects in life are used as experimental objects and placed in the working range of the robot arm in random poses. The stable grasping pose is generated by the six-degree-of-freedom grasping method described by the algorithm flowchart as shown in Figure 1 and handed over to the robot arm for execution.

[0109] The specific implementation steps are as follows:

[0110] Step S1: The depth camera is installed at the end of the robot arm, the checkerboard calibration board is fixed in the camera field of view, the robot arm is moved to different positions for multiple times, the pictures of the calibration board in different poses are obtained, the pixel coordinates of each corner point of the checkerboard are extracted, and the intrinsic parameters of the camera are calculated using Zhang's calibration method; in the moving process, the corresponding pose of the robot arm is recorded, and the hand-eye matrix T camera2gripper of the camera relative to the end of the robot arm is solved using Tsai's two-step method.

[0111] After the scene point cloud containing the target object is collected using the depth camera, the scene point cloud needs to be preprocessed. First, the random sample consensus algorithm is used to identify and segment the desktop point cloud, and only the target object point cloud is retained; in order to reduce the calculation amount, the target object is down-sampled by voxel, reducing the number of points in the point cloud; in order to prevent discrete points from interfering with the generation of the grasping pose, a statistical-based method is used to remove noise points.

[0112] Step S2: After obtaining the target object point cloud set J through the point cloud preprocessing of step S1, random down-sampling is performed to obtain an initial sampling point set I, and the local L1 median within a specified neighborhood radius r is iteratively solved according to the following formula for each sampling point in the set I. The local L1 median obtained by adding a repulsive term for local adjustment is uniformly distributed on the skeleton of the target object, as shown in Figure 4

[0113] Step S3: The point cloud of the original object is used as the neighborhood point cloud to calculate the surface normal vector of each skeleton point; the point cloud composed of the skeleton points is used as the neighborhood point cloud to calculate the principal axis vector of the skeleton model; and the corresponding grasping posture of each skeleton point is determined through the surface normal vector n at each skeleton point and the principal axis vector v of the skeleton model. As shown in Figure 5

[0114] Step S4: Each grasping posture is evaluated to select the optimal grasping posture.

[0115] Step S5: The optimal grasping posture is converted from the camera coordinate system to the mechanical arm base coordinate system to perform a grasping operation on the target object.

[0116] The embodiment of the application provides a six-degree-of-freedom grasping device for unknown objects considering point cloud skeleton features

[0117] A preprocessing module is configured to collect a scene point cloud including a target object by using a depth camera, and to obtain a target object point cloud J by preprocessing the scene point cloud; wherein the depth camera is installed at the end of a mechanical arm in an eye-in-hand manner.

[0118] An iterative adjustment module is configured to down-sample the target object point cloud to obtain an initial sampling point set I, and to iteratively adjust the position of each initial sampling point according to the formula to obtain each skeleton point.

[0119] wherein, is the position of the initial sampling point x i after k+1th iteration adjustment, J r is a set of points in J within a neighborhood with x as the center and a radius of r, I r is a set of points in I within a neighborhood with x as the center and a radius of r, q j , x i′ are points in J r , I r , respectively.

[0120] ​​The grasping pose module is configured to determine the grasping pose of each skeleton point.

[0121] The evaluation module is configured to evaluate each grasping pose to select an optimal grasping pose.

[0122] The conversion module is configured to convert the optimal grasping pose from the camera coordinate system to the base coordinate system of the robot arm, so as to perform the grasping operation on the target object.

[0123] The embodiment of the present application provides a six-degree-of-freedom grasping system for unknown objects considering point cloud skeleton features, comprising a computer readable storage medium and a processor.

[0124] The computer readable storage medium is configured to store executable instructions.

[0125] The processor is configured to read the executable instructions stored in the computer readable storage medium, and execute the method according to any one of the above embodiments.

[0126] The embodiment of the present application provides a computer readable storage medium, which stores computer instructions, and the computer instructions are used to make the processor execute the method according to any one of the above embodiments.

[0127] Those skilled in the art can easily understand that the above description is only the preferred embodiment of the present application, and is not used to limit the present application, and any modification, equivalent replacement and improvement made within the spirit and principle of the present application should be included in the protection scope of the present application.

Claims

1. A method for six degrees of freedom grasping of an unknown object considering the skeleton features of a point cloud, characterized in that, The method comprises: S1, collecting a scene point cloud including a target object by using a depth camera, and preprocessing the scene point cloud to obtain a target object point cloud J; wherein the depth camera is installed at the end of a mechanical arm in an eye-in-hand manner; S2, down-sampling the target object point cloud to obtain an initial sampling point set I according to the formula respectively adjusting the positions of the initial sampling points to obtain the skeleton points; wherein, is the position of the initial sampling point, is the position of the sampling point after the first iteration of adjustment, is the position of the sampling point after the second iteration of adjustment, r is the set of points in a neighborhood of with radius in J, r is the set of points in a neighborhood of with radius in I, , are points in J r , I r , respectively, ， ； S3, determining a grasping pose corresponding to each skeleton point; S4, evaluating each grasping pose to select an optimal grasping pose; wherein each grasping pose is evaluated according to a positional relationship between the gripper and the target object; the positional relationship includes: a maximum depth of the target object in the gripper, distances of the target object to left and right sides of the gripper, and a fitting degree of a boundary of the target object and the gripper; each grasping pose is evaluated according to the positional relationship between the gripper and the target object based on a heuristic evaluation function; The heuristic evaluation function is: ; wherein, , , , , , are weight coefficients, l and l' are the maximum depth of the target object in the gripper and the gripper jaw depth respectively, are the distances between the target object and the left and right sides of the gripper respectively, are the acute angles between the slope and the Y-axis obtained by linear fitting the contour boundaries of the left and right sides of the gripping area of the gripper, the gripping area is obtained by projecting the object point cloud in the closed space of the gripper to the YOZ plane, O is the center point of the jaw bottom, the closing direction of the jaw is the Y-axis, and the normal vector of the skeleton point is the X-axis; S5, converting the optimal grasping pose from a camera coordinate system to a mechanical arm base coordinate system to perform a grasping operation on the target object.

2. The method of claim 1, wherein, In step S3, for each skeleton point, a preset operation is performed to obtain a grasping pose corresponding to each skeleton point; The preset operation includes: taking a region with a skeleton point as a center point and a radius r as a target region; taking points in the target region in the target object point cloud and the initial sample point set as neighborhood point clouds respectively, and constructing covariance matrices of each neighborhood point cloud by using a PCA method; taking a feature vector corresponding to a minimum eigenvalue of the covariance matrix of the target object point cloud as a normal vector of the skeleton point; taking a feature vector corresponding to a maximum eigenvalue of the covariance matrix of the initial sample point set as a principal axis vector of the skeleton point; and determining the grasping pose of the skeleton point according to the normal vector and the principal axis vector.

3. The method of claim 1, wherein, In step S1, the preprocessing includes: identifying and segmenting a tabletop point cloud, voxel downsampling processing, and removing noise points.

4. The method of claim 1, wherein, In step S5, the optimal grasping pose is converted from the camera coordinate system to the mechanical arm base coordinate system according to a hand-eye matrix of the depth camera relative to the end of the mechanical arm.

5. An unknown object six-degree-of-freedom grasping device considering point cloud skeleton features, characterized in that, The method comprises: A preprocessing module is configured to collect a scene point cloud including a target object by using a depth camera, and preprocess the scene point cloud to obtain a target object point cloud J; wherein the depth camera is installed at the end of a mechanical arm in an eye-in-hand manner; An iterative adjustment module is configured to down-sample the target object point cloud to obtain an initial sampling point set I, and to adjust the position of each initial sampling point according to the formula The position of each initial sampling point is adjusted iteratively to obtain each skeleton point. wherein, is the position of the initial sampling point, is the position of the sampling point after the first iteration adjustment, is the position of the sampling point after the second iteration adjustment, r is the set of points in the neighborhood of with radius in J, r is the set of points in the neighborhood of with radius in I, , are points in J r and I r respectively, , ; A grasping pose module is configured to determine grasping poses of the skeleton points; An evaluation module is configured to evaluate each grasping pose to select an optimal grasping pose; wherein the evaluation module evaluates each grasping pose according to a positional relationship between the gripper and the target object; the positional relationship includes: a maximum depth of the target object in the gripper, distances of the target object to left and right sides of the gripper, and a fitting degree of a boundary of the target object and the gripper; The evaluation module evaluates each grasping pose according to the positional relationship between the gripper and the target object based on a heuristic evaluation function; The heuristic evaluation function is: ; wherein, , , , , , are weight coefficients, l and l' are the maximum depth of the target object in the gripper and the gripper jaw depth respectively, are the distances between the target object and the left and right sides of the gripper respectively, are the acute angles between the slopes obtained by linear fitting of the left and right profile boundaries of the gripped region of the gripper and the Y axis, the gripped region is obtained by projecting the object point cloud in the closed space of the gripper to the YOZ plane, O is the center point of the jaw bottom, the closed direction of the jaw is the Y axis, and the normal vector of the skeleton point is the X axis. A conversion module is configured to convert the optimal grasping pose from a camera coordinate system to a mechanical arm base coordinate system to perform a grasping operation on the target object.

6. An unknown object 6-DOF grasping system considering point cloud skeleton features, characterized in that, The method comprises: A computer readable storage medium and a processor; The computer readable storage medium is configured to store executable instructions; The processor is configured to read executable instructions stored in the computer readable storage medium and perform the method of any one of claims 1-4.

7. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer instructions configured to cause a processor to perform the method of any one of claims 1-4.

Images