A data processing method and pose estimation method for point cloud data

Abstract

The present invention discloses a data processing method and a pose estimation method for point cloud data. Each point in a scene point cloud is calculated respectively by a preset manifold calculation method, and a first normal vector corresponding to each point is obtained. After determining the first normal vector, the boundary points in the scene point cloud are determined in turn by point cloud boundary recognition, and a closed linear point cloud in the scene point cloud is determined based on the boundary points. The normal vector of the closed linear point cloud is recalculated by a preset principal component analysis method and a rotation index theorem, and the obtained second normal vector is provided as the normal vector of the corresponding closed linear point cloud for subsequent calculations. The normal vector calculated by the data processing method provided by the present invention is more accurate, and the accuracy of the pose estimation result obtained by the normal vector calculation is also further improved.

Description

Technical Field

[0001] The present invention relates to the field of graphics and visualization, and in particular to a data processing method and a pose estimation method for point cloud data. Background Art

[0002] In recent years, with the advancement of industrial upgrading, manufacturing automation has become a key driver of economic development. In manufacturing, the combination of machine vision and robotic arms to achieve automated workpiece grasping is increasingly being used on production lines. Therefore, applying machine vision technology to estimate the pose of target workpieces from 3D point clouds has become a key and challenging task in the industry. Existing techniques for pose estimation often use a method based on point pair features (PPFs), the most critical step of which is obtaining the normal vector of the point cloud to be calculated.

[0003] Existing technologies use common normal vector calculation methods for point cloud normal vector calculations, but fail to recognize that common normal vector calculation methods cannot accurately calculate the normal vectors of a point cloud with a special shape. This special point cloud is a closed-loop linear point cloud. If the acquired scene point cloud contains this closed-loop point cloud, the accuracy of the scene point cloud features calculated using the normal vector calculation method, and thus the pose estimated based on this scene point pair feature, will be greatly reduced. How to more effectively calculate normal vectors for this special point cloud has become a major challenge in the industry. Summary of the Invention

[0004] The present invention provides a data processing method, device and system for point cloud data, so as to achieve the technical effect of improving the accuracy of normal vectors and thus improving the accuracy of pose estimation.

[0005] In order to solve the above technical problems, the present invention provides a data processing method for point cloud data, comprising the following steps:

[0006] Retrieving a pre-acquired scene point cloud, and using a preset manifold calculation method to calculate each point in the scene point cloud and obtain a first normal vector corresponding to each point in the scene point cloud;

[0007] Determining boundary points in the scene point cloud by performing point cloud boundary recognition on each point in the scene point cloud, and determining a closed linear point cloud in the scene point cloud based on the boundary points;

[0008] Call the preset principal component analysis method to calculate and obtain the point cloud plane of the closed linear point cloud, calculate and determine the second normal vector of the closed linear point cloud through the rotation index theorem and the point cloud plane, and obtain the scene point cloud normal vector based on the integration of the second normal vector and the first normal vector of the points outside the closed linear point cloud.

[0009] The processing method provided by the present invention uses a preset manifold calculation method to calculate each point in the scene point cloud and obtain a one-to-one corresponding first normal vector. Unlike the normal vector calculation method used in the prior art, the normal vector calculation method used in the present invention is a manifold calculation method. The use of a manifold calculation method will improve the accuracy of the calculated normal vector and also better conform to the actual pose and morphology of the workpiece to be grasped. After determining the normal vector, this estimation method sequentially determines the boundary points in the scene point cloud through point cloud boundary identification, and then determines the closed linear point cloud in the scene point cloud based on the boundary points, thereby providing a data basis for subsequent adjustment of the normal vector of the closed linear point cloud.

[0010] At the same time, after determination, the processing method recalculates the normal vector of the closed linear point cloud through the preset principal component analysis method and rotation index theorem, and provides the obtained second normal vector as the normal vector of the corresponding closed linear point cloud to subsequent calculations, thereby improving the accuracy of the calculated normal vector and further improving the accuracy of the final pose calculated based on the normal vector.

[0011] As a preferred example, the preset manifold calculation method is used to calculate each point in the scene point cloud and obtain a first normal vector corresponding to each point in the scene point cloud, specifically:

[0012] Selecting and determining a point in the scene point cloud as a first starting point, searching for remaining points in the scene point cloud with the first starting point as the origin and a preset first length as the radius to form a first open cover, and marking all points in the first open cover;

[0013] After marking, select and determine a point in the scene point cloud other than the first starting point as a second starting point, use the second starting point as the origin, and search for remaining points in the scene point cloud with a preset first length as a radius to form a second open cover, and mark all points in the second open cover until all points in the scene point cloud are marked;

[0014] Calculate the normal vector corresponding to the first starting point to obtain an initial normal vector. Taking the initial normal vector as the standard, calculate the inner product of the normal vectors of other points in the scene point cloud except the first starting point and the initial normal vector, and negate the corresponding normal vectors whose inner products are negative, so that the inner products of the normal vectors of each point in the scene point cloud and the initial normal vector are all positive numbers, and use the adjusted normal vectors of each point in the scene point cloud as the first normal vector.

[0015] The specific implementation of the manifold calculation method provided by this invention calculates the local coordinate system of the manifold based on its specific definition, namely the open cover determination method provided by this invention. The accuracy of the point cloud normal vectors calculated using this manifold calculation method is improved, thereby improving the accuracy of the point pair features subsequently calculated using these normal vectors, further improving the accuracy of the estimated pose.

[0016] At the same time, the open coverage determination method provided by the present invention fully meets the conditions for adjusting and inverting the normal vector. Since there must be an intersection between the two open covers in the scene point cloud, after completing the normal vector inversion adjustment of all points in an open cover, there must be an open cover containing a point corresponding to the adjusted normal vector in other open covers in the scene point cloud except the adjusted open cover. Therefore, the open coverage determination method provided by the present invention can ensure the integrity of the subsequent normal vector inversion adjustment of each point in the scene point cloud.

[0017] In addition, the step of performing inverted adjustment on the normal vector also improves the accuracy of the first normal vector.

[0018] As a preferred example, the step of determining the boundary points in the scene point cloud by performing point cloud boundary recognition on each point in the scene point cloud is specifically as follows:

[0019] Calling the preset principal component analysis method to calculate a first fitting plane of a neighborhood corresponding to a first starting point in the scene point cloud, and projecting all points in the neighborhood corresponding to the first starting point onto the first fitting plane;

[0020] All points on the first fitting plane are traversed in a counterclockwise direction and characteristic angles of all points are calculated, and points with characteristic angles greater than a first threshold are taken as boundary points in the scene point cloud.

[0021] The present invention provides a point cloud boundary recognition method, which calculates the fitting plane of the point neighborhood in the point cloud by calling the principal component analysis method, and determines whether the point corresponding to the fitting plane is a boundary point based on the characteristic angles of all points on the fitting plane, and then determines all the boundary points in the scene point cloud, providing a data basis for the subsequent determination of a closed linear point cloud based on the boundary points.

[0022] As a preferred example, the second normal vector of the closed linear point cloud is calculated and determined by the rotation index theorem and the point cloud plane, specifically:

[0023] Calling the rotation index theorem and a preset point cloud sequence to calculate the closed linear point cloud, and determining the vector direction of the second normal vector according to the calculation result;

[0024] The second normal vector is determined according to the vector direction and the point cloud plane.

[0025] The present invention provides a method for determining a second normal vector, which determines the vector direction of the second normal vector according to the rotation index theorem and a preset point cloud order. After determination, the second normal vector is determined according to the determined vector direction and the point cloud plane determined in the previous text according to the closed linear point cloud. This avoids the problem of inaccurate normal vector calculation due to the existence of closed linear point clouds, and also avoids the problem of high error rate of pose estimation due to inaccurate normal vector calculation, improves the accuracy of normal vector calculation, and further improves the accuracy of pose estimated based on the normal vector.

[0026] Accordingly, the present invention also provides a method for estimating a pose based on point cloud data, comprising the following steps:

[0027] receiving a first model point cloud of a workpiece to be grasped input by a user, and collecting a first scene point cloud of the workpiece to be grasped;

[0028] Respectively performing scaling processing and downsampling processing on the first model point cloud and the first scene point cloud in sequence to obtain a model point cloud and a scene point cloud;

[0029] Performing normal vector calculation on the model point cloud and the scene point cloud respectively to obtain a model point cloud normal vector and a scene point cloud normal vector; wherein the scene point cloud normal vector is generated by processing the point cloud data data processing method described in any one of the above items;

[0030] The pose estimation result of the workpiece to be grasped is determined by calculation based on the model point cloud, the model point cloud normal vector, the scene point cloud, and the scene point cloud normal vector.

[0031] In order to better utilize the scene normal vectors obtained according to the above data processing method, the present invention also provides a pose estimation method, which performs scaling processing on the model point cloud and the scene point cloud to make the sizes of the two consistent, thereby making it more convenient to subsequently compare the point pair features generated by the above two to obtain approximate point pair features.

[0032] At the same time, the present invention also provides a step for downsampling the scaled scene point cloud and model point cloud. This step reduces the density of points in the above two point clouds, thereby reducing the computing resources required for subsequent calculations on the above two point clouds, thereby improving the computing efficiency of the estimation method provided by the present invention.

[0033] The normal vector of the processed model point cloud is calculated to obtain the corresponding model normal vector, and the pose estimation result of the grasped workpiece is determined together with the scene normal vector obtained by the above processing method, thereby improving the accuracy of the pose estimation result.

[0034] As a preferred example, the calculation and determination of the pose estimation result of the workpiece to be grasped based on the model point cloud, the model point cloud normal vector, the scene point cloud, and the scene point cloud normal vector specifically includes:

[0035] Performing point pair feature calculation on all points in the model point cloud according to the model point cloud normal vector, and constructing a model hash table according to the calculated model point pair features;

[0036] Calculating each point in the scene point cloud according to the scene point cloud normal vector to obtain a scene point pair feature corresponding to each point in the scene point cloud;

[0037] An initial pose is obtained by calculating the scene point pair features and the model hash table, and the initial pose is processed to obtain the pose estimation result.

[0038] The present invention provides point pair feature calculation for the model point cloud and establishment of a hash table based on the calculation results, which improves the efficiency of subsequent calculation and processing of the model point pair features; traverses the scene point cloud and calculates the corresponding scene point pair features for each point in the scene point cloud respectively, and after determination, obtains the estimated result of the initial pose based on the model point pair features and scene point pair features in the above hash table, providing a data basis for the subsequent determination of the pose estimation result.

[0039] As a preferred example, the initial pose is obtained by calculating the scene point pair features and the model hash table, specifically:

[0040] traversing the model hash table according to the scene point pair feature to determine an approximate point pair feature of the scene point pair feature, and voting for a hash table position corresponding to the approximate point pair feature in the model hash table;

[0041] A first position queue is determined according to the voting result, and the initial pose is obtained by calculation according to the first position queue and the corresponding scene point.

[0042] The present invention provides a specific method for calculating and obtaining an initial pose estimate based on the model point pair features and the scene point pair features, namely the PPF voting strategy. Specifically, the model hash table is traversed according to the scene point pair features, the scene point pair features are compared with all the model point pair features in the model hash table, and the model point pair features similar to the scene point pair features are determined based on the comparison results, and the similar model point pair features are used as approximate point pair features. After determining the approximate point pair features, the positions of the corresponding model point pair features in the model hash table are voted, and the first position queue is determined based on the voting results to obtain the initial pose. The calculation method provided by the present invention can stably and accurately determine the initial pose, providing a data basis for subsequent pose estimation results.

[0043] As a preferred example, the processing of the initial pose to obtain the pose estimation result specifically includes:

[0044] Performing a gravity center transformation on the initial posture, and calculating a distance between the gravity center of the transformed initial posture and the gravity center of the initial posture;

[0045] If the distance is greater than the first distance, determining the corresponding initial posture as the result posture, and sorting the result postures;

[0046] The pose estimation result is determined according to the sorting result, and the pose estimation result is sent to the grasping device so that the grasping device grasps the workpiece to be grasped according to the pose estimation result.

[0047] To further improve the accuracy of pose estimation, the present invention also provides a clustering method based on a center of gravity transformation approach. This method determines the resulting pose, sorts the resulting poses, and determines the pose estimation result based on the sorting results. Clustering the initial poses using the clustering method provided by the present invention improves the accuracy of the pose estimation results and is also more stable than existing clustering methods.

[0048] Accordingly, the present invention also provides a data processing device for point cloud data, the data processing device comprising: a manifold calculation module, a closure determination module, and a vector adjustment module;

[0049] The manifold calculation module is used to retrieve a pre-acquired scene point cloud, and use a preset manifold calculation method to calculate each point in the scene point cloud and obtain a first normal vector corresponding to each point in the scene point cloud;

[0050] The closure determination module is used to determine boundary points in the scene point cloud by performing point cloud boundary recognition on each point in the scene point cloud, and determine a closed linear point cloud in the scene point cloud based on the boundary points;

[0051] The vector adjustment module is used to call the preset principal component analysis method to calculate and obtain the point cloud plane of the closed linear point cloud, calculate and determine the second normal vector of the closed linear point cloud through the rotation index theorem and the point cloud plane, and obtain the scene point cloud normal vector based on the integration of the second normal vector and the first normal vector of the points outside the closed linear point cloud.

[0052] Accordingly, the present invention also provides a posture estimation device based on point cloud data, the posture estimation device comprising a point cloud acquisition module, a point cloud processing module, a normal vector calculation module and a posture estimation module;

[0053] The point cloud acquisition module is used to receive a first model point cloud of a workpiece to be grasped input by a user, and to acquire a first scene point cloud of the workpiece to be grasped;

[0054] The point cloud processing module is used to sequentially perform scaling processing and downsampling processing on the first model point cloud and the first scene point cloud to obtain a model point cloud and a scene point cloud;

[0055] The normal vector calculation module is used to calculate the normal vectors of the model point cloud and the scene point cloud respectively to obtain the model point cloud normal vector and the scene point cloud normal vector; wherein the scene point cloud normal vector is generated by processing the point cloud data data processing method described in any one of the above items;

[0056] The pose estimation module is used to calculate and determine the pose estimation result of the workpiece to be grasped based on the model point cloud, the model point cloud normal vector and the scene point cloud, the scene point cloud normal vector. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 : A schematic flow chart of an embodiment of a method for processing point cloud data provided by the present invention;

[0058] Figure 2 : A schematic flow chart of an embodiment of step 101 provided by the present invention;

[0059] Figure 3 : A flow chart of an embodiment of step 102 provided by the present invention;

[0060] Figure 4 : A schematic flow chart of an embodiment of a method for estimating a pose based on point cloud data provided by the present invention;

[0061] Figure 5 : A schematic flow chart of an embodiment of step 403 provided by the present invention;

[0062] Figure 6 : A structural diagram of an embodiment of a data processing device for point cloud data provided by the present invention;

[0063] Figure 7 : A structural diagram of an embodiment of a posture estimation device based on point cloud data provided by the present invention. DETAILED DESCRIPTION

[0064] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0065] Example 1

[0066] Please refer to Figure 1 , which is a flow chart of an embodiment of a method for processing point cloud data provided by an embodiment of the present invention, including steps 101 to 103, each of which is specifically as follows:

[0067] Step 101: Retrieve a pre-acquired scene point cloud, and use a preset manifold calculation method to calculate each point in the scene point cloud and obtain a first normal vector corresponding to each point in the scene point cloud.

[0068] In this embodiment, each point in the scene point cloud is calculated separately by a preset manifold calculation method, and a one-to-one corresponding first normal vector is obtained. Different from the normal vector calculation method used in the prior art, the normal vector calculation method used in the present invention is a manifold calculation method. The use of the manifold calculation method will improve the accuracy of the calculated normal vector, and it is also more in line with the actual posture of the workpiece to be grasped.

[0069] Step 102: performing point cloud boundary recognition on each point in the scene point cloud to determine boundary points in the scene point cloud, and determining closed linear point clouds in the scene point cloud based on the boundary points.

[0070] In this embodiment, after determining the normal vector, the estimation method determines the boundary points in the scene point cloud through point cloud boundary identification in turn, and determines the closed linear point cloud in the scene point cloud based on the boundary points, thereby providing a data basis for subsequent adjustment of the normal vector of the closed linear point cloud.

[0071] Step 103: Call the preset principal component analysis method to calculate and obtain the point cloud plane of the closed linear point cloud, calculate and determine the second normal vector of the closed linear point cloud through the rotation index theorem and the point cloud plane, and obtain the scene point cloud normal vector based on the integration of the second normal vector and the first normal vector of the points outside the closed linear point cloud.

[0072] In this embodiment, the rotation index theorem and the preset point cloud order are called to calculate the closed linear point cloud, and the vector direction of the second normal vector is determined according to the calculation result. After determination, the second normal vector of the closed current point cloud is determined according to the vector direction and the point cloud plane.

[0073] Among them, the preset principal component analysis method is called to calculate and obtain the point cloud plane of the closed linear point cloud. Specifically, the three eigenvectors of the closed linear point cloud are determined by the preset principal component analysis method, and the three eigenvalues ​​corresponding to the three eigenvectors are determined. The above three eigenvalues ​​are sorted, and the eigenvectors corresponding to the two eigenvalues ​​with smaller values ​​are formed into a plane, which is the point cloud plane.

[0074] In order to apply the rotation index theorem, this embodiment presets the point cloud order for the closed current point cloud. The rotation index theorem is specifically the rotation index i of the plane regular second-order continuous differentiable simple closed curve C. r =±1.

[0075] The second normal vector of the closed current point cloud is determined according to the vector direction and the point cloud plane as follows: a point p is preset, and the previous point of the point p is set as p1 and the next point of the point p is set as p2 according to the point cloud sequence. Then, there are vectors (p-p1) and (p2-p), and then there is a vector and Where × represents the outer product, represents the direction of a vector, and the vector and The average value of is the second normal vector of the point p.

[0076] The second normal vector obtained by the above method is used as the normal vector of the closed current point cloud, and the first normal vectors of other point clouds in the scene point cloud except the closed current point cloud are integrated to form the scene point cloud normal vector, thereby improving the accuracy of the scene point cloud normal vector and further improving the accuracy of the final pose calculated based on the normal vector.

[0077] As another example of this embodiment, see Figure 2 , Figure 2 The flowchart of an embodiment of step 101 provided by the present invention includes steps 201 to 203, and each step is specifically as follows:

[0078] Step 201: Select and determine a point in the scene point cloud as a first starting point, use the first starting point as the origin, and search for the remaining points in the scene point cloud with a preset first length as the radius to form a first open cover, and mark all points in the first open cover.

[0079] In this example, the manifold is calculated specifically by calculating the local coordinate system of the manifold according to the definition of the manifold, that is, determining the first open cover corresponding to the first starting point in the above steps, and the first open cover is a set of point cloud subsets corresponding to the first starting point.

[0080] In this example, the preset first length can be adjusted by the user based on actual needs. The first length provided in this example is preferably one-fiftieth of the minimum bounding box of the scene point cloud. The minimum bounding box of the scene point cloud is determined by determining the point cloud length L with the largest value among the lengths corresponding to the scene point cloud in the X-axis, Y-axis, and Z-axis directions. After determining L, the quotient of the point cloud lengths corresponding to the scene point cloud in the X-axis, Y-axis, and Z-axis directions and L is calculated. The combination of the three obtained point cloud lengths is the minimum bounding box of the scene point cloud.

[0081] Step 202: After marking, select and determine a point as a second starting point from other points in the scene point cloud except the first starting point, use the second starting point as the origin, and search for the remaining points in the scene point cloud with a preset first length as the radius to form a second open cover, and mark all points in the second open cover until all points in the scene point cloud are marked.

[0082] Through the above operations, the execution is completed until all points in the scene point cloud are marked, and the union of several open covers formed is the scene point cloud. Since there is an intersection between the open covers, the connectivity between the open covers formed above can be guaranteed, which provides a physical basis for the subsequent adjustment of the normal vector.

[0083] The above method fully meets the conditions for adjusting and inverting the normal vector. Since there must be an intersection between the two open covers in the scene point cloud, after completing the normal vector inversion adjustment of all points in an open cover, there must be an open cover containing a point corresponding to the adjusted normal vector in other open covers in the scene point cloud except for the adjusted open cover. Therefore, the open cover determination method provided by the present invention can ensure the integrity of the subsequent normal vector inversion adjustment of each point in the scene point cloud.

[0084] Step 203: Calculate the normal vector corresponding to the first starting point to obtain an initial normal vector. Taking the initial normal vector as the standard, calculate the inner product of the normal vectors of other points in the scene point cloud except the first starting point and the initial normal vector, and invert the corresponding normal vectors whose inner products are negative, so that the inner products of the normal vectors of each point in the scene point cloud and the initial normal vector are all positive numbers, and use the adjusted normal vectors of each point in the scene point cloud as the first normal vector.

[0085] Since it is determined in step 202 that an open cover must have another open cover that has an intersection with the open cover, the inner product adjustment is performed on the other points in the first open cover except the first starting point based on the first starting point, until the normal vectors of all points in the first open cover are adjusted, the normal vectors of the points in other open covers can be adjusted through the intersection of the first open cover and the other open covers in the scene point cloud except the first open cover, until the normal vectors of all points in the scene point cloud are adjusted, and the first normal vector of the scene point cloud is obtained.

[0086] In summary, the specific implementation of the manifold calculation method provided by the present invention calculates the local coordinate system of the manifold based on its specific definition, namely, the open cover determination method provided by the present invention. The accuracy of the point cloud normal vectors calculated using this manifold calculation method is improved, thereby improving the accuracy of the point pair features subsequently calculated using these normal vectors, further improving the accuracy of the estimated pose.

[0087] As another example of this embodiment, see Figure 3 , Figure 3 The flowchart of an embodiment of step 102 provided by the present invention includes steps 301 to 302, and each step is specifically as follows:

[0088] Step 301: Call the preset principal component analysis method to calculate a first fitting plane of a neighborhood corresponding to a first starting point in the scene point cloud, and project all points in the neighborhood corresponding to the first starting point onto the first fitting plane.

[0089] In this example, the method for determining the boundary points in the scene point cloud is Angle Criterion, which is the AC edge detection algorithm in the PLC point cloud edge detection algorithm. Specifically, the fitting plane of the neighborhood corresponding to each point in the scene point cloud is determined by the preset principal component analysis method, and all points in the corresponding neighborhood are projected onto the fitting plane.

[0090] Step 302: traverse all points on the first fitting plane in a counterclockwise direction and calculate the characteristic angles of all points, and use points with characteristic angles greater than a first threshold as boundary points in the scene point cloud.

[0091] After determining all boundary points in the scene point cloud using the above method, several boundary points are connected to form several boundary paths. After forming each boundary path, a breadth-first exploration is performed on each boundary path. Based on the exploration results, it is determined whether the corresponding boundary path is a closed linear point cloud. If the exploration depth is 1, the corresponding boundary path is a closed linear point cloud; if the exploration depth is not 1, the corresponding boundary path is not a closed linear point cloud.

[0092] After the boundary points are determined by the above method, the closed linear point cloud can be judged and calculated based on the determined boundary points.

[0093] Example 2

[0094] See also Figure 4 , Figure 4 A flow chart of an embodiment of a method for estimating a pose based on point cloud data provided by the present invention includes steps 401 to 403, each of which is specifically as follows:

[0095] Step 401: receiving a first model point cloud of a workpiece to be grasped input by a user, collecting a first scene point cloud of the workpiece to be grasped, and performing scaling processing and downsampling processing on the first model point cloud and the first scene point cloud respectively to obtain a model point cloud and a scene point cloud.

[0096] In order to better utilize the scene normal vectors obtained according to the above data processing method, the present invention also provides a pose estimation method, which performs scaling processing on the model point cloud and the scene point cloud to make the sizes of the two consistent, thereby making it more convenient to subsequently compare the point pair features generated by the above two to obtain approximate point pair features.

[0097] In this embodiment, the model point cloud and the scene point cloud are scaled, specifically by calculating the minimum bounding box of the model point cloud and the scene point cloud. The calculation of the minimum bounding box of the model point cloud specifically involves determining the point cloud length S with the largest value among the point cloud lengths corresponding to the model point cloud in the X-axis, Y-axis, and Z-axis directions. After determining S, the quotient of the point cloud length of the scene point cloud in the X-axis, Y-axis, and Z-axis directions and S is calculated. The three quotients, i.e., the three point cloud lengths, are combined to form the minimum bounding box of the model point cloud.

[0098] At the same time, an embodiment of the present invention also provides a step for downsampling the scaled scene point cloud and model point cloud. This step reduces the density of points in the above two point clouds, thereby reducing the computing resources required for subsequent calculations on the above two point clouds, thereby improving the computing efficiency of the estimation method provided by the present invention.

[0099] In this embodiment, the specific implementation method for downsampling the model point cloud and the scene point cloud is to perform voxel segmentation on the minimum bounding box formed by the above-mentioned scaling processing method, and take the average value of the point set within each voxel after segmentation. The averaged point is the downsampled point.

[0100] The model point cloud and scene point cloud determined by the above method provide a data basis for subsequent normal vector calculation for the model point cloud and scene point cloud.

[0101] Step 402: Calculate normal vectors for the model point cloud and the scene point cloud respectively to obtain the model point cloud normal vector and the scene point cloud normal vector.

[0102] In this embodiment, the method for calculating the normal vector of the model point cloud to obtain the model normal vector is specifically as follows: Figure 1 The preset manifold calculation method described in step 101, and the calculation method for obtaining the scene normal vector by performing normal vector calculation on the scene point cloud in this embodiment adopts the data processing method of the point cloud data described in the first embodiment.

[0103] Step 403: Calculate and determine a pose estimation result of the workpiece to be grasped based on the model point cloud, the model point cloud normal vector, the scene point cloud, and the scene point cloud normal vector.

[0104] The accuracy of the pose estimation result of the workpiece to be grasped determined by the above method will be effectively improved.

[0105] As another example of this embodiment, see Figure 5 , Figure 5 The flowchart of an embodiment of step 403 provided by the present invention includes steps 501 to 503, and each step is specifically as follows:

[0106] Step 501: performing point pair feature calculation on all points in the model point cloud according to the model point cloud normal vector, and constructing a model hash table according to the calculated model point pair features.

[0107] In order to better perform traversal operations on model point pair features in the future, this example provides a method for constructing a hash table for model point pair features, thereby reducing subsequent computing resource consumption and improving the computing efficiency of subsequent traversal operations on model point pair features.

[0108] In the hash table, point pairs with the same characteristics will be connected together and associated with each other, that is, the key of the hash table is the corresponding point pair feature, and the value corresponding to the key is the set of point pairs with the same characteristics.

[0109] Step 502: Calculate each point in the scene point cloud according to the scene point cloud normal vector to obtain a scene point pair feature corresponding one-to-one to each point in the scene point cloud.

[0110] Step 503: Obtain an initial pose by calculating the scene point pair features and the model hash table, and process the initial pose to obtain the pose estimation result.

[0111] In this example, the specific implementation method for determining the initial pose through scene point pair features and model hash table is to traverse the model hash table according to the scene point pair features to determine the approximate point pair features of the scene point pair features, and vote for the hash table positions corresponding to the approximate point pair features in the model hash table, and determine the first position queue according to the voting results, and calculate the initial pose according to the first position queue and the corresponding scene point.

[0112] The above determination method is the PPF voting strategy, which can stably and accurately determine the initial pose and provide a data basis for subsequent pose estimation results.

[0113] At the same time, to further improve the accuracy of the pose estimation results, an embodiment of the present invention also provides a clustering method based on a center of gravity transformation method. The resulting pose is determined by the center of gravity transformation method, and the resulting poses are sorted. The pose estimation result is determined based on the sorting result. Using the clustering method provided by the present invention to cluster the initial poses improves the accuracy of the obtained pose estimation results. At the same time, the stability of this clustering method is also higher than that provided by the clustering method provided by the prior art.

[0114] Among them, the clustering method of the center of gravity transformation is to perform a center of gravity transformation on the initial posture, calculate the distance between the center of gravity of the transformed initial posture and the center of gravity of the initial posture, if the distance is greater than the first distance, then the corresponding initial posture is determined as the result posture, and the result posture is sorted, and the posture estimation result is determined according to the sorting result, and the posture estimation result is sent to the grasping device so that the grasping device grasps the workpiece to be grasped according to the posture estimation result.

[0115] In order to better illustrate the working principle and step flow of the data processing method and pose estimation method of point cloud data of the present invention, you can refer to but are not limited to the relevant records above.

[0116] Accordingly, see Figure 6 , Figure 6 This is a schematic diagram of the structure of an embodiment of the data processing device for point cloud data provided by the present invention. Figure 6 As shown, the processing device includes a manifold calculation module 601 , a closure determination module 602 and a vector adjustment module 603 .

[0117] The manifold calculation module 601 is used to retrieve a pre-acquired scene point cloud, and use a preset manifold calculation method to calculate each point in the scene point cloud and obtain a first normal vector corresponding to each point in the scene point cloud.

[0118] The closure determination module 602 is configured to determine boundary points in the scene point cloud by performing point cloud boundary recognition on each point in the scene point cloud, and determine a closed linear point cloud in the scene point cloud based on the boundary points;

[0119] The vector adjustment module 603 is used to call the preset principal component analysis method to calculate and obtain the point cloud plane of the closed linear point cloud, calculate and determine the second normal vector of the closed linear point cloud through the rotation index theorem and the point cloud plane, and obtain the scene point cloud normal vector based on the integration of the second normal vector and the first normal vector of the points outside the closed linear point cloud.

[0120] Accordingly, see Figure 7 , Figure 7 This is a schematic diagram of the structure of an embodiment of the device for estimating the posture based on point cloud data provided by the present invention. Figure 7 As shown, the estimation device includes a point cloud acquisition module 701, a point cloud processing module 702, a normal vector calculation module 703 and a pose estimation module 704.

[0121] The point cloud acquisition module 701 is configured to receive a first model point cloud of a workpiece to be grasped input by a user, and acquire a first scene point cloud of the workpiece to be grasped;

[0122] The point cloud processing module 702 is used to sequentially perform scaling processing and downsampling processing on the first model point cloud and the first scene point cloud to obtain a model point cloud and a scene point cloud;

[0123] The normal vector calculation module 703 is used to calculate the normal vectors of the model point cloud and the scene point cloud respectively to obtain the model point cloud normal vector and the scene point cloud normal vector; wherein the scene point cloud normal vector is generated by processing the point cloud data data processing method described in any one of the above items;

[0124] The pose estimation module 704 is used to calculate and determine a pose estimation result of the workpiece to be grasped based on the model point cloud, the model point cloud normal vector, the scene point cloud, and the scene point cloud normal vector.

[0125] In summary, the present invention provides a data processing method and a pose estimation method for point cloud data, which calculates each point in the scene point cloud respectively through a preset manifold calculation method, and obtains the first normal vector corresponding to each point. After determining the first normal vector, the boundary points in the scene point cloud are determined in turn through point cloud boundary identification, and the closed linear point cloud in the scene point cloud is determined based on the boundary points. The normal vector of the closed linear point cloud is recalculated through a preset principal component analysis method and rotation index theorem, and the obtained second normal vector is provided as the normal vector of the corresponding closed linear point cloud for subsequent calculations. The normal vector calculated by the data processing method provided by the present invention is more accurate, and the accuracy of the pose estimation result obtained by the normal vector calculation is also further improved.

[0126] The specific embodiments described above further illustrate the objectives, technical solutions, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. In particular, it should be noted that any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included within the scope of protection of the present invention for those skilled in the art.

Claims

1. A method for processing point cloud data, characterized in that: The following steps are involved: Retrieving a pre-acquired scene point cloud, and using a preset manifold calculation method to calculate each point in the scene point cloud and obtain a first normal vector corresponding to each point in the scene point cloud; Determining boundary points in the scene point cloud by performing point cloud boundary recognition on each point in the scene point cloud, and determining a closed linear point cloud in the scene point cloud based on the boundary points; Call the preset principal component analysis method to calculate and obtain the point cloud plane of the closed linear point cloud, calculate and determine the second normal vector of the closed linear point cloud through the rotation index theorem and the point cloud plane, and obtain the scene point cloud normal vector based on the integration of the second normal vector and the first normal vectors of other point clouds in the scene point cloud except the closed linear point cloud.

2. The method for processing point cloud data according to claim 1, wherein: The method of using a preset manifold calculation method to calculate each point in the scene point cloud and obtain a first normal vector corresponding to each point in the scene point cloud is specifically as follows: Selecting and determining a point in the scene point cloud as a first starting point, searching for remaining points in the scene point cloud with the first starting point as the origin and a preset first length as the radius to form a first open cover, and marking all points in the first open cover; After marking, select and determine a point in the scene point cloud other than the first starting point as a second starting point, use the second starting point as the origin, and search for remaining points in the scene point cloud with a preset first length as a radius to form a second open cover, and mark all points in the second open cover until all points in the scene point cloud are marked; Calculate the normal vector corresponding to the first starting point to obtain an initial normal vector. Taking the initial normal vector as the standard, calculate the inner product of the normal vectors of other points in the scene point cloud except the first starting point and the initial normal vector, and negate the corresponding normal vectors whose inner products are negative, so that the inner products of the normal vectors of each point in the scene point cloud and the initial normal vector are all positive numbers, and use the adjusted normal vectors of each point in the scene point cloud as the first normal vector.

3. The method for processing point cloud data according to claim 2, wherein: The step of determining the boundary points in the scene point cloud by performing point cloud boundary recognition on each point in the scene point cloud is specifically as follows: Calling the preset principal component analysis method to calculate a first fitting plane of a neighborhood corresponding to a first starting point in the scene point cloud, and projecting all points in the neighborhood corresponding to the first starting point onto the first fitting plane; All points on the first fitting plane are traversed in a counterclockwise direction and characteristic angles of all points are calculated, and points with characteristic angles greater than a first threshold are taken as boundary points in the scene point cloud.

4. The method for processing point cloud data according to claim 1, wherein: The second normal vector of the closed linear point cloud is calculated and determined by the rotation index theorem and the point cloud plane as follows: Calling the rotation index theorem and a preset point cloud sequence to calculate the closed linear point cloud, and determining the vector direction of the second normal vector according to the calculation result; The second normal vector is determined according to the vector direction and the point cloud plane.

5. A pose estimation method based on point cloud data, characterized in that: The following steps are involved: receiving a first model point cloud of a workpiece to be grasped input by a user, and collecting a first scene point cloud of the workpiece to be grasped; Respectively performing scaling processing and downsampling processing on the first model point cloud and the first scene point cloud in sequence to obtain a model point cloud and a scene point cloud; Normal vectors are calculated for the model point cloud and the scene point cloud respectively to obtain a model point cloud normal vector and a scene point cloud normal vector; wherein the scene point cloud normal vector is generated by processing the point cloud data data processing method according to any one of claims 1 to 4; The pose estimation result of the workpiece to be grasped is determined by calculation based on the model point cloud, the model point cloud normal vector, the scene point cloud, and the scene point cloud normal vector.

6. The method for estimating a pose based on point cloud data according to claim 5, wherein: The calculating and determining the pose estimation result of the workpiece to be grasped based on the model point cloud, the model point cloud normal vector, the scene point cloud, and the scene point cloud normal vector specifically includes: Performing point pair feature calculation on all points in the model point cloud according to the model point cloud normal vector, and constructing a model hash table according to the calculated model point pair features; Calculating each point in the scene point cloud according to the scene point cloud normal vector to obtain a scene point pair feature corresponding to each point in the scene point cloud; An initial pose is obtained by calculating the scene point pair features and the model hash table, and the initial pose is processed to obtain the pose estimation result.

7. The method for estimating a pose based on point cloud data according to claim 6, wherein: The initial pose is obtained by calculating the scene point pair features and the model hash table, specifically: traversing the model hash table according to the scene point pair feature to determine an approximate point pair feature of the scene point pair feature, and voting for a hash table position corresponding to the approximate point pair feature in the model hash table; A first position queue is determined according to the voting result, and the initial pose is obtained by calculating features of the first position queue and the corresponding scene point pairs.

8. The method for estimating a pose based on point cloud data according to claim 7, wherein: The processing of the initial pose to obtain the pose estimation result specifically includes: Performing a gravity center transformation on the initial posture, and calculating a distance between the gravity center of the transformed initial posture and the gravity center of the initial posture; If the distance is greater than the first distance, determining the corresponding initial posture as the result posture, and sorting the result postures; The pose estimation result is determined according to the sorting result, and the pose estimation result is sent to the grasping device so that the grasping device grasps the workpiece to be grasped according to the pose estimation result.

9. A data processing device for point cloud data, characterized in that: The data processing device includes: a manifold calculation module, a closure determination module and a vector adjustment module; The manifold calculation module is used to retrieve a pre-acquired scene point cloud, and use a preset manifold calculation method to calculate each point in the scene point cloud and obtain a first normal vector corresponding to each point in the scene point cloud; The closure determination module is used to determine boundary points in the scene point cloud by performing point cloud boundary recognition on each point in the scene point cloud, and determine a closed linear point cloud in the scene point cloud based on the boundary points; The vector adjustment module is used to call the preset principal component analysis method to calculate and obtain the point cloud plane of the closed linear point cloud, calculate and determine the second normal vector of the closed linear point cloud through the rotation index theorem and the point cloud plane, and obtain the scene point cloud normal vector based on the integration of the second normal vector and the first normal vectors of other point clouds in the scene point cloud except the closed linear point cloud.

10. A pose estimation device based on point cloud data, characterized in that: The posture estimation device includes a point cloud acquisition module, a point cloud processing module, a normal vector calculation module and a posture estimation module; The point cloud acquisition module is used to receive a first model point cloud of a workpiece to be grasped input by a user, and to acquire a first scene point cloud of the workpiece to be grasped; The point cloud processing module is used to sequentially perform scaling processing and downsampling processing on the first model point cloud and the first scene point cloud to obtain a model point cloud and a scene point cloud; The normal vector calculation module is used to calculate the normal vectors of the model point cloud and the scene point cloud respectively to obtain the model point cloud normal vector and the scene point cloud normal vector; wherein the scene point cloud normal vector is generated by processing any one of the point cloud data processing methods according to claims 1 to 4; The pose estimation module is used to calculate and determine the pose estimation result of the workpiece to be grasped based on the model point cloud, the model point cloud normal vector and the scene point cloud, the scene point cloud normal vector.

Images