A method for estimating the pose of stacked workpieces based on the combination of 2D images and 3D point clouds.

Abstract

This invention discloses a method for estimating the pose of stacked workpieces based on the combination of 2D images and 3D point clouds. Using depth images as the processing object, the method performs 2D image processing, including background separation, hole filling, noise removal, and connected component analysis, to remove useless background information. The 2D pixel coordinates of each stacked workpiece are extracted, and point cloud data for each workpiece is generated using mapping relationships. The quality of the generated data is improved through downsampling, filtering, and surface smoothing. The model point cloud is obtained by multi-angle stitching. Key points are extracted from both the model and workpiece point clouds to establish FPFH feature descriptions. An improved Kuhn-Munkres algorithm is used to pair the feature points of both, and clustering is performed based on geometric constraints. The precise pose of the workpiece is obtained using SVD and ICP nearest point iteration. This method solves the problems of poor accuracy in 2D image detection and slow 3D point cloud registration, enabling stacked workpiece recognition to achieve good accuracy and real-time performance without requiring a large amount of training data.

Description

Technical Field

[0001] This invention relates to the fields of machine vision technology and industrial production, specifically to a method for estimating the pose of stacked workpieces based on a combination of two-dimensional images and three-dimensional point clouds. Background Technology

[0002] With the advent of Industry 4.0, industrial robotic arms have been widely used in various fields, such as milling, box stacking, metal cutting, and glass conveying. They have liberated human labor, alleviated the problems of labor shortage and increased labor costs caused by the aging population, and brought great convenience to production and life.

[0003] Workpiece recognition and grasping is a crucial application area in industrial robotic arm production. Industrial cameras capture image data, which is then processed to identify the workpiece's type, pose, and other information, allowing the robotic arm to grasp the target. However, the working environment for robotic arm grasping is becoming increasingly complex, often involving unstructured scenarios with problems such as densely packed targets, complex and variable postures, stacking, and occlusion. This poses significant challenges to precise grasping by the robotic arm. Currently, to address the problem of stacked and occluded workpieces, industrial production often uses vibration tables or manual placement to disperse the workpieces and reduce the difficulty of recognition and grasping. However, this method incurs additional equipment and manpower costs. Therefore, a method directly targeting the recognition of stacked workpieces is essential.

[0004] Currently, methods for recognizing stacked workpieces mainly fall into three categories: 2D image detection, 3D point cloud registration, and deep learning. Each method has its own advantages and disadvantages. 2D image detection primarily targets planar workpieces, is a mature technology, and offers fast recognition speed, but it lacks height information, limiting its grasping accuracy. 3D point cloud registration offers high recognition accuracy, but it requires a large amount of data, resulting in slow processing speed and poor real-time performance. Deep learning offers superior recognition efficiency and accuracy compared to the other two methods, but it requires extensive training with large amounts of data to ensure reliable recognition. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a method for estimating the pose of stacked workpieces based on the combination of two-dimensional images and three-dimensional point clouds. This method solves the problems of poor detection accuracy of two-dimensional images and slow registration speed of three-dimensional point clouds, enabling stacked workpiece recognition to have good accuracy and real-time performance without requiring a large amount of training data.

[0006] The present invention achieves the above-mentioned technical objectives through the following technical means.

[0007] A method for estimating the pose of stacked workpieces based on the combination of 2D images and 3D point clouds is proposed. Using depth images as the processing object, 2D image processing is performed, including background separation, hole filling, noise point removal, and connected component analysis, to remove useless background information and extract the 2D pixel coordinates of each stacked workpiece. Point cloud data for each workpiece is generated using mapping relationships, and the quality of the generated data is improved through downsampling, filtering, and surface smoothing. The model point cloud is obtained by multi-angle stitching. Key points are extracted from both the model point cloud and the workpiece point cloud to establish FPFH feature descriptions. An improved Kuhn-Munkres algorithm is used to pair the feature points of both, and clustering is performed based on geometric constraints. The precise pose of each workpiece is obtained using SVD and ICP nearest point iteration.

[0008] The above scheme includes two-dimensional image segmentation of the workpiece, three-dimensional point cloud data generation, point cloud data preprocessing, model point cloud generation for template matching, workpiece point cloud feature extraction, workpiece point cloud feature matching, and workpiece point cloud pose estimation.

[0009] The workpiece 2D image segmentation uses the depth map acquired by the camera as the processing object. After background separation, hole filling, noise point removal, and connected component analysis, the region where each workpiece is located and the 2D pixel coordinates are extracted to achieve the segmentation of the workpiece 2D image.

[0010] The generation of the three-dimensional point cloud data is based on the mapping relationship between the pixel coordinates of the two-dimensional image and the three-dimensional spatial coordinates, realizing the conversion of two-dimensional image data into three-dimensional point cloud data;

[0011] The point cloud data preprocessing includes point cloud downsampling, point cloud filtering, and point cloud surface smoothing, which are used to handle defects in the generated point cloud data and improve the quality of the point cloud data.

[0012] The generation of the model point cloud is achieved by acquiring workpiece images from multiple angles using a camera, obtaining multi-angle point cloud data of the workpiece through two-dimensional image segmentation, three-dimensional point cloud data generation, and point cloud data preprocessing, and then stitching the collected angles together to generate a complete point cloud, which serves as the model point cloud for template matching.

[0013] The workpiece point cloud feature extraction involves extracting key points from the workpiece point cloud and establishing FPFH feature descriptions based on the distribution of data points around the key points, which serve as the basis for point cloud registration.

[0014] The workpiece point cloud feature matching uses an improved Kuhn-Munkres algorithm to achieve point cloud feature point pairing.

[0015] The workpiece point cloud pose estimation involves filtering and clustering point pairs obtained from feature matching based on geometric constraints, calculating the transformation matrix between clustered point pairs using SVD, and further employing ICP nearest point iteration to solve for the precise pose of the workpiece point cloud.

[0016] In the above scheme, the mapping relationship is a transformation matrix from two-dimensional pixel coordinates to three-dimensional spatial coordinates, and the values ​​of the matrix elements are the intrinsic parameters of the calibrated camera.

[0017] In the above scheme, the point cloud downsampling uses a voxel downsampling algorithm to divide the voxel space into units, and takes the average coordinates of each data point in the space unit as the sampling point coordinates to reduce the amount of point cloud data and reduce the point cloud density; the point cloud filtering uses a radius filtering algorithm to search for the number of nearest neighbor points within a specified radius and delete discrete points whose number is less than a set threshold; the point cloud surface smoothing uses the moving least squares (MLS) method to perform surface fitting on the local surface of the point cloud, repairs the coordinate values ​​of abrupt data points, smooths the undulations of the point cloud surface, and reduces the roughness of the data surface.

[0018] In the above scheme, the key point extraction adopts a uniform sampling algorithm to extract data points at fixed spatial intervals as feature points of the workpiece point cloud.

[0019] In the above scheme, the improved Kuhn-Munkres algorithm takes the K points that are closest to the distance norm of the key point feature description of the model point cloud and the point cloud to be identified as the matching point pairs, and uses the reciprocal of the distance norm to represent the weight of each matching point pair. The algorithm uses the principle of Kuhn-Munkres to solve for the optimal matching with the maximum weight sum of the weighted bipartite graph, thereby realizing point cloud feature matching.

[0020] In the above scheme, the camera is a three-dimensional camera used to acquire depth images of the workpiece.

[0021] In the above scheme, the background separation is achieved by determining a suitable threshold using a grayscale distribution histogram and then using threshold segmentation to remove useless background information from the identification scene; the hole filling is used to fill holes in the workpiece image where the pixel grayscale value is 0, and the average grayscale value of the pixels in the adjacent window of the hole pixel is used for filling; the noise point removal is achieved by using morphological opening operations to delete some abnormal grayscale pixel points that cannot be removed by background separation; and the connected component analysis is used to find and mark mutually independent regions in the image, and then extract the regions where different workpieces are located and their two-dimensional pixel coordinates based on the markings.

[0022] The above solution specifically includes the following steps:

[0023] The first step is 3D camera calibration. The camera is calibrated according to the calibration method recommended by the camera manufacturer, correcting radial and tangential distortions to obtain the camera's intrinsic and extrinsic parameters. The intrinsic parameters {f} are then used as the basis for this calibration. x ,f y Generate a transformation matrix from two-dimensional pixel coordinates to three-dimensional spatial coordinates using the formulas ,cx,cy}.

[0024] The second step is model point cloud generation. A 3D camera is used to collect image data of the workpiece from various angles. The area where the workpiece is located and its two-dimensional pixel coordinates are extracted from the collected images through background separation, hole filling, noise point removal, and connected component analysis. Point cloud data is generated by using the mapping relationship between pixel coordinates and spatial coordinates. The point cloud data is then subjected to voxel downsampling, radius filtering, and MLS smoothing to reduce the amount of point cloud data, remove the remaining discrete points, and smooth the data surface. Finally, the complete point cloud is generated by stitching together the collected angles to obtain the template-matched model point cloud.

[0025] The third step is to extract features from the model point cloud. The key points of the model point cloud are extracted by uniform sampling. An FPFH feature description is established for each key point in the point cloud as template data for feature matching.

[0026] The fourth step is to generate workpiece point clouds. A 3D camera is used to collect image data of the stacked workpiece scene. The workpiece area and its two-dimensional pixel coordinates are extracted from the scene image through background separation, hole filling, noise point removal, and connected component analysis. Point cloud data is generated by using the mapping relationship between pixel coordinates and spatial coordinates. The point cloud data is then subjected to voxel downsampling, radius filtering, and MLS smoothing to reduce the amount of point cloud data, remove the remaining discrete points, and smooth the data surface to obtain the point cloud of each workpiece in the recognition scene.

[0027] The fifth step is to extract the feature of the workpiece point cloud. The key points of each workpiece point cloud are extracted by uniform sampling, and an FPFH feature description is established for each key point in the point cloud.

[0028] Step 6: Feature matching of workpiece point cloud. Based on the FPFH feature description, the K feature points in the workpiece point cloud that are closest to the distance norm of the model point cloud feature points are selected as feature point pairs to be matched. The weight of each matching point pair is represented by the reciprocal of the distance norm. The optimal matching scheme is solved using the Kuhn-Munkres algorithm to generate feature matching point pairs between the model point cloud and the workpiece point cloud.

[0029] Step 7: Workpiece point cloud pose estimation. Based on geometric constraints, the matched feature point pairs are filtered and clustered. The transformation matrix between clustered point pairs is calculated using SVD. The nearest point iteration (ICP) is used to make the model point cloud match and overlap with the workpiece point cloud. The accurate pose of the workpiece is solved by the transformation matrix obtained through this process.

[0030] The present invention has the following beneficial effects:

[0031] This invention uses depth images as the primary processing method. Through a two-dimensional image processing workflow involving background separation, hole filling, noise removal, and connected component analysis, useless background information is removed, and the two-dimensional pixel coordinates of each stacked workpiece are extracted. Point cloud data for each workpiece is generated using mapping relationships, and the quality of the generated data is improved through downsampling, filtering, and surface smoothing. The model point cloud is obtained by multi-angle stitching. Key points are extracted from both the model and workpiece point clouds to establish FPFH feature descriptions. An improved Kuhn-Munkres algorithm is used to pair the feature points of both, and clustering is performed based on geometric constraints. The precise pose of the workpiece is obtained using SVD and ICP nearest point iteration. This method combines the advantages of two-dimensional image detection and three-dimensional point cloud registration, solving the problems of poor accuracy in two-dimensional image detection and slow speed in three-dimensional point cloud registration. It uses two-dimensional image processing methods to achieve rapid segmentation of stacked workpieces and improves the feature matching algorithm in three-dimensional point cloud registration, further enhancing registration accuracy. This method achieves stacked workpiece recognition with good accuracy and real-time performance without requiring a large amount of training data. Attached Figure Description

[0032] Figure 1 This is a flowchart of the stacked workpiece pose estimation method based on the combination of two-dimensional images and three-dimensional point clouds of the present invention;

[0033] Figure 2 A schematic diagram illustrating the process of segmenting a 2D image of a workpiece in a scene;

[0034] Figure 3 A schematic diagram of the preprocessing flow for workpiece point cloud data;

[0035] Figure 4 This is a schematic diagram illustrating the principle of the Kuhn-Munkres algorithm.

[0036] Figure 5 Flowchart for improving the Kuhn-Munkres algorithm. Detailed Implementation

[0037] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0038] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0039] Combined with appendix Figure 1 As shown, a method for estimating the pose of stacked workpieces based on the combination of 2D images and 3D point clouds is divided into offline and online parts. The offline part includes 2D image segmentation of the workpiece, 3D point cloud data generation, point cloud data preprocessing, model point cloud generation for template matching, workpiece point cloud feature extraction, workpiece point cloud feature matching, and workpiece point cloud pose estimation. In the offline part, the invention acquires workpiece images from multiple angles using a camera. Through 2D image segmentation, 3D point cloud data generation, and point cloud data preprocessing, multi-angle point cloud data of the workpiece is obtained. Model point clouds are generated by stitching together the acquired angles as template matching data. Key points are extracted from the model point clouds, and FPFH feature descriptions are established as the basis for point cloud registration. During the online workpiece recognition process, a 3D camera is used to acquire image data of the stacked workpiece scene. The workpiece point cloud is obtained by 2D image segmentation, 3D point cloud data generation and point cloud data preprocessing. Key points are extracted from the workpiece point cloud, and FPFH feature description is established. The optimal matching scheme is obtained through point cloud feature matching, and feature matching point pairs between the model point cloud and the workpiece point cloud are generated. The transformation matrix of the obtained matching point pairs can be solved to obtain the accurate pose of the workpiece point cloud.

[0040] Combined with appendix Figure 1 , 2 As shown, the two-dimensional image segmentation of the workpiece in the recognition scene uses the depth map acquired by the camera as the main processing element, including background separation, hole filling, noise removal, and connected component analysis. Through the above processes, the region where each workpiece is located and its two-dimensional pixel coordinates are extracted to achieve the segmentation of the two-dimensional image of the workpiece. The background separation determines an appropriate threshold using the gray-level distribution histogram of the depth map, and removes useless background information in the recognition scene through threshold segmentation. The hole filling uses the average gray-level value of pixels in the adjacent window of the hole pixel to fill the hole in the image. The noise removal uses morphological opening operations to delete pixels with abnormal gray-level values ​​that cannot be removed by background separation. The connected component analysis finds and marks mutually independent regions in the image, and extracts the region where different workpieces are located and their two-dimensional pixel coordinates based on the markings.

[0041] Combined with appendix Figure 1 , 2As shown, the 3D point cloud data is generated by transforming the pixel coordinates of each workpiece obtained from 2D image segmentation, and the values ​​of the transformation matrix elements are the intrinsic parameters {f} of the calibrated camera. x ,f y The matrix form of the transformation relationship between two-dimensional pixel coordinates and three-dimensional spatial coordinates is as follows:

[0042]

[0043] Where (X,Y,Z) are the 3D spatial coordinates of the point cloud, (u,v) are the pixel coordinates of the image, (cx,cy) are the coordinates of the center point of the image, and f x =f / S x f y =f / S y f is the focal length of the camera, S x and S y These are the scaling factors for the physical size of the image sensor, typically the reciprocal of the pixel's corresponding size value.

[0044] like Figure 1 , 3 As shown, the point cloud data preprocessing includes point cloud downsampling, point cloud filtering, and point cloud surface smoothing, which are used to handle defects in the generated point cloud data and improve the quality of the point cloud data. Point cloud downsampling uses a voxel downsampling algorithm to divide the voxel space unit and takes the average coordinates of each data point in the space unit as the sampling point coordinates, thereby reducing the amount of point cloud data and reducing the point cloud density. Point cloud filtering uses a radius filtering algorithm to search for the number of nearest neighbor points within a specified radius and delete discrete points whose number is less than a set threshold. Point cloud surface smoothing uses the moving least squares (MLS) method to perform surface fitting on the local surface of the point cloud, repairs the coordinate values ​​of abrupt data points, smooths the undulations of the point cloud surface, and reduces the roughness of the data surface.

[0045] Combined with appendix Figure 4 As shown, the Kuhn-Munkres algorithm is used to solve the matching problem in a bipartite graph. Each matching path in the graph has a corresponding weight value, and the weight value of a path that does not match is 0. The algorithm finds the matching scheme with the largest sum of weight values ​​by continuously backtracking.

[0046] Combined with appendix Figure 1 , 4As shown in Figure 5, the improved Kuhn-Munkres algorithm uses FPFH feature description as the basis for calculation. It selects the K feature points in the workpiece point cloud whose distance norm is closest to the feature points in the model point cloud as feature point pairs to be matched. The weight of each matching point pair is represented by the reciprocal of its distance norm. The Kuhn-Munkres algorithm principle is used to solve the optimal matching scheme that maximizes the sum of weights, thereby generating feature matching point pairs between the model point cloud and the workpiece point cloud. The algorithm takes the FPFH feature descriptions of the model and workpiece point clouds as input. The size of the weight matrix is ​​set according to the input size, where m is the size of the model point cloud feature description and n is the size of the workpiece point cloud feature description. The weight matrix is ​​initialized, and a distance threshold `dist` is set. Feature descriptions of each point in the workpiece point cloud are extracted iteratively. The K nearest points in the model point cloud feature description are found to form K paths to be matched, and the distance value `neigh_dists` for each pair is recorded. Paths to be matched are filtered based on the `neigh_dists` value; those less than the set threshold are considered Kuhn-Munkres matching paths. The reciprocal of the `neigh_dists` value is taken as the weight value of the matching path in the algorithm. The smaller the `neigh_dists` value, the larger the weight value, indicating a higher similarity between the two paths. The Kuhn-Munkres algorithm is executed based on the set matching paths and weight values ​​to find the optimal matching scheme that maximizes the sum of the weights.

[0047] A method for estimating the pose of stacked workpieces based on a combination of 2D images and 3D point clouds, specifically including the following steps:

[0048] The first step is 3D camera calibration. The camera is calibrated according to the calibration method recommended by the camera manufacturer to correct the radial and tangential distortion of the camera and obtain the intrinsic and extrinsic parameters of the camera. The transformation matrix from 2D pixel coordinates to 3D spatial coordinates is generated using the camera intrinsic parameters {fx,fy,cx,cy}.

[0049] The second step is model point cloud generation. A 3D camera is used to collect image data of the workpiece from various angles. After background separation, hole filling, noise point removal, and connected component analysis, the area where the workpiece is located and its two-dimensional pixel coordinates are extracted from the collected images. Point cloud data is generated by using the mapping relationship between pixel coordinates and spatial coordinates. The point cloud data is then subjected to voxel downsampling, radius filtering, and MLS smoothing to reduce the amount of point cloud data, remove remaining discrete points, and smooth the data surface. Finally, the complete point cloud is generated by stitching together the collected angles to obtain the template-matched model point cloud.

[0050] The third step is to extract features from the model point cloud. The key points of the model point cloud are extracted by uniform sampling. An FPFH feature description is established for each key point in the point cloud as template data for feature matching.

[0051] The fourth step is to generate workpiece point clouds. A 3D camera is used to collect image data of the stacked workpiece scene. After background separation, hole filling, noise point removal, and connected component analysis, the area where the workpiece is located and its two-dimensional pixel coordinates are extracted from the acquired image. Point cloud data is generated by using the mapping relationship between pixel coordinates and spatial coordinates. The point cloud data is then subjected to voxel downsampling, radius filtering, and MLS smoothing to reduce the amount of point cloud data, remove the remaining discrete points, and smooth the data surface to obtain the point cloud of each workpiece in the recognition scene.

[0052] The fifth step is to extract the feature of the workpiece point cloud. The key points of each workpiece point cloud are extracted by uniform sampling, and an FPFH feature description is established for each key point in each point cloud.

[0053] Step 6: Feature matching of workpiece point cloud. Based on the FPFH feature description, the K feature points of the model point cloud feature points that are closest to the feature description norm in the workpiece point cloud are selected as feature point pairs to be matched. The reciprocal of the feature description norm of the point pair is used to represent the weight of each matching point pair. The Kuhn-Munkres algorithm is used to solve the optimal matching scheme that maximizes the sum of weights.

[0054] Step 7: Workpiece point cloud pose estimation. Based on geometric constraints, the matched feature point pairs are filtered and clustered. The transformation matrix between clustered point pairs is calculated using SVD. The nearest point iteration (ICP) is used to make the model point cloud match and overlap with the workpiece point cloud. The accurate pose of the workpiece is solved by the transformation matrix obtained through this process.

[0055] This invention uses depth images as the primary processing method. Through a two-dimensional image processing workflow involving background separation, hole filling, noise removal, and connected component analysis, useless background information is removed, and the two-dimensional pixel coordinates of each stacked workpiece are extracted. Point cloud data for each workpiece is generated using mapping relationships, and the quality of the generated data is improved through downsampling, filtering, and surface smoothing. The model point cloud is obtained by multi-angle stitching. Key points are extracted from both the model and workpiece point clouds to establish FPFH feature descriptions. An improved Kuhn-Munkres algorithm is used to pair the feature points of both, and clustering is performed based on geometric constraints. The precise pose of the workpiece is obtained using SVD and ICP nearest point iteration. This method combines the advantages of two-dimensional image detection and three-dimensional point cloud registration, solving the problems of poor accuracy in two-dimensional image detection and slow speed in three-dimensional point cloud registration. It uses two-dimensional image processing methods to achieve rapid segmentation of stacked workpieces and improves the feature matching algorithm in three-dimensional point cloud registration, further enhancing registration accuracy. This method achieves stacked workpiece recognition with good accuracy and real-time performance without requiring a large amount of training data.

[0056] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0057] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention without departing from the principles and spirit of the present invention.

Claims

1. A stack workpiece pose estimation method based on combination of two-dimensional images and three-dimensional point clouds, characterized in that, The method comprises workpiece two-dimensional image segmentation, three-dimensional point cloud data generation, point cloud data preprocessing, model point cloud generation for template matching, workpiece point cloud feature extraction, workpiece point cloud feature matching and workpiece point cloud pose estimation. The workpiece two-dimensional image segmentation takes a depth map collected by a camera as a processing object, extracts a region where each workpiece is located and two-dimensional pixel coordinates after background separation, hole filling, noise point removal and connected domain analysis, and realizes segmentation of the workpiece two-dimensional image. The three-dimensional point cloud data generation realizes conversion of two-dimensional image data into three-dimensional point cloud data based on a mapping relationship between two-dimensional image pixel coordinates and three-dimensional space coordinates. The point cloud data preprocessing includes point cloud downsampling, point cloud filtering and point cloud surface smoothing, and is used for processing defects of the generated point cloud data and improving quality of the point cloud data. The model point cloud generation obtains workpiece multi-angle point cloud data through workpiece two-dimensional image segmentation, three-dimensional point cloud data generation and point cloud data preprocessing, and generates complete point cloud according to collection angles, as model point cloud for template matching. The workpiece point cloud feature extraction extracts key points of the workpiece point cloud, establishes an FPFH feature description according to a distribution of data points around the key points, and takes the feature description as a basis for point cloud registration. The workpiece point cloud feature matching realizes pairing of point cloud feature points by using an improved Kuhn-Munkres algorithm. The workpiece point cloud pose estimation filters and clusters point pairs obtained by feature matching according to geometric constraints, calculates a conversion matrix between clustered point pairs by using SVD, and further solves an accurate pose of the workpiece point cloud by using ICP nearest point iteration. The improved Kuhn-Munkres algorithm takes K points closest to a distance norm of a model point cloud and a key feature description of to-be-identified point cloud as a to-be-matched point pair, takes an inverse of the distance norm as a weight of each matched point pair, solves an optimal matching with maximum weight of a weighted bipartite graph by using a principle of the Kuhn-Munkres algorithm, and realizes point cloud feature matching.

2. The method of claim 1, wherein, The mapping relationship is a conversion matrix of two-dimensional pixel coordinates to three-dimensional space coordinates, and numerical values of matrix elements are internal parameters of the calibrated camera.

3. The method of claim 1, wherein, The point cloud downsampling divides a voxel space unit by using a voxel downsampling algorithm, takes average coordinates of data points in the space unit as a sampling point coordinate, reduces a point cloud data amount and reduces point cloud density; the point cloud filtering deletes discrete points with a number less than a set threshold by using a radius filtering algorithm to search for a number of neighbor points within a specified radius; and the point cloud surface smoothing performs surface fitting on a local surface of the point cloud by using a moving least square (MLS) method, repairs coordinate values of jumping data points, smooths ups and downs of the point cloud surface, and reduces roughness of a data surface.

4. The method of claim 1, wherein, The key point extraction extracts data points as feature points of the workpiece point cloud by using a uniform sampling algorithm to extract data points at fixed space distances.

5. The method of claim 1, wherein, The camera is a three-dimensional camera, and is used for collecting a depth image of the workpiece.

6. The method of claim 1, wherein, The background separation determines a suitable threshold value by using a grayscale distribution histogram, and removes useless background information in an identification scene by using a threshold segmentation method. The hole filling is used for filling the holes with pixel gray value of 0 in the workpiece image, and the average gray value of the pixels in the adjacent window of the hole pixel is used for filling; the noise point removal is used for deleting the part of the abnormal pixel points with gray value which cannot be removed by background separation; the connected domain analysis is used for finding out the regions independent of each other in the image and marking them out, and the region where the different workpieces are located and the two-dimensional pixel coordinates are extracted according to the marked search.

7. The method of claim 1, wherein, Specifically includes the following steps: The first step is 3D camera calibration. The camera is calibrated according to the calibration method recommended by the camera manufacturer, correcting radial and tangential distortions to obtain the camera's intrinsic and extrinsic parameters. The intrinsic parameters of the camera are then used as the basis for this calibration. , ,cx, cy} generates a transformation matrix from two-dimensional pixel coordinates to three-dimensional spatial coordinates; Second step, model point cloud generation, using a three-dimensional camera to collect image data of the workpiece at each angle, through background separation, hole filling, noise point removal, connected domain analysis, the workpiece region and its two-dimensional pixel coordinates are segmented and extracted from the collected image, the pixel coordinates and spatial coordinates mapping relationship is used to generate point cloud data, the point cloud data is down-sampled by voxel, radius filtering and MLS smoothing, the point cloud data amount is reduced, the remaining discrete points are removed, the data surface is smoothed, finally the complete point cloud is generated by splicing according to the collection angle, and the model point cloud for template matching is obtained; Third step, model point cloud feature extraction, the key points of the model point cloud are extracted by uniform sampling, and the FPFH feature description is established for each key point in the point cloud as the template data for feature matching; Fourth step, workpiece point cloud generation, using a three-dimensional camera to collect image data of the stacked workpiece scene, through background separation, hole filling, noise point removal, connected domain analysis, the workpiece region and its two-dimensional pixel coordinates are segmented and extracted from the scene image, the pixel coordinates and spatial coordinates mapping relationship is used to generate point cloud data, the point cloud data is down-sampled by voxel, radius filtering and MLS smoothing, the point cloud data amount is reduced, the remaining discrete points are removed, the data surface is smoothed, and the point cloud of each workpiece in the recognition scene is obtained; Fifth step, workpiece point cloud feature extraction, the key points are extracted from each workpiece point cloud by uniform sampling, and the FPFH feature description is established for each key point in the point cloud; Sixth step, workpiece point cloud feature matching, taking the FPFH feature description as the calculation basis, the K feature points closest to the model point cloud feature points in the workpiece point cloud are calculated, as the feature point pairs to be matched, the weight of each matched point pair is represented by the reciprocal of the distance norm, and the optimal matching scheme is solved by using the Kuhn-Munkres algorithm principle, and the feature matching point pairs between the model point cloud and the workpiece point cloud are generated; Seventh step, workpiece point cloud pose estimation, the feature point pairs obtained by matching are screened and clustered according to the geometric constraint relationship, the conversion matrix between the clustered point pairs is calculated by SVD, the ICP nearest point iteration is adopted, the model point cloud and the workpiece point cloud are matched and coincided, and the conversion matrix obtained through this process is used to solve the accurate pose of the workpiece.

Images