Part 6D attitude estimation method and device based on PCA and CAD models

Abstract

The invention discloses a part 6D attitude estimation method and device based on PCA (Principal Component Analysis) and CAD (Computer Aided Design) models. The method comprises the following steps: firstly, acquiring a three-dimensional point cloud of an original part in any posture on a mechanical arm grabbing workbench through a three-dimensional depth camera, processing the three-dimensional point cloud of the original part, acquiring a point cloud containing a part to be subjected to posture estimation, and preprocessing to obtain a point cloud only containing the part; reading a part CAD model point cloud and establishing a local coordinate system by using a PCA algorithm; and finally, respectively calculating contour features and FPFH features of to-be-registered point clouds, performing coarse registration according to the similarity of the extracted different point cloud set feature points and feature descriptors, performing fine registration by using an ICP algorithm, and obtaining a transformation matrix to realize part attitude estimation. According to the method, the three-dimensional depth camera is used for obtaining the part point cloud on the mechanical arm grabbing workbench so as to estimate the posture of the part, and the posture estimation precision and efficiency are improved.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a method and device for estimating a 6D posture of a part based on PCA and a CAD model. Background Art

[0002] Object 6D pose estimation is an important task in the field of 3D vision, which aims to obtain the pose and size of an object in 3D space through 3D depth images. Obtaining the 6-DOF pose of an object is the basis for realizing other operations, such as robot grasping. Compared with the pose estimation method based on real images, the method based on CAD images is more suitable for industrial products, and it is easy to obtain CAD from industrial products. The virtual point cloud generated by the CAD model is more accurate as a template than the real point cloud, and its rendering process is not affected by lighting or blur. Summary of the invention

[0003] Based on this, an embodiment of the present application provides a method and device for 6D posture estimation of parts based on PCA and CAD models, which obtains the part point cloud on the workbench of the robotic arm through a three-dimensional depth camera to realize the posture estimation of the part, solves the problems existing in the prior art, and improves the accuracy and efficiency of posture estimation.

[0004] In a first aspect, a method for estimating a 6D pose of a part based on PCA and a CAD model is provided, the method comprising:

[0005] The 3D point cloud of the original part in any posture on the workbench of the robot arm is obtained through the 3D depth camera, and the 3D point cloud of the original part is processed to obtain the point cloud of the part to be estimated in posture;

[0006] Preprocessing the part point cloud to obtain a point cloud containing only the parts;

[0007] Read the point cloud of the part CAD model and use the PCA algorithm to establish the local coordinate system;

[0008] The contour features and FPFH features of the point cloud to be registered are calculated respectively, and coarse registration is performed according to the similarity of the feature points and feature descriptors extracted from different point cloud sets. The ICP algorithm is used for fine registration, and the transformation matrix is ​​obtained to realize part posture estimation.

[0009] Optionally, the original part 3D point cloud is processed, including:

[0010] Filter the original part's 3D point cloud to remove noise points and outliers in the point cloud; segment the filtered point cloud, remove the workbench point cloud in the filtered point cloud, and obtain a 3D point cloud containing only the parts;

[0011] Among them, the original part three-dimensional point cloud is conditionally filtered, and the distance between the points in the original part three-dimensional point cloud is compared with the preset distance threshold L. If the distance between the points in the original point cloud is greater than this threshold, the point is filtered and removed; the point cloud after conditional filtering is radius filtered, and a sphere is generated with each point in the point cloud obtained by filtering as the center and the preset radius value R as the radius. The number of points in the sphere is calculated, and if the number is less than the number threshold M, the point is removed.

[0012] Optionally, the filtered point cloud is segmented, specifically including:

[0013] The plane model of the workbench is obtained, and the distance between each point in the filtered point cloud and the plane is calculated. If the distance is less than the preset distance threshold ε1, the point is a point in the plane and is segmented and removed. The iteration is stopped when the set maximum value is reached or the total number of point clouds remains unchanged, and a point cloud containing only parts is obtained.

[0014] Optionally, read the part CAD model point cloud and use the PCA algorithm to establish a local coordinate system, including:

[0015] First, the decentralized covariance matrix of the CAD model point cloud is obtained, the three eigenvalues ​​of the covariance matrix from large to small are calculated, and the corresponding eigenvectors are calculated. The three eigenvectors are used as the x-axis, y-axis and z-axis to establish a local coordinate system, and the local coordinate system is transformed into the camera coordinate system.

[0016] Among them, reading the part CAD model point cloud and establishing the local coordinate system using the PCA algorithm includes decentralizing the CAD model point cloud, obtaining the covariance matrix, calculating the eigenvalues ​​of the covariance matrix, arranging them from large to small, and respectively obtaining the eigenvectors corresponding to the first three eigenvalues. The eigenvectors are mutually orthogonal, and the local coordinate system is established with the obtained three eigenvectors as the x-axis, y-axis and z-axis respectively.

[0017] Optionally, contour features and FPFH features of the point cloud to be registered are calculated respectively, and rough registration is performed according to the similarity between the extracted feature points and feature descriptors of different point cloud sets, including:

[0018] First, the contour features and FPFH features of the point cloud are extracted. Based on the contour features and FPFH features, the CAD point cloud is used as the source point cloud and the processed part point cloud is used as the target point cloud for rough registration. Based on the rough registration, the ICP algorithm is used for fine registration. After the two point clouds overlap, the position and posture of the part can be obtained.

[0019] Any point in the point cloud is projected onto its tangent plane from its neighborhood, and the point is connected with the projected point to obtain a series of angles with the point as the corner point. If the maximum angle is greater than the set angle threshold ξ, the point is a boundary contour point.

[0020] The contour features and FPFH features of the two point clouds to be tested are extracted, and the point pairs to be matched are screened out according to the similarity between the extracted feature points and feature descriptors of different point cloud sets. The amount of registration data calculation is reduced by screening the point pairs to be matched. The point clouds are roughly registered by combining the two features, and the CAD point cloud and the part point cloud partially overlap.

[0021] Optionally, an ICP algorithm is used for fine registration, including:

[0022] Each point in the CAD point cloud searches for the nearest point in the part point cloud as a matching point pair, calculates the direction vector of the matching point pair, and compares it with the preset direction vector threshold. If the direction vector threshold is met, it is a correctly matched point pair. When the distance error between the registered point cloud and the part point cloud is less than the distance threshold ε2 or the maximum number of iterations is reached, the iteration is stopped to complete the point cloud precise registration. The transformation matrices of the coarse and fine registration are calculated, and the global transformation matrix is ​​obtained from the obtained transformation matrix as the part pose.

[0023] In a second aspect, a device for estimating a 6D pose of a part based on PCA and a CAD model is provided, the device comprising:

[0024] An acquisition module is used to acquire a 3D point cloud of an original part in any posture on a robotic arm grabbing workbench through a 3D depth camera, process the 3D point cloud of the original part, and acquire a point cloud containing a part to be estimated in posture;

[0025] A point cloud preprocessing module, used for preprocessing the part point cloud to obtain a point cloud containing only the parts;

[0026] The local coordinate system establishment module is used to read the point cloud of the part CAD model and establish the local coordinate system using the PCA algorithm;

[0027] The point cloud registration module is used to calculate the contour features and FPFH features of the point cloud to be registered, perform coarse registration based on the similarity of the extracted feature points and feature descriptors of different point cloud sets, use the ICP algorithm for fine registration, and obtain the transformation matrix to realize part posture estimation.

[0028] In a third aspect, an electronic device is provided, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the method for estimating a part 6D posture as described in any one of the first aspects is implemented.

[0029] In a fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the method for estimating a part 6D posture as described in any one of the first aspects is implemented.

[0030] In a fifth aspect, a computer program product is provided, comprising a computer program / instruction, which, when executed by a processor, implements the part 6D posture estimation method described in any one of the first aspects above.

[0031] The beneficial effects brought about by the technical solution provided in the embodiment of the present application include at least accurately capturing the three-dimensional point cloud of the part through a three-dimensional depth camera, and using the PCA algorithm combined with the ICP algorithm for coarse and fine alignment, which significantly improves the accuracy of the part's 6D posture estimation and ensures that the robotic arm can accurately grasp and place the part. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the implementation methods of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for the implementation methods or the description of the prior art. Obviously, the drawings in the following description are only exemplary, and for ordinary technicians in this field, other implementation drawings can be derived from the provided drawings without creative work.

[0033] Figure 1 A flowchart of a method for estimating a 6D pose of a part based on PCA and CAD model provided in an embodiment of the present application;

[0034] Figure 2 A block diagram of a device for estimating 6D posture of a part based on PCA and CAD model provided in an embodiment of the present application;

[0035] Figure 3 A schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0036] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0037] In the description of the present invention, the terms "comprises", "has" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or apparatus comprising a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may also include other steps or units that are not explicitly listed but are inherent to these processes, methods, products or apparatuses, or steps or units added based on further optimization schemes conceived by the present invention.

[0038] To facilitate understanding of this embodiment, a method for estimating a 6D pose of a part based on PCA and a CAD model disclosed in an embodiment of the present application is first introduced in detail.

[0039] Please refer to Figure 1 , which shows a flow chart of a method for estimating a 6D pose of a part based on PCA and a CAD model provided in an embodiment of the present application. The method may include the following steps:

[0040] S1, obtain the three-dimensional point cloud of the original part in any posture on the robot arm grasping workbench through the three-dimensional depth camera, process the original three-dimensional point cloud of the part, and obtain the point cloud containing the part to be estimated.

[0041] This step includes point cloud acquisition. The point cloud acquisition device is a three-dimensional depth camera. The acquired object is the parts on the workbench. The point cloud of the parts is obtained through the three-dimensional depth camera as the original point cloud.

[0042] S2, preprocessing the part point cloud to obtain a point cloud containing only the parts.

[0043] This step includes preprocessing. In addition to the part point cloud, the original point cloud also contains noise points, outliers and workbench point clouds. These irrelevant point clouds will interfere with the part posture estimation. In order to ensure the accuracy of the part posture estimation, preprocessing is required to remove noise points, outliers and workbench point clouds. Specifically, it includes:

[0044] Point cloud filtering:

[0045] The distance between the outlier and the main body of the part point cloud is much greater than the distance between the main bodies of the part point cloud. The distance threshold L is set by the conditional filtering algorithm, and the points in the original point cloud with a distance greater than the threshold are filtered out, that is, the outlier points are filtered out. Noise points are sparser and less dense than part point clouds. Noise points can be removed by radius filtering. The radius filter first sets a radius value R, takes each point in the point cloud as the center, and R as the radius to generate a virtual sphere. The number of points in the sphere is calculated. When the number is greater than the number threshold M, the point is retained. The noise point is filtered out because of its small density and does not meet the number threshold. At this time, the point cloud is downsampled voxel filtering, the voxel cube size is set, the entire point cloud space is divided by voxels, the center of mass of all points in each voxel is calculated, and the center of mass replaces all the original points in the voxel to obtain a new point cloud. Voxel filtering simplifies point cloud data, improves the calculation speed of subsequent point cloud segmentation and point cloud registration, and reduces time cost.

[0046] Point cloud segmentation:

[0047] After point cloud filtering, the point cloud data only contains part point cloud and workbench point cloud, and the noise points and outliers in the point cloud are eliminated. The geometric shape of the workbench is a plane, and its spatial model is defined as ax+by+cz+d=0. The RANSAC algorithm is used to extract the part point cloud.

[0048] Aα=0 (1)

[0049] In the formula and They are all plane parameters of the workbench.

[0050] The RANSAC algorithm is divided into four steps:

[0051] (1) Data initialization, let n max =0,a0=0;

[0052] (2) Select three non-collinear points A j =(x j ,y j ,z j ), j = 1, 2, 3. Find the plane model, the normal vector of the plane is α';

[0053] (3) Every point n in the point cloud space i =(x i ,y i ,z i ) T , where i = 1, 2, 3…m, from this we can get its secondary vector matrix is ​​P = (x i ,y i ,z i ,1) T , the distance from the point to the workbench plane is P*α';

[0054] (4) Repeat the iterative calculation steps (2) and (3) k times. When n is obtained max The algorithm stops when the corresponding plane α has been found. If the distance is less than a certain threshold, that is, P*α'<ε1, the point is a point in the plane, and the point greater than the distance threshold belongs to the part point cloud.

[0055] Distance threshold ε1 and maximum number of point clouds in the region n max It needs to be obtained through experiments. The maximum number of iterations k can be estimated based on theoretical results and finally corrected through experiments.

[0056] Assume that the probability that each point in the filtered point cloud belongs to the part point cloud is w, where w is the number of points that actually belong to the part point cloud after the experiment / the total number of filtered point clouds, and each point is independently distributed. Assuming that the number of points required is n, the probability that all n points belong to the part point cloud is w n , the probability that at least one point cloud does not belong to a part is (1-w n ), after k iterations, the probability that all selected points do not belong to the part point cloud is (1-w n ) k Assuming that the probability that all points selected after k iterations belong to the part point cloud is p, we can get:

[0057] 1-p=(1-w n ) k (2)

[0058] Taking logarithms respectively, we get:

[0059]

[0060] From formula 3, it can be concluded that when the number of points n in the plane remains unchanged, increasing the number of iterations k increases the probability p that all points selected after iteration belong to the part point cloud. When the probability w of each point belonging to the part point cloud decreases, p increases. It can be concluded that to increase the segmentation success rate, the proportion of part point clouds in the data point cloud can be appropriately reduced by increasing the number of iterations k, that is, adjusting the field of view and angle of the 3D camera when acquiring the point cloud.

[0061] S3, read the part CAD model point cloud and use the PCA algorithm to establish the local coordinate system.

[0062] In this step, the local coordinate system of the CAD point cloud is established. When the 3D camera acquires the part point cloud, the original point cloud it acquires is incomplete and lacks key information for posture estimation due to factors such as viewing angle and part shape. By aligning the CAD model with the part point cloud acquired by the 3D camera, the CAD point cloud and the part point cloud information can be fused. The CAD point cloud can make up for the lack of part point cloud information. PCA analysis is performed on the CAD point cloud to determine the eigenvectors of the CAD point cloud, and the local coordinate system of the CAD point cloud is established using the three eigenvectors as the main axes of the coordinates.

[0063] There are m points in the point cloud, and the point cloud is N = [n1, n2, ..., n i ] means, i=1,…,m,n i Represents the coordinate column vector of the i-th point.

[0064] (1) Decentralize the point cloud, in

[0065] (2) Calculate the covariance matrix, Perform eigenvalue decomposition, arrange according to the size of the eigenvalue, and calculate the eigenvectors P1, P2, and P3 respectively.

[0066] (3) Establish a local coordinate system with P1, P2, and P3 as the xyz axes and the centroid of the CAD point cloud as the coordinate origin.

[0067] (4) Transform the local coordinate system so that it coincides with the camera coordinate system.

[0068] Through the above steps, the PCA algorithm is used to complete the establishment of the local coordinate system of the CAD point cloud. The local coordinate system is established on the CAD model point cloud with high accuracy. The CAD models of the same parts are consistent, so no matter from which angle the 3D camera obtains the part point cloud, the local coordinate system of the CAD point cloud of the same part remains unchanged.

[0069] S4, respectively calculate the contour features and FPFH features of the point cloud to be registered, perform coarse registration based on the similarity of the extracted feature points and feature descriptors of different point cloud sets, use the ICP algorithm for fine registration, and obtain the transformation matrix to realize part posture estimation.

[0070] In this step, point cloud registration is realized, with CAD point cloud as the source point cloud and point cloud captured by 3D camera as the target point cloud. By extracting the contour features and FPFH features of the point cloud, feature matching and coarse registration are performed, and finally the ICP algorithm is used to complete the precise registration between the two point clouds to obtain the part posture.

[0071] The steps to extract contour features are as follows:

[0072] (1) For any point n in the point cloud i =(x i ,y i ,z i ) T , where i = 1, 2, 3…m, for its neighborhood N i Project onto its tangent plane;

[0073] (2) The neighboring points after projection are compared with n i Connected to form n i is a series of angles of the corner points, sorted counterclockwise to obtain θ = [θ1, θ2, ..., θ n ],n=||N i ||-1;

[0074] (3) Determine the maximum angle in θ, i.e., θ max =max(θ), with θ max As the basis for judgment, if θ max >ξ, then n i is the edge contour point.

[0075] After extracting the contour features of the part point cloud and the CAD point cloud, the FPFH features of the two point clouds are calculated. According to the similarity of the extracted feature points and feature descriptors of different point cloud sets, the point pairs to be matched are screened out, and the point clouds are roughly aligned by combining the two features. Based on the good rough alignment, the ICP algorithm is used for fine alignment to finally obtain the transformation matrix.

[0076] The steps of ICP fine registration are as follows:

[0077] (1) The CAD point cloud data P′ obtained after rough registration is i |i=1,2,3…n},part point cloud dataset Q={q i |i=1,2,3…n}, each point p in P' i , find the closest point q in Q i As a matching point pair;

[0078] (2) The obtained matching point pairs are screened for correctly matched point pairs by calculating the direction vector threshold to improve the accuracy of registration;

[0079] (3) Calculate the rotation matrix and translation matrix so that the mean square error between the point cloud set after rough registration and the target point cloud is minimized. The mean square error can be calculated as follows:

[0080]

[0081] (4) Obtain the required distance threshold ε2 and maximum number of iterations N through experiments max ,After iterative transformation, we get P", and calculate the distance error between P" and the target point cloud Q. If the distance threshold ε2 is met or the number of iterations reaches N max Then stop iterating.

[0082] Assuming that the coordinates of a pair of matching points in the scene target point cloud and the object source point cloud are P = (x1, y1, z1) and P' = (x1', y1', z1'), then:

[0083]

[0084] P=RP'+t (6)

[0085] Where R is a 3×3 rotation matrix and t is a 3×1 translation matrix. After homogenizing the above formula, we can get:

[0086]

[0087] Where 0 T is a 1×3 matrix, and T is a 4×4 matrix between matching point pairs.

[0088] According to the transformation of the Euler angles α, β, and γ between the two matching points on the xyz axis of the coordinate system, we can get:

[0089]

[0090] The solution of Equation 5 requires three non-collinear points, which can be substituted into the formula to obtain the translation and rotation matrices.

[0091] An optional embodiment of a method is given below:

[0092] An embodiment of the present application provides a method for part posture estimation, the method comprising: obtaining a three-dimensional point cloud of an original part in any posture on a robotic arm grasping workbench through a three-dimensional depth camera, processing the original point cloud to obtain a point cloud containing only the part, reading the part CAD model point cloud and using the PCA algorithm to establish a local coordinate system, extracting features of two point clouds to be aligned, using the CAD point cloud as the source point cloud and the processed part point cloud as the target point cloud for alignment, and obtaining the position and posture of the part.

[0093] In the above scheme, the original point cloud is processed. First, the three-dimensional point cloud of the original part is filtered to remove noise points and outliers in the point cloud, which can prevent outliers from having a negative impact on subsequent processing. Secondly, the filtered point cloud is segmented, and the workbench point cloud in the filtered point cloud is removed to obtain a three-dimensional point cloud containing only the parts.

[0094] In the above scheme, the part CAD model point cloud is read and the local coordinate system is established using the PCA algorithm. First, the decentralized covariance matrix of the CAD model point cloud is obtained, the three eigenvalues ​​of the covariance matrix from large to small are calculated, and the corresponding eigenvectors are calculated. The three eigenvectors obtained are used as the x-axis, y-axis and z-axis to establish a local coordinate system, and the local coordinate system is transformed to the camera coordinate system.

[0095] In the above scheme, the features of the two point clouds to be registered are extracted. First, the contour features and FPFH features of the point clouds are extracted. On the basis of the contour features and FPFH features, the CAD point cloud is used as the source point cloud and the processed part point cloud is used as the target point cloud for coarse registration. Based on the coarse registration, the ICP algorithm is used for fine registration. After the two point clouds overlap, the position and posture of the part can be obtained.

[0096] This scheme uses the PCA algorithm to establish a local coordinate system on the part CAD model point cloud, takes the CAD point cloud as the source point cloud and the processed part point cloud as the target point cloud for alignment to obtain the part's pose. The local coordinate system established by this scheme is not affected by the shooting angle of the three-dimensional depth camera. By transforming the local coordinate system to the camera coordinate system, the alignment result with the CAD point cloud as the source point cloud and the processed part point cloud as the target point cloud is the part's pose.

[0097] In one possible implementation, the original point cloud is filtered, including: conditional filtering of the original point cloud to filter out outliers in the original point cloud, radius filtering of the conditionally filtered point cloud to filter out noise points in the original point cloud, and downsampling voxel filtering of the conditionally filtered point cloud to simplify point cloud data and reduce point cloud density.

[0098] In the above scheme, the original point cloud is conditionally filtered, and the distance between the points in the original point cloud is compared with the preset distance threshold L. If the distance between the points in the original point cloud is greater than this threshold, the point is filtered and removed; the point cloud after conditional filtering is radius filtered, and a sphere is generated with each point in the point cloud obtained by filtering as the center and the preset radius value R as the radius. The number of points in the sphere is calculated, and if the number is less than the number threshold M, the point is removed.

[0099] In one possible implementation, the point cloud obtained by filtering is segmented, including: finding a workbench plane model, calculating the distance between each point in the filtered point cloud and the plane, if the distance is less than a preset distance threshold ε1, the point is a point in the plane and is segmented and removed, and the iteration is stopped when the iteration reaches a set maximum value or the total number of point clouds remains unchanged, to obtain a point cloud containing only parts.

[0100] In one possible implementation, the part CAD model point cloud is read and a local coordinate system is established using a PCA algorithm, including: decentralizing the CAD model point cloud, obtaining a covariance matrix, calculating the eigenvalues ​​of the covariance matrix, arranging them from large to small, and respectively obtaining the eigenvectors corresponding to the first three eigenvalues, wherein the eigenvectors are mutually orthogonal, and a local coordinate system is established using the three eigenvectors obtained as the x-axis, y-axis, and z-axis.

[0101] In the above scheme, the local coordinate system established based on the CAD model point cloud and the local coordinate system established by the PCA algorithm are related to the distribution of the CAD model point cloud. The projection variance of the point cloud on the x-axis of the established local coordinate system is the largest, the projection variance on the y-axis is the second largest, and the projection variance on the z-axis is the smallest. The CAD model point clouds of the same parts are the same, that is, the distribution of the CAD point clouds is the same. Therefore, the established local coordinate system is not affected by the shooting angle of the three-dimensional depth camera.

[0102] In a possible implementation, the local coordinate system of the established CAD model point cloud is transformed into the camera coordinate system, and the origin, x-axis, y-axis and z-axis of the local coordinate system coincide with the origin, x-axis, y-axis and z-axis of the camera coordinate system respectively. The registration transformation matrix obtained by taking the CAD point cloud as the source point cloud and the preprocessed part point cloud as the target point cloud is the pose of the part. Through this coordinate transformation, the calculation of the part pose is simplified and the efficiency of part pose estimation is improved.

[0103] In a possible implementation, contour features of two point clouds to be tested are extracted, including: projecting any point in the point cloud onto its tangent plane from its neighborhood, connecting the point with the projected point, and obtaining a series of angles with the point as the corner point. If the maximum value of the angle is greater than a set angle threshold ξ, the point is a boundary contour point.

[0104] In one possible implementation, the contour features and FPFH features of the two point clouds to be tested are extracted, and the point pairs to be matched are screened out based on the similarity between the extracted feature points and feature descriptors of different point cloud sets. The amount of registration data calculation is reduced by screening the point pairs to be matched, and the point clouds are roughly registered by combining the two features, so that the CAD point cloud and the part point cloud partially overlap.

[0105] In a possible implementation, the ICP algorithm is used to perform fine registration of the point cloud based on the rough registration, including: each point of the CAD point cloud searches for the nearest point in the part point cloud as a matching point pair, calculates the direction vector of the matching point pair, and compares it with a preset direction vector threshold. If the direction vector threshold is met, it is a correctly matched point pair, thereby improving the accuracy of the registration. When the distance error between the registered point cloud and the part point cloud is less than the distance threshold ε2 or the maximum number of iterations is reached, the iteration is stopped to complete the fine registration of the point cloud.

[0106] In a possible implementation, the transformation matrices of the coarse registration and the fine registration are calculated, and the global transformation matrix can be obtained from the obtained transformation matrix, and this matrix is ​​the part pose.

[0107] Please refer to Figure 2 , which shows a block diagram of a part 6D posture estimation device based on PCA and CAD model provided in an embodiment of the present application.

[0108] An acquisition module is used to acquire a 3D point cloud of an original part in any posture on a robotic arm grabbing workbench through a 3D depth camera, process the 3D point cloud of the original part, and acquire a point cloud containing a part to be estimated in posture;

[0109] Point cloud preprocessing module, used to preprocess the part point cloud to obtain a point cloud containing only the parts;

[0110] The local coordinate system establishment module is used to read the point cloud of the part CAD model and establish the local coordinate system using the PCA algorithm;

[0111] The point cloud registration module is used to calculate the contour features and FPFH features of the point cloud to be registered, perform coarse registration based on the similarity of the extracted feature points and feature descriptors of different point cloud sets, use the ICP algorithm for fine registration, and obtain the transformation matrix to realize part posture estimation.

[0112] For the specific definition of the 6D posture estimation device for parts based on PCA and CAD models, please refer to the definition of the 6D posture estimation method for parts based on PCA and CAD models above, which will not be repeated here. Each module in the above-mentioned 6D posture estimation device for parts based on PCA and CAD models can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0113] In one embodiment, an electronic device is provided. The electronic device may be a computer, and its internal structure diagram may be as follows: Figure 3 As shown. The electronic device includes a processor, a memory and a network interface connected through a system bus. Among them, the processor of the device is used to provide computing and control capabilities. The memory of the device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used for 6D posture estimation data of parts based on PCA and CAD models. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for estimating the 6D posture of a part based on PCA and CAD models is implemented.

[0114] Those skilled in the art will understand that Figure 3 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0115] In one embodiment of the present application, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned method for estimating the 6D pose of a part based on PCA and CAD model are implemented.

[0116] The computer-readable storage medium provided in this embodiment has similar implementation principles and technical effects to those of the above method embodiments, and will not be described in detail here.

[0117] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing related hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in M ​​forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (SyMchliMk) DRAM (SLDRAM), memory bus (RaMbus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0118] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0119] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be construed as limiting the scope of the patent application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent application shall be subject to the attached claims.

Claims

1. A method for estimating 6D pose of a part based on PCA and CAD model, characterized in that: The method comprises: The 3D point cloud of the original part in any posture on the workbench of the robot arm is obtained through the 3D depth camera, and the 3D point cloud of the original part is processed to obtain the point cloud of the part to be estimated in posture; Preprocessing the part point cloud to obtain a point cloud containing only the parts; Read the point cloud of the part CAD model and use the PCA algorithm to establish the local coordinate system; The contour features and FPFH features of the point cloud to be registered are calculated respectively, and coarse registration is performed according to the similarity of the feature points and feature descriptors extracted from different point cloud sets. The ICP algorithm is used for fine registration, and the transformation matrix is ​​obtained to realize part posture estimation.

2. The part 6D posture estimation method according to claim 1, characterized in that: Process the original part 3D point cloud, including: Filter the original part's 3D point cloud to remove noise points and outliers in the point cloud; segment the filtered point cloud, remove the workbench point cloud in the filtered point cloud, and obtain a 3D point cloud containing only the parts; Among them, the original part three-dimensional point cloud is conditionally filtered, and the distance between the points in the original part three-dimensional point cloud is compared with the preset distance threshold L. If the distance between the points in the original point cloud is greater than this threshold, the point is filtered and removed; the point cloud after conditional filtering is radius filtered, and a sphere is generated with each point in the point cloud obtained by filtering as the center and the preset radius value R as the radius. The number of points in the sphere is calculated, and if the number is less than the number threshold M, the point is removed.

3. The part 6D posture estimation method according to claim 2, characterized in that: The filtered point cloud is segmented, including: The plane model of the workbench is obtained, and the distance between each point in the filtered point cloud and the plane is calculated. If the distance is less than the preset distance threshold ε1, the point is a point in the plane and is segmented and removed. The iteration is stopped when the set maximum value is reached or the total number of point clouds remains unchanged, and a point cloud containing only parts is obtained.

4. The part 6D posture estimation method according to claim 1, characterized in that: Read the part CAD model point cloud and use the PCA algorithm to establish the local coordinate system, including: First, the decentralized covariance matrix of the CAD model point cloud is obtained, the three eigenvalues ​​of the covariance matrix from large to small are calculated, and the corresponding eigenvectors are calculated. The three eigenvectors are used as the x-axis, y-axis and z-axis to establish a local coordinate system, and the local coordinate system is transformed into the camera coordinate system. Among them, reading the part CAD model point cloud and establishing the local coordinate system using the PCA algorithm includes decentralizing the CAD model point cloud, obtaining the covariance matrix, calculating the eigenvalues ​​of the covariance matrix, arranging them from large to small, and respectively obtaining the eigenvectors corresponding to the first three eigenvalues. The eigenvectors are mutually orthogonal, and the local coordinate system is established with the obtained three eigenvectors as the x-axis, y-axis and z-axis respectively.

5. The part 6D posture estimation method according to claim 1, characterized in that: Calculate the contour features and FPFH features of the point cloud to be registered respectively, and perform rough registration based on the similarity between the extracted feature points and feature descriptors of different point cloud sets, including: First, the contour features and FPFH features of the point cloud are extracted. Based on the contour features and FPFH features, the CAD point cloud is used as the source point cloud and the processed part point cloud is used as the target point cloud for rough registration. Based on the rough registration, the ICP algorithm is used for fine registration. After the two point clouds overlap, the position and posture of the part can be obtained. Any point in the point cloud is projected onto its tangent plane from its neighborhood, and the point is connected with the projected point to obtain a series of angles with the point as the corner point. If the maximum angle is greater than the set angle threshold ξ, the point is a boundary contour point. The contour features and FPFH features of the two point clouds to be tested are extracted, and the point pairs to be matched are screened out according to the similarity between the extracted feature points and feature descriptors of different point cloud sets. The amount of registration data calculation is reduced by screening the point pairs to be matched. The point clouds are roughly registered by combining the two features, and the CAD point cloud and the part point cloud partially overlap.

6. The part 6D posture estimation method according to claim 1, characterized in that: Use the ICP algorithm for precise registration, including: Each point in the CAD point cloud searches for the nearest point in the part point cloud as a matching point pair, calculates the direction vector of the matching point pair, and compares it with the preset direction vector threshold. If the direction vector threshold is met, it is a correctly matched point pair. When the distance error between the registered point cloud and the part point cloud is less than the distance threshold ε2 or the maximum number of iterations is reached, the iteration is stopped to complete the point cloud precise registration. The transformation matrices of the coarse and fine registration are calculated, and the global transformation matrix is ​​obtained from the obtained transformation matrix as the part pose.

7. A device for estimating 6D posture of a part based on PCA and CAD model, characterized in that: The device comprises: An acquisition module is used to acquire a 3D point cloud of an original part in any posture on a robotic arm grabbing workbench through a 3D depth camera, process the 3D point cloud of the original part, and acquire a point cloud containing a part to be estimated in posture; A point cloud preprocessing module, used for preprocessing the part point cloud to obtain a point cloud containing only the parts; The local coordinate system establishment module is used to read the point cloud of the part CAD model and establish the local coordinate system using the PCA algorithm; The point cloud registration module is used to calculate the contour features and FPFH features of the point cloud to be registered, perform coarse registration based on the similarity of the extracted feature points and feature descriptors of different point cloud sets, use the ICP algorithm for fine registration, and obtain the transformation matrix to realize part posture estimation.

8. An electronic device, characterized in that: It comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the part 6D posture estimation method as described in any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a processor, the method for estimating the 6D posture of a part as described in any one of claims 1 to 6 is implemented.

10. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by a processor, the method for estimating the 6D posture of a part described in any one of claims 1 to 6 is implemented.

Images