A method and apparatus for registration of stacked workpiece occlusion point clouds based on mesh feature description

Abstract

This invention discloses a method and apparatus for registering occluded point clouds of stacked workpieces based on mesh feature description. The method includes: acquiring the pose point cloud of the workpiece and the target point cloud of the workpiece and performing preprocessing to obtain a synthetic candidate point set of the workpiece template point cloud and a candidate point set of the workpiece target point cloud; calculating the Gaussian curvature of each point in the two candidate point sets, selecting points with Gaussian curvature greater than a threshold as key points, and obtaining a key point set of the template point cloud and a key point set of the workpiece target point cloud; using the geometric features of neighborhood triangular patches to perform feature description on the two key point sets and calculating the chi-square similarity to determine the initial set of corresponding point pairs; using the cosine similarity difference method to remove erroneous point pairs to obtain the correct point pair relationship; randomly selecting n sets of corresponding point pairs, solving the rigid transformation matrix through SVD singular value decomposition to complete coarse registration, selecting the rigid transformation matrix with the smallest error among several coarse registrations as the coarse registration result, and combining it with the ICP algorithm to obtain the final registration result.

Description

Technical Field

[0001] This invention belongs to the field of three-dimensional registration technology, and particularly relates to a method and apparatus for registering occluded point clouds of stacked workpieces based on mesh feature description. Background Technology

[0002] To enable a robotic arm to grasp randomly placed and stacked workpieces in a deep basket with the correct posture, it is necessary to estimate the pose of the target workpiece. In recent years, 3D data acquisition equipment such as laser sensors, millimeter-wave radar, and structured light have become increasingly inexpensive. Researchers use 3D scanning equipment to obtain 3D point cloud data of the target workpiece and perform rigid point cloud registration between the 3D target point cloud and the template point cloud with known pose, thereby realizing the identification and localization of the target workpiece. This has become a research focus in the field of 3D registration technology.

[0003] Traditional scenarios involving disordered workpiece grasping are often ideal, where workpieces are arranged in an orderly fashion on a horizontal workbench with no obvious occlusion between adjacent workpieces, making pose estimation relatively easy. However, in actual industrial production, workpieces are often scattered and stacked in deep baskets, with occlusion and stacking between them. This results in significant information loss in the acquired 3D point cloud data, leading to reduced registration accuracy and weak robustness of existing point cloud registration methods. For example, existing methods using 2D images cannot effectively address the workpiece occlusion problem, and deep learning methods that directly use point clouds as input in recent years cannot adequately solve the problem of missing point cloud information.

[0004] In the process of realizing this invention, the inventors discovered at least the following problems in the prior art:

[0005] Existing point cloud registration methods in industry suffer from reduced registration accuracy and slow registration speed in complex environments such as occlusion caused by stacked workpieces in deep baskets and missing point cloud information. Summary of the Invention

[0006] To address the problems existing in the prior art, the purpose of this application is to provide a method and apparatus for registering occluded point clouds of stacked workpieces based on mesh feature description. Compared with existing methods, it has stronger robustness and registration accuracy in complex environments such as missing point cloud information caused by stacked parts and occlusion.

[0007] According to a first aspect of the embodiments of this application, a method for registering occlusion point clouds of stacked workpieces based on mesh feature description is provided, including:

[0008] (1) Obtain the pose point cloud and target point cloud of the workpiece, and perform data preprocessing on the pose point cloud and target point cloud to obtain a synthetic candidate point set P of the workpiece template point cloud. c and the candidate point set Q of the target point cloud of the workpiece c;

[0009] (2) Calculate the candidate point set P of the workpiece template point cloud. c and the candidate point set Q of the target point cloud of the workpiece c Gaussian curvature at each point as well as Where m and n are the candidate point sets P c and Q c The number of points is determined, and points with a Gaussian curvature greater than a predetermined threshold ε are selected as key points to obtain the template point cloud key point set S and the workpiece target point cloud key point set T.

[0010] (3) Use the geometric features of the neighborhood triangular facets to perform feature description on the key point set S of the template point cloud and the key point set T of the workpiece target point cloud to obtain the feature description matrix;

[0011] (4) Calculate the chi-square similarity of the feature description matrix and determine the initial set of corresponding point pairs E = {(M i , H j M i ∈T,H j ∈S}, where T is the key point set of the target point cloud of the workpiece, and S is the key point set of the template point cloud;

[0012] (5) For the initial set of corresponding point pairs E, the cosine similarity difference method is used to remove erroneous point pairs and obtain the correct point pair relationship;

[0013] (6) Randomly select n pairs of corresponding points from the correct point pair relationships, where n≥3, and solve the rigidity change matrix using SVD singular value decomposition to obtain the candidate point set P of the workpiece template point cloud. c Coarse registration is achieved by transforming the target point cloud of the workpiece into the same coordinate system using a rigid transformation matrix. After several rounds of random selection of n corresponding point pairs for coarse registration, the rigid transformation matrix with the smallest error is selected as the coarse registration result, and the final registration result is obtained by combining it with the ICP algorithm.

[0014] Further, in step (1), data preprocessing is performed on the pose point cloud and the working target point cloud, including:

[0015] Step 1.1: Use three-dimensional voxel filtering to obtain downsampled workpiece pose point cloud P and workpiece target point cloud Q from the workpiece pose point cloud and workpiece target point cloud.

[0016] Step 1.2: Merge the workpiece pose point cloud P into a complete workpiece template point cloud P. s The workpiece target point cloud Q is divided into multiple target point clouds Q. s ;

[0017] Step 1.3: Assess the point cloud P of the workpiece template. s and target point cloud Q s The greedy projection triangulation method is used for mesh triangulation, where points that do not form triangular meshes are regarded as discrete noise points.

[0018] Step 1.4: Extract non-edge mesh vertices from the established triangulated mesh as candidate point sets to obtain the workpiece template point cloud candidate point set P. c and the candidate point set Q of the target point cloud of the workpiece c The non-edge grid vertex refers to a point whose state is a grid vertex, and whose complete first-order neighborhood topological points are all grid vertices.

[0019] Further, in step (2), the candidate point set P of the workpiece template point cloud is calculated. c and the candidate point set Q of the target point cloud of the workpiece c Gaussian curvature at each point as well as include:

[0020] Step 2.1: For the candidate point set P of the workpiece template point cloud c and the candidate point set Q of the target point cloud of the workpiece c For each point in the array, extract the first-order neighborhood polygon of that point;

[0021] Step 2.2: Calculate the Gaussian curvature K at each point. i The formula is as follows:

[0022]

[0023] Among them, A i The neighborhood area is the area of ​​the polygonal region formed by the perpendicular bisectors of the adjacent sides of the triangular facets in the first-order neighborhood polygon, θ. j Let be the included angle between adjacent sides, N be the number of triangular faces contained in the first-order neighborhood polygon, i represent the i-th point, and 1≤m+n.

[0024] Further, in step (3), the key point set S of the template point cloud and the key point set T of the workpiece target point cloud are described using the geometric features of the neighborhood triangular facets, including:

[0025] Step 3.1: Traverse each key point P in the template point cloud key point set S and the workpiece target point cloud key point set T. i And extract the first-order neighborhood polygon of that point;

[0026] Step 3.2: Calculate the mesh quaternions for each of the first-order neighborhood polygons, where the mesh quaternions are... in

[0027]

[0028]

[0029]

[0030]

[0031] Point P i For the key points of the template point cloud key point set S or the workpiece target point cloud key point set T, the points are... and For the key point P i Two adjacent first-order neighborhood topological points, point and and the key point P i Together they form a neighborhood triangular facet, where F1, F2, and F3 are the three included angles of the triangular facet, and F4 is the adjacent edge of the triangular facet. and The perpendicular bisector and point P i The area of ​​the polygon formed by the two polygons;

[0032] Step 3.3: For the four elements of the grid Histogram analysis is performed, where F1, F2, F3, and F4 are set with intervals f1, f2, f3, and f4 respectively, thereby generating key point P. i The local feature description vector MSPH(P) in f1+f2+f3+f4 dimensions i );

[0033] Step 3.4: Use a weighted approach to generate the final local feature description vector (MPPH) for each keypoint:

[0034]

[0035] Where P i Let j be the key point and P be the key point. i The number of key points in the first-order neighborhood topology, w l For key point P i and key points The Euclidean distance between them For P i It is a first-order neighborhood topological point and a key point.

[0036] Further, in step (4), the chi-square similarity of the feature description matrix is ​​calculated to determine the initial set of corresponding point pairs, including:

[0037] Step 4.1: Calculate any key point M in the key point set T of the workpiece target point cloud. iWith any key point H in the template point cloud key point set S j Chi-square similarity d(M) i H j ), where M i The local feature description vector is MPPH = {a1, a2, ..., a...} N}, H j The local feature description vector is MPPH(H) j )={b1,b2,…,b N}:

[0038]

[0039] in MPPH(M) i The mean of the local feature description vector. MPPH(H) j The mean of the local feature description vector, where N is the dimension of the local feature description vector;

[0040] Step 4.2: Combine the key points in the key point set T of the workpiece target point cloud with the key points in the key point set S of the template point cloud that have the highest chi-square similarity to form an initial corresponding point pair, thereby obtaining the initial corresponding point pair set E = {(M i , H j M i ∈T,H j ∈S}.

[0041] Further, in step (5), the initial set of corresponding point pairs E is processed using a cosine similarity difference method to remove erroneous point pairs, including:

[0042] Step 5.1: In the initial set of corresponding point pairs E = {(M i , H j M i ∈T,H j Three groups of elements are randomly selected from ∈S} to form a sample subset e={(p 1i h 1j ), (p 2i h 2j ), (p 3i h 3j ), p i ∈T,h j ∈S};

[0043] Step 5.2: Select point p from the sample subset. 1i p 2i p 3i Composition vector And calculate the cosine similarity according to the following formula:

[0044]

[0045] Step 5.3: Calculate the point h in the sample subset. 1j , h 2j , h 3j The vector formed The cosine similarity Sim2 is calculated, and |Sim1-Sim2| is calculated. An appropriate threshold ω>0 is set. When |Sim1-Sim2|<ω, the three sets of corresponding point pairs are the subset of correct corresponding point pairs and are retained. Otherwise, when |Sim1-Sim2|≥ω, they are incorrect corresponding point pairs and are removed.

[0046] Step 5.4: Repeat steps 5.1-5.3 to include the correct corresponding point pair sample subset e into the correct point pair set L until the number of point pairs in set L reaches the threshold or the number of iterations reaches the upper limit.

[0047] According to a second aspect of the embodiments of this application, a registration device for stacked workpiece occlusion point clouds based on mesh feature description is provided, comprising:

[0048] The acquisition module is used to acquire the pose point cloud of the workpiece and the target point cloud of the workpiece, and to perform data preprocessing on the pose point cloud and the target point cloud to obtain a synthetic candidate point set P for the workpiece template point cloud. c and the candidate point set Q of the target point cloud of the workpiece c ;

[0049] The filtering module is used to calculate the candidate point set P of the workpiece template point cloud. c and the candidate point set Q of the target point cloud of the workpiece c Gaussian curvature at each point as well as Where m and n are the candidate point sets P c and Q c The number of points is determined, and points with a Gaussian curvature greater than a predetermined threshold ε are selected as key points to obtain the template point cloud key point set S and the workpiece target point cloud key point set T.

[0050] The feature description module is used to describe the key point set S of the template point cloud and the key point set T of the workpiece target point cloud using the geometric features of the neighborhood triangular facets, and obtain the feature description matrix.

[0051] The calculation module is used to calculate the chi-square similarity of the feature description matrix and determine the initial set of corresponding point pairs E = {(M i , H j M i ∈T,H j∈S}, where T is the key point set of the target point cloud of the workpiece, and S is the key point set of the template point cloud;

[0052] The removal module is used to remove erroneous point pairs from the initial set of corresponding point pairs E using the cosine similarity difference method, so as to obtain the correct point pair relationships;

[0053] The registration module is used to randomly select n sets of corresponding point pairs from the correct point pair relationships, where n≥3, and solve the rigidity change matrix through SVD singular value decomposition to obtain the candidate point set P of the workpiece template point cloud. c Coarse registration is achieved by transforming the target point cloud of the workpiece into the same coordinate system using a rigid transformation matrix. After several rounds of random selection of n corresponding point pairs for coarse registration, the rigid transformation matrix with the smallest error is selected as the coarse registration result, and the final registration result is obtained by combining it with the ICP algorithm.

[0054] According to a third aspect of the embodiments of this application, an electronic device is provided, comprising:

[0055] one or more processors;

[0056] Memory, used to store one or more programs;

[0057] When the one or more programs are executed by the one or more processors, the one or more processors perform the method as described in the first aspect.

[0058] According to a fourth aspect of the embodiments of this application, a computer-readable storage medium is provided, having stored thereon computer instructions, characterized in that, when executed by a processor, the instructions implement the steps of the method as described in the first aspect.

[0059] The technical solutions provided by the embodiments of this application may include the following beneficial effects:

[0060] As can be seen from the above embodiments, this application proposes a point cloud registration method for stacked workpieces with occlusion based on mesh feature description to address the pose estimation problem of stacked workpieces in deep baskets. This solves the problems of reduced point cloud registration accuracy and slow registration speed of existing point cloud registration methods in industry under complex environments such as occlusion caused by stacked workpieces in deep baskets and missing point cloud information.

[0061] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0062] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0063] Figure 1 This is a flowchart illustrating a method for registering occluded point clouds of stacked workpieces based on mesh feature description, according to an exemplary embodiment.

[0064] Figure 2 This is a schematic diagram illustrating the extraction of candidate points from the point cloud of a gimbal workpiece according to an exemplary embodiment.

[0065] Figure 3 This is a schematic diagram of a first-order neighborhood polygon according to an exemplary embodiment.

[0066] Figure 4 This is a schematic diagram illustrating a mesh vertex curvature estimation method according to an exemplary embodiment.

[0067] Figure 5 This is a schematic diagram of a four-element grid according to an exemplary embodiment.

[0068] Figure 6 This is a schematic diagram illustrating a cosine similarity difference elimination method according to an exemplary embodiment, where (a) is point p. 1i p 2i p 3i The vector formed A schematic diagram of the included angle between them, where (b) represents a case where point h is... 1j , h 2j , h 3j The vector formed A diagram showing the included angle between the two points, where (c) represents point h in another case. 1j , h 2j , h 3j The vector formed A diagram showing the included angle between them.

[0069] Figure 7 This is a schematic diagram illustrating the point cloud registration result of two stacked gimbal workpieces occlusion according to an exemplary embodiment.

[0070] Figure 8 This is a schematic diagram illustrating the point cloud registration result of a three-layer stacked gimbal workpiece occlusion according to an exemplary embodiment.

[0071] Figure 9 This is a block diagram illustrating a stacked workpiece occlusion point cloud registration apparatus based on a mesh feature description, according to an exemplary embodiment.

[0072] Figure 10 This is a schematic diagram of an electronic device according to an exemplary embodiment. Detailed Implementation

[0073] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application.

[0074] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. As used in this application and the appended claims, the singular forms "a," "an," "the," and "the" are intended to include the plural forms, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0075] It should be understood that although the terms first, second, third, etc., may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0076] Figure 1 This is a flowchart illustrating a method for registering occlusion point clouds of stacked workpieces based on mesh feature description, according to an exemplary embodiment. Figure 1 As shown, this method, when applied to a terminal, may include the following steps:

[0077] (1) Obtain the pose point cloud and target point cloud of the workpiece, and perform data preprocessing on the pose point cloud and target point cloud to obtain a synthetic candidate point set P of the workpiece template point cloud. c and the candidate point set Q of the target point cloud of the workpiece c ;

[0078] (2) Calculate the candidate point set P of the workpiece template point cloud. c and the candidate point set Q of the target point cloud of the workpiece c Gaussian curvature at each point as well as Where m and n are the candidate point sets P c and Q c The number of points is determined, and points with a Gaussian curvature greater than a predetermined threshold ε are selected as key points to obtain the template point cloud key point set S and the workpiece target point cloud key point set T.

[0079] (3) Use the geometric features of the neighborhood triangular facets to perform feature description on the key point set S of the template point cloud and the key point set T of the workpiece target point cloud to obtain the feature description matrix;

[0080] (4) Calculate the chi-square similarity of the feature description matrix and determine the initial set of corresponding point pairs E = {(M i , H j M i ∈T,H j ∈S}, where T is the key point set of the target point cloud of the workpiece, and S is the key point set of the template point cloud;

[0081] (5) For the initial set of corresponding point pairs E, the cosine similarity difference method is used to remove erroneous point pairs and obtain the correct point pair relationship;

[0082] (6) Randomly select n pairs of corresponding points from the correct point pair relationships, where n≥3, and solve the rigidity change matrix using SVD singular value decomposition to obtain the candidate point set P of the workpiece template point cloud. c Coarse registration is achieved by transforming the target point cloud of the workpiece into the same coordinate system using a rigid transformation matrix. After several rounds of random selection of n corresponding point pairs for coarse registration, the rigid transformation matrix with the smallest error is selected as the coarse registration result, and the final registration result is obtained by combining it with the ICP algorithm.

[0083] As can be seen from the above embodiments, this application proposes a point cloud registration method for stacked workpieces with occlusion based on mesh feature description to address the pose estimation problem of stacked workpieces in deep baskets. This solves the problems of reduced point cloud registration accuracy and slow registration speed of existing point cloud registration methods in industry under complex environments such as occlusion caused by stacked workpieces in deep baskets and missing point cloud information.

[0084] In the specific implementation of step (1), the pose point cloud of the workpiece and the target point cloud of the workpiece are obtained. The pose point cloud and the target point cloud of the workpiece are preprocessed to obtain the synthesized candidate point set P of the workpiece template point cloud. c and the candidate point set Q of the target point cloud of the workpiece c ;

[0085] Specifically, data preprocessing is performed on the pose point cloud and the target point cloud, including:

[0086] Step 1.1: Use three-dimensional voxel filtering to obtain downsampled workpiece pose point cloud P and workpiece target point cloud Q from the workpiece pose point cloud and workpiece target point cloud.

[0087] Specifically, the process begins with an offline phase where the workpiece pose point cloud is input, followed by an online phase where the workpiece target point cloud is input. A 3D voxel filtering method is then used to obtain downsampled workpiece pose point cloud P and workpiece target point cloud Q. In one embodiment, four gimbal workpiece pose point clouds are input offline, with point cloud counts for poses 1, 2, 3, and 4 being 4967, 4875, 5545, and 4651 respectively, totaling 20038. In the online phase, two stacked gimbal workpiece target point clouds are input, totaling 24966 points. A 3D voxel grid of the point clouds is created, with a grid resolution of 2. A cube that perfectly encloses the point cloud is first created on the input point cloud data. Then, according to the set grid resolution, this large cube is divided into smaller cubes. The centroid of each point within a smaller cube is calculated, and the centroid is used to approximate the other points within the smaller cube. After voxel filtering downsampling, the total number of workpiece pose point clouds decreased from 20038 to 6368, and the number of target point clouds of stacked gimbal workpieces decreased from 24966 to 12741. In specific implementations, downsampling methods such as mean filtering and uniform sampling can also be used. In this embodiment, the three-dimensional voxel filtering method can not only effectively reduce the amount of point cloud data, but also preserve the basic geometric features of the point cloud to the greatest extent.

[0088] Step 1.2: Merge the workpiece pose point cloud P into a complete workpiece template point cloud P. s The workpiece target point cloud Q is divided into multiple target point clouds Q. s ;

[0089] Specifically, after downsampling, the amount of point cloud data is significantly reduced. In the offline stage, the workpiece pose point cloud P is merged into a complete workpiece template point cloud P. s In the online phase, the workpiece target point cloud Q is segmented into multiple target point clouds Q. s The reason for segmentation in this step is that the stacked workpieces in the deep frame cause the acquired point cloud to stick together.

[0090] Step 1.3: Assess the point cloud P of the workpiece template. s and target point cloud Q s The greedy projection triangulation method is used for mesh triangulation, where points that do not form triangular meshes are regarded as discrete noise points.

[0091] Specifically, then the point cloud P s and the point cloud Q s A greedy projection triangulation method is used for mesh triangulation; points that do not form triangular meshes can be considered as discrete noise points; (Reference) Figure 2 , Figure 2This is a schematic diagram illustrating candidate point extraction from a gimbal workpiece point cloud according to an exemplary embodiment. In this embodiment, the gimbal workpiece point cloud is basically meshed after greedy projection triangulation, but some point cloud meshing fails. The meshed point cloud contains three types of points: outliers indicated by rectangular boxes, edge mesh vertices indicated by circular boxes, and non-edge mesh vertices indicated by triangular boxes. The greedy projection triangulation method is suitable for processing scattered point clouds with multiple connections and has good adaptability.

[0092] Step 1.4: Extract non-edge mesh vertices from the established triangulated mesh as candidate point sets to obtain the workpiece template point cloud candidate point set P. c and the candidate point set Q of the target point cloud of the workpiece c The non-edge grid vertex refers to a point whose state is a grid vertex, and whose complete first-order neighborhood topological points are all grid vertices.

[0093] Specifically, non-edge mesh vertices of the point cloud are extracted from the established point cloud mesh as candidate point sets to obtain the workpiece template point cloud candidate point set P. c and the candidate point set Q of the target point cloud of the workpiece c ; specifically as Figure 2 The point cloud shown in the middle triangle box.

[0094] In this embodiment, the number of gimbal workpiece pose point clouds after voxel filtering downsampling is 6368, and the number of target point clouds of stacked gimbal workpieces is 12741. After extracting its non-edge mesh vertices as candidate point sets, the total number of extracted workpiece pose point cloud candidate point sets is 3313, and the number of target point cloud candidate points is 6331.

[0095] In the specific implementation of step (2), the candidate point set P of the workpiece template point cloud is calculated. c and the candidate point set Q of the target point cloud of the workpiece c Gaussian curvature at each point as well as Where m and n are the candidate point sets P c and Q c The number of points is determined, and points with a Gaussian curvature greater than a predetermined threshold ε are selected as key points to obtain the template point cloud key point set S and the workpiece target point cloud key point set T.

[0096] Specifically, the candidate point set P of the workpiece template point cloud is calculated. c and the candidate point set Q of the target point cloud of the workpiece c Gaussian curvature at each point as well as include:

[0097] Step 2.1: For the candidate point set P of the workpiece template point cloudc and the candidate point set Q of the target point cloud of the workpiece c For each point in the array, extract the first-order neighborhood polygon of that point;

[0098] Specifically, refer to Figure 3 and Figure 4 In this embodiment, the candidate point set P is first... c and Q c For each point in the array, extract the first-order neighborhood polygon of that point. For example, Figure 3 As shown, the circular box selects a non-edge mesh vertex in the candidate point set of the gimbal workpiece point cloud. The lines connecting this point and the surrounding mesh vertices form multiple triangular patches, and all adjacent triangular patches together form a complete first-order neighborhood polygon.

[0099] Step 2.2: Calculate the Gaussian curvature K at each point. i The formula is as follows:

[0100]

[0101] Among them, A i The neighborhood area is the area of ​​the polygonal region formed by the perpendicular bisectors of the adjacent sides of the triangular facets in the first-order neighborhood polygon, θ. j Let be the included angle between adjacent sides, N be the number of triangular faces contained in the first-order neighborhood polygon, i represent the i-th point, and 1≤m+n.

[0102] For example, Figure 4 As shown, point P i As candidate points, and Let A be two adjacent topological points in a first-order neighborhood polygon, and let A be the neighborhood area. i for and The perpendicular bisector and point P i The area of ​​the polygon formed by the two, and θ j Adjacent edges and The included angle.

[0103] In this embodiment, after extracting non-edge mesh vertices from the point cloud as candidate point sets, the total number of candidate point sets for the gimbal workpiece pose point cloud is 3313, and the number of candidate point sets for the target point cloud is 6331. After calculating the Gaussian curvature of each point in the candidate point set, a threshold of 0.02 is set according to an empirical value (retaining about 1000 points with larger curvature values), and points with Gaussian curvature greater than 0.02 are selected as key points. Thus, the number of key points extracted from the gimbal workpiece pose point cloud is 670, and the number of key points in the target point cloud is 1043.

[0104] In the specific implementation of step (3), the key point set S of the template point cloud and the key point set T of the workpiece target point cloud are described by the geometric features of the neighborhood triangular facets to obtain the feature description matrix;

[0105] Specifically, the key point set S of the template point cloud and the key point set T of the workpiece target point cloud are described using the geometric features of neighboring triangular facets, including:

[0106] Step 3.1: Traverse each key point P in the template point cloud key point set S and the workpiece target point cloud key point set T. i And extract the first-order neighborhood polygon of that point;

[0107] Specifically, refer to Figure 5 In this embodiment, according to the index value, each point in the template point cloud key point set S and the workpiece target point cloud key point set T is traversed, and the first-order neighborhood polygon of the point is extracted. The process of extracting the first-order neighborhood polygon is as described in step 2.1, and will not be repeated here.

[0108] Step 3.2: Calculate the mesh quaternions of the first-order neighborhood polygon, wherein the mesh quaternions are as follows:

[0109] Point P i These are the key points of the template point cloud key point set S or the workpiece target point cloud key point set T. Let P be the point i First-order neighborhood topological points, defined as grid four elements in

[0110]

[0111]

[0112]

[0113]

[0114] midpoint and For the key point P i Two adjacent first-order neighborhood topological points, point and and the key point P i Together they form a neighborhood triangular facet, where F1, F2, and F3 are the three included angles of the triangular facet, and F4 is the adjacent edge of the triangular facet. and The perpendicular bisector and point P i The area of ​​the polygon formed by the two polygons;

[0115] exist Figure 5 In the middle, point P i These are the key points in the key point set T of the target point cloud of the workpiece. Let P be the point i First-order neighborhood topological points, calculate the topological points between two adjacent topological points. and and key point P i The three included angles F1, F2, and F3 that together form the neighborhood triangle, along with the adjacent sides of the triangle, constitute the triangle. and The perpendicular bisector and point P i The area of ​​the polygon formed by the two elements is F4, and thus the four elements of the grid are obtained according to the above formula.

[0116] Step 3.3: For the four elements of the grid Histogram analysis is performed, where F1, F2, F3, and F4 are set with intervals f1, f2, f3, and f4 respectively, thereby generating key point P. i The local feature description vector MSPH(P) in f1+f2+f3+f4 dimensions i );

[0117] In practice, the size of the interval can be set according to the actual situation and needs. It is acceptable if it is set too small, the features will not be obvious, and if it is set too large, the amount of calculation will increase.

[0118] Step 3.4: The final local feature description vector (MPPH) for each keypoint is generated using a weighted method; the weighting formula is as follows:

[0119]

[0120] Where P i k is the key point P. i The number of key points in the first-order neighborhood topology, w l For key point P i and key points The Euclidean distance between them For P i It is a first-order neighborhood topological point and a key point.

[0121] In the specific implementation of step (4), the chi-square similarity is calculated on the feature description matrix to determine the initial set of corresponding point pairs E = {(P i , H j ), P i ∈T,H j ∈S}, where T is the key point set of the target point cloud of the workpiece, and S is the key point set of the template point cloud;

[0122] Specifically, calculating the chi-square similarity of the feature description matrix to determine the initial set of corresponding point pairs includes:

[0123] Step 4.1: Calculate any key point M in the key point set T of the workpiece target point cloud. i With any key point H in the template point cloud key point set S j Chi-square similarity d(M) i H j ), where M i The local feature description vector is MPPH(M) i )={a1,a2,…,a N}, H j The local feature description vector is MPPH(H) j )={b1,b2,…,b N}:

[0124]

[0125] in MPPH(M) i The mean of the local feature description vector. MPPH(H) j The mean of the local feature description vector, N = f1 + f2 + f3 + f4, is the dimension of the local feature description vector; d(P i H j )∈(-1,1), the closer the value is to 1, the stronger the two local feature description vectors MPPH(P) i ) and MPPH(H j The stronger the similarity;

[0126] In one embodiment, a key point M is defined in the key point set T of the gimbal workpiece target point cloud. i Local feature description vector MPPH(M) i )={a1,a2,…,a 39}, where f1=f2=f3=11 and f4=6 are set based on empirical values, as follows:

[0127]

[0128] Similarly, define a key point H in the key point set S of the point cloud of the universal joint workpiece template. j The local feature description vector is MPPH(H) j )={b1,b2,…,b 39}for:

[0129]

[0130] Using the formula above, the correlation between the two points is calculated as d(M). i H jIf )≈0.95, then it indicates that the key point M i and key point H j There is a strong correlation between them.

[0131] Step 4.2: Combine the key points in the key point set T of the workpiece target point cloud with the key points in the key point set S of the template point cloud that have the highest chi-square similarity to form an initial corresponding point pair, thereby obtaining the initial corresponding point pair set E = {(M i , H j M i ∈T,H j ∈S}.

[0132] In the specific implementation of step (5), the cosine similarity difference method is used to remove erroneous point pairs from the initial set of corresponding point pairs E, so as to obtain the correct point pair relationship;

[0133] Specifically, for the initial set of corresponding point pairs E, the cosine similarity difference method is used to remove erroneous point pairs, including:

[0134] Step 5.1: In the initial set of corresponding point pairs E = {(M i , H j M i ∈T,H j Three groups of elements are randomly selected from ∈S} to form a sample subset e={(p 1i h 1j ), (p 2i h 2j ), (p 3i h 3j ), p i ∈T,h j ∈S};

[0135] Step 5.2: Select point p from the sample subset. 1i p 2i p 3i Composition vector And calculate the cosine similarity according to the following formula:

[0136]

[0137] Step 5.3: Calculate the point h in the sample subset. 1j h 2j h 3j The vector formed The cosine similarity Sim2 is calculated, and |Sim1-Sim2| is calculated. An appropriate threshold ω>0 is set. When |Sim1-Sim2|<ω, the three sets of corresponding point pairs are the subset of correct corresponding point pairs and are retained. Otherwise, when |Sim1-Sim2|≥ω, they are incorrect corresponding point pairs and are removed.

[0138] For example, Figure 6 (a) in the diagram represents point p. 1i p 2i p 3i The vector formed A diagram showing the included angle between them is provided. The cosine value is calculated to be Sim1. Figure 6 (b) in the diagram represents point h. 1j h 2j h 3j The vector formed A diagram illustrating the angle between them is used. The cosine value is calculated as Sim2, and then |Sim1-Sim2| is calculated. An appropriate threshold ω > 0 is set empirically, and a subset of samples with a cosine similarity difference |Sim1-Sim2| < ω is selected and included in the effective set of corresponding point pairs. If... Figure 6 The cosine similarity difference between (a) and (b) is used as the criterion for the value of ω. Clearly... Figure 6 If the cosine similarity difference |Sim1-Sim2| between (a) and (c) is greater than ω, then these three pairs of corresponding points do not meet the screening criteria and should be removed.

[0139] Step 5.4: Repeat steps 5.1-5.3 to include the correct corresponding point pair sample subset e into the correct point pair set L until the number of point pairs in set L reaches the threshold or the number of iterations reaches the upper limit.

[0140] Specifically, based on experience, the threshold for the number of correct point pairs is set to within 150, and the number of iterations can be set within a reasonable range of 1000 to 1500. The aim is to effectively improve the efficiency of point cloud registration and fully include the sample subset that meets the conditions into the effective point pair set.

[0141] In the specific implementation of step (6), n sets of corresponding point pairs are randomly selected from the correct point pair relationships, n≥3, and the rigidity change matrix is ​​solved by SVD singular value decomposition to obtain the candidate point set P of the workpiece template point cloud. c Coarse registration is completed by transforming the workpiece target point cloud into the same coordinate system using a rigid transformation matrix. After several rounds of random selection of n corresponding point pairs for coarse registration, the rigid transformation matrix with the smallest error is selected as the coarse registration result, and the final registration result is obtained by combining it with the ICP algorithm.

[0142] Specifically, n pairs of corresponding points are randomly selected from the valid point-to-point relationships, where n ≥ 3. For example, when n = 3, 3 valid point pairs are selected: {(x1, y1), (x2, y2), (x3, y3), x...} i ∈T,y j ∈S}, (i=1,2,3), solve for the rigid transformation matrix between the three pairs of points using SVD singular value decomposition, and then use the rigid transformation matrix to select the candidate point set P of the workpiece template point cloud. c Coarse registration is completed by transforming the coordinates to the same coordinate system as the target point cloud of the workpiece. Then, the point set P is traversed according to the index value. c For each point in the target point cloud, calculate and sum the distances between that point and the nearest point in the target point cloud of the workpiece, and use this sum as the error of the rigid transformation. Based on empirical values, the number of iterations is reasonably set within the range of 200-250. In each iteration, the same n pairs of corresponding points are selected, and the corresponding rigid transformation errors are statistically analyzed. The rigid transformation matrix with the smallest error is selected as the coarse registration result, and then optimized using the ICP algorithm to complete the final registration.

[0143] The purpose of Singular Value Decomposition (SVD) is to solve for the coordinate system transformation between point pairs, and the number of point pairs must be greater than or equal to 3. Assuming n is 3, then 3 corresponding point pairs mean that three points in the template point cloud correspond to three points in the target point cloud. Therefore, SVD can solve for the coordinate system transformation between the template point cloud and the target point cloud, achieving coarse registration. The purpose of the Integrated Collision Program (ICP) algorithm is optimization; our entire registration process is a coarse registration, providing a good fine-registration pose for the ICP algorithm.

[0144] This application also conducted a registration experiment on the actual acquired stacked and scattered workpiece scene point cloud, and the acquired gimbal workpiece target point cloud showed obvious stacking, occlusion, and missing parts. Figure 7 and Figure 8 The actual registration results of point clouds occluded by two-layer stacked universal joint workpieces and three-layer stacked universal joint workpieces are shown respectively. It can be seen that the registration method of stacked workpiece occlusion point cloud based on mesh feature description has strong robustness and registration accuracy in the face of complex situations such as component stacking, occlusion, and missing point cloud information.

[0145] Corresponding to the aforementioned embodiments of the stacked workpiece occlusion point cloud registration method based on mesh feature description, this application also provides embodiments of the stacked workpiece occlusion point cloud registration device based on mesh feature description.

[0146] Figure 9 This is a block diagram illustrating a stacked workpiece occlusion point cloud registration device based on mesh feature description, according to an exemplary embodiment. (Refer to...) Figure 9 The device may include:

[0147] The acquisition module 21 is used to acquire the pose point cloud of the workpiece and the target point cloud of the workpiece, and to perform data preprocessing on the pose point cloud and the target point cloud to obtain a synthetic candidate point set P of the workpiece template point cloud. c and the candidate point set Q of the target point cloud of the workpiece c ;

[0148] Filtering module 22 is used to calculate the candidate point set P of the workpiece template point cloud. c and the candidate point set Q of the target point cloud of the workpiece c Gaussian curvature at each point as well as Where m and n are the candidate point sets P c and Q c The number of points is determined, and points with a Gaussian curvature greater than a predetermined threshold ε are selected as key points to obtain the template point cloud key point set S and the workpiece target point cloud key point set T.

[0149] Feature description module 23 is used to perform feature description on the key point set S of the template point cloud and the key point set T of the workpiece target point cloud using the geometric features of the neighborhood triangular facets, and obtain a feature description matrix.

[0150] Calculation module 24 is used to calculate the chi-square similarity of the feature description matrix and determine the initial set of corresponding point pairs E = {(M i , H j M i ∈T,H j ∈S}, where T is the key point set of the target point cloud of the workpiece, and S is the key point set of the template point cloud;

[0151] The removal module 25 is used to remove erroneous point pairs from the initial set of corresponding point pairs E using the cosine similarity difference method, so as to obtain the correct point pair relationship;

[0152] Registration module 26 is used to randomly select n sets of corresponding point pairs from the correct point pair relationships, where n≥3, and solve the rigidity change matrix through SVD singular value decomposition to obtain the candidate point set P of the workpiece template point cloud. c Coarse registration is achieved by transforming the target point cloud of the workpiece into the same coordinate system using a rigid transformation matrix. After several rounds of random selection of n corresponding point pairs for coarse registration, the rigid transformation matrix with the smallest error is selected as the coarse registration result, and the final registration result is obtained by combining it with the ICP algorithm.

[0153] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0154] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this application according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0155] Accordingly, this application also provides an electronic device, including: one or more processors; a memory for storing one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the stacked workpiece occlusion point cloud registration method based on mesh feature description as described above. Figure 10 The diagram shown is a hardware structure diagram of any device with data processing capabilities, used in an embodiment of the present invention to provide a method for registering occluded point clouds of stacked workpieces based on mesh feature description. (Except for...) Figure 10 In addition to the processor, memory, and network interface shown, any data processing device in the embodiment may also include other hardware depending on the actual function of the data processing device, which will not be described in detail here.

[0156] Accordingly, this application also provides a computer-readable storage medium storing computer instructions, which, when executed by a processor, implement the stacked workpiece occlusion point cloud registration method described above based on mesh feature description. The computer-readable storage medium can be an internal storage unit of any data processing device as described in any of the foregoing embodiments, such as a hard disk or memory. The computer-readable storage medium can also be an external storage device, such as a plug-in hard disk, smart media card (SMC), SD card, flash card, etc., equipped on the device. Furthermore, the computer-readable storage medium can include both internal storage units of any data processing device and external storage devices. The computer-readable storage medium is used to store the computer program and other programs and data required by the data processing device, and can also be used to temporarily store data that has been output or will be output.

[0157] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the disclosure herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein.

[0158] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope.

Claims

1. A method for registering occlusion point clouds of stacked workpieces based on mesh feature description, characterized in that, include: (1) Obtain the pose point cloud and the target point cloud of the workpiece, and perform data preprocessing on the pose point cloud and the target point cloud to obtain a synthetic candidate point set of the workpiece template point cloud. and candidate point set of target point cloud of workpiece ; (2) Calculate the candidate point set of the workpiece template point cloud. and candidate point set of target point cloud of workpiece Gaussian curvature at each point as well as ,in and Candidate point sets and The number of points is used to filter out points whose Gaussian curvature is greater than a predetermined threshold. The points are used as key points to obtain the template point cloud key point set. and the key point set of the target point cloud of the workpiece ; (3) Utilize the geometric features of the neighborhood triangular facets to analyze the key point set of the template point cloud. and the key point set of the target point cloud of the workpiece Perform feature description to obtain the feature description matrix; (4) Calculate the chi-square similarity of the feature description matrix to determine the initial set of corresponding point pairs E={( ), }, where T is the key point set of the target point cloud of the workpiece, and S is the key point set of the template point cloud; (5) For the initial set of corresponding point pairs E, the cosine similarity difference method is used to remove erroneous point pairs and obtain the correct point pair relationship; (6) Randomly select from the correct point pairs. Group corresponding point pairs, The rigidity transformation matrix is ​​solved by Singular Value Decomposition (SVD) to obtain the candidate point set of the workpiece template point cloud. Coarse registration was achieved by transforming the workpiece target point cloud to the same coordinate system using a rigid transformation matrix. This was followed by several random selections of the same number of points. Coarse registration is performed on the corresponding point pairs. The rigid transformation matrix with the smallest error is selected as the coarse registration result, and the final registration result is obtained by combining it with the ICP algorithm. In step (3), the key point set of the template point cloud is analyzed using the geometric features of the neighborhood triangular facets. and the key point set of the target point cloud of the workpiece Feature description includes: Step 3.1: Traverse the template point cloud keypoint set and the key point set of the target point cloud of the workpiece Each key point in And extract the first-order neighborhood polygon of that point; Step 3.2: Calculate the mesh quaternions for each of the first-order neighborhood polygons, where the mesh quaternions are... ,in ； ； ； ； midpoint For the template point cloud key point set Or the key point set of the target point cloud of the workpiece Key point, point and For the key points Two adjacent first-order neighborhood topological points, point and and the key points mentioned Together they form a neighborhood triangular facet. These are the three included angles of the triangular facet. Adjacent edges of the triangular facet and perpendicular bisector and point The area of ​​the polygon formed by the two polygons; Step 3.3: For the four elements of the grid Perform histogram statistics, where Set separately This generates key points by dividing the interval into several ranges. of Local feature description vector MSPH (dimensional) ); Step 3.4: Use a weighted approach to generate the final local feature description vector (MPPH) for each keypoint: ， in As the key point, Key point The number of key points in the first-order neighborhood topology. Key point and key points The Euclidean distance between them for It is a first-order neighborhood topological point and a key point.

2. The method according to claim 1, characterized in that, In step (1), the pose point cloud and the target point cloud are preprocessed, including: Step 1.1: Use a three-dimensional voxel filter to obtain a downsampled workpiece pose point cloud from the workpiece pose point cloud and the workpiece target point cloud. and workpiece target point cloud ; Step 1.2: Generate the workpiece pose point cloud Merge into a complete workpiece template point cloud The target point cloud of the workpiece Segmented into multiple target point clouds ; Step 1.3: Point cloud of the workpiece template and target point cloud Greedy projection triangulation is used for mesh triangulation, where points that do not form triangular meshes are considered as discrete noise points. Step 1.4: Extract non-edge mesh vertices from the established triangulated mesh as candidate point sets to obtain the candidate point set of the workpiece template point cloud. and candidate point set of target point cloud of workpiece The non-edge grid vertex refers to a point whose state is a grid vertex, and whose complete first-order neighborhood topological points are all grid vertices.

3. The method according to claim 1, characterized in that, In step (2), the candidate point set of the workpiece template point cloud is calculated. and candidate point set of target point cloud of workpiece Gaussian curvature at each point as well as ,include: Step 2.1: For the candidate point set of the workpiece template point cloud and candidate point set of target point cloud of workpiece For each point in the array, extract the first-order neighborhood polygon of that point; Step 2.2: Calculate the Gaussian curvature at each point. , the formula is as follows: ,j=1,2,…,N; in, The neighborhood area is the area of ​​the polygonal region formed by the perpendicular bisectors of the adjacent sides of the triangular facets in the first-order neighborhood polygon. Let be the included angle between adjacent sides, and N be the number of triangular faces contained in the first-order neighborhood polygon. Indicates the first One point, .

4. The method according to claim 1, characterized in that, In step (4), the chi-square similarity of the feature description matrix is ​​calculated to determine the initial set of corresponding point pairs, including: Step 4.1: Calculate the key point set of the target point cloud of the workpiece. any key point in With template point cloud key point set any key point in Chi-square similarity ,in The local feature description vector is , The local feature description vector is : ， in for The mean of the local feature description vector. for The mean of the local feature description vector, where N is the dimension of the local feature description vector; Step 4.2: Set the key points of the workpiece target point cloud. Key points in the template point cloud and key point set The keypoints with the highest chi-square similarity in the corresponding data form an initial pair of corresponding points, thus obtaining the initial set of corresponding point pairs E={( ), } 5. The method according to claim 1, characterized in that, In step (5), the cosine similarity difference method is used to remove erroneous point pairs from the initial set of corresponding point pairs E, including: Step 5.1: In the initial set of corresponding point pairs E={( ), Randomly select 3 groups of elements from} to form a sample subset e = {( ), ( , ( ), }; Step 5.2: Select points from the sample subset. , , Composition vector , And calculate the cosine similarity according to the following formula: ， Step 5.3: Calculate the points in the sample subset. , , The vector formed , cosine similarity and calculate Set an appropriate threshold When satisfied If the three sets of corresponding point pairs are true, then these three sets are the subset of correctly corresponding point pairs and are retained; otherwise, if the three sets are false, they are retained. Error-related points are identified and removed. Step 5.4: Repeat steps 5.1-5.3 to include the correct corresponding point pair sample subset e into the correct point pair set L until the number of point pairs in set L reaches the threshold or the number of iterations reaches the upper limit.

6. A registration device for occlusion point clouds of stacked workpieces based on mesh feature description, characterized in that, include: The acquisition module is used to acquire the pose point cloud of the workpiece and the target point cloud of the workpiece, and to perform data preprocessing on the pose point cloud and the target point cloud to obtain a synthetic candidate point set of the workpiece template point cloud. and candidate point set of target point cloud of workpiece ; The filtering module is used to calculate the candidate point set of the workpiece template point cloud. and candidate point set of target point cloud of workpiece Gaussian curvature at each point as well as ,in and Candidate point sets and The number of points is used to filter out points whose Gaussian curvature is greater than a predetermined threshold. The points are used as key points to obtain the template point cloud key point set. and the key point set of the target point cloud of the workpiece ; The feature description module is used to describe the key point set of the template point cloud using the geometric features of the neighborhood triangular facets. and the key point set of the target point cloud of the workpiece Perform feature description to obtain the feature description matrix; The calculation module is used to calculate the chi-square similarity of the feature description matrix and determine the initial set of corresponding point pairs E={( ), }, where T is the key point set of the target point cloud of the workpiece, and S is the key point set of the template point cloud; The removal module is used to remove erroneous point pairs from the initial set of corresponding point pairs E using the cosine similarity difference method, so as to obtain the correct point pair relationships; The registration module is used to randomly select from the correct point pairs. Group corresponding point pairs, The rigidity transformation matrix is ​​solved by Singular Value Decomposition (SVD) to obtain the candidate point set of the workpiece template point cloud. Coarse registration was achieved by transforming the workpiece target point cloud to the same coordinate system using a rigid transformation matrix. This was followed by several random selections of the same number of points. Coarse registration is performed on the corresponding point pairs. The rigid transformation matrix with the smallest error is selected as the coarse registration result, and the final registration result is obtained by combining it with the ICP algorithm. In the feature description module, the geometric features of the neighborhood triangular facets are used to describe the key point set of the template point cloud. and the key point set of the target point cloud of the workpiece Feature description includes: Step 3.1: Traverse the template point cloud keypoint set and the key point set of the target point cloud of the workpiece Each key point in And extract the first-order neighborhood polygon of that point; Step 3.2: Calculate the mesh quaternions for each of the first-order neighborhood polygons, where the mesh quaternions are... ,in ； ； ； ； midpoint For the template point cloud key point set Or the key point set of the target point cloud of the workpiece Key point, point and For the key points Two adjacent first-order neighborhood topological points, point and and the key points mentioned Together they form a neighborhood triangular facet. These are the three included angles of the triangular facet. Adjacent edges of the triangular facet and perpendicular bisector and point The area of ​​the polygon formed by the two polygons; Step 3.3: For the four elements of the grid Perform histogram statistics, where Set separately This generates key points by dividing the interval into several ranges. of Local feature description vector MSPH (dimensional) ); Step 3.4: Use a weighted approach to generate the final local feature description vector (MPPH) for each keypoint: ， in As the key point, Key point The number of key points in the first-order neighborhood topology. Key point and key points The Euclidean distance between them for It is a first-order neighborhood topological point and a key point.

7. An electronic device, characterized in that, include: One or more processors; Memory, used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-5.

8. A computer-readable storage medium storing computer instructions thereon, characterized in that, When executed by the processor, this instruction implements the steps of the method as described in any one of claims 1-5.

Images