A point cloud data registration method, system, electronic device and storage medium

Abstract

The application discloses a point cloud data registration method and system, electronic equipment and a storage medium, and the method comprises the following steps: acquiring a model point cloud of a three-dimensional model of a scene and a scene point cloud; performing plane feature extraction on the model point cloud and the scene point cloud to obtain a model point cloud plane feature set and a scene point cloud plane feature set respectively; performing line feature extraction on the model point cloud and the scene point cloud according to the two feature sets to obtain a corresponding model point cloud line feature set and a scene point cloud line feature set; performing feature matching on the model point cloud and the scene point cloud based on the model point cloud plane feature set, the scene point cloud plane feature set, the model point cloud line feature set and the scene point cloud line feature set to determine a feature matching relationship; and determining a coordinate transformation relationship of the model point cloud and the scene point cloud based on the feature matching relationship to register the model point cloud and the scene point cloud. Through the above method, the registration between the three-dimensional model and the point cloud can be realized through the plane feature and the line feature, and the registration efficiency is improved.

Description

Technical Field

[0001] This application relates to the field of 3D scene modeling technology, and in particular to a point cloud data registration method, system electronic equipment, and storage medium. Background Technology

[0002] With the increasing maturity and popularity of computer technology, virtual content and the real world can interact through 3D AR. Among them, the registration between 3D models and point clouds is particularly important. The existing point cloud data registration method is to register directly between point clouds.

[0003] In the process of researching and practicing existing technologies, the inventors of this application have discovered that with the popularity of 3D AR, more and more large-scale scenes need to register point clouds. Registering a large amount of point cloud data will increase the amount of computation and increase the time consumption, affecting the registration speed. Insufficient matching points will affect the matching accuracy. In addition, when the initial value of the iteration is not selected properly, it will lead to the inability to obtain the optimal registration result. Summary of the Invention

[0004] The main technical problem addressed in this application is to provide a point cloud data registration method, system electronic equipment, and storage medium that can achieve registration between a 3D model and a point cloud through the planar and line features of the 3D model and the point cloud.

[0005] To address the aforementioned technical problems, this application provides a point cloud data registration method, comprising: acquiring a 3D model of a scene and a scene point cloud; acquiring a model point cloud based on the vertex coordinates of the 3D model; extracting planar features from the model point cloud and the scene point cloud to obtain a model point cloud planar feature set and a scene point cloud planar feature set, respectively; extracting line features from the model point cloud and the scene point cloud based on the model point cloud planar feature set and the scene point cloud planar feature set to obtain corresponding model point cloud line feature sets and scene point cloud line feature sets; performing feature matching on the model point cloud and the scene point cloud based on the model point cloud planar feature set, the scene point cloud planar feature set, the model point cloud line feature set, and the scene point cloud line feature set to determine a feature matching relationship; and determining a coordinate transformation relationship between the model point cloud and the scene point cloud based on the feature matching relationship, so as to register the model point cloud and the scene point cloud.

[0006] In one embodiment of this application, the step of obtaining a 3D model of a scene and a scene point cloud, and obtaining the model point cloud based on the vertex coordinates of the 3D model, includes: obtaining a 3D model of the scene and obtaining the vertex coordinates of the 3D model to construct a model point cloud; wherein, the scene point cloud is a standard point cloud; the step of performing planar feature extraction on the model point cloud and the scene point cloud to obtain a model point cloud planar feature set and a scene point cloud planar feature set respectively includes: performing planar feature extraction on the model point cloud to obtain a first model point cloud planar feature set; and performing planar feature extraction on the scene point cloud to obtain a first scene point cloud planar feature set.

[0007] In one embodiment of this application, the step of extracting planar features from the model point cloud to obtain a first model point cloud planar feature set, and extracting planar features from the scene point cloud to obtain a first scene point cloud planar feature set, includes: extracting planar features from the data in the model point cloud using random sampling to obtain a first model point cloud planar feature set and point cloud data corresponding to each plane; extracting planar features from the data in the scene point cloud using random sampling to obtain a first scene point cloud planar feature set and point cloud data corresponding to each plane; wherein, the first model point cloud planar feature set and the first scene point cloud planar feature set are merged into a plane to obtain a corresponding second model point cloud planar feature set and a second scene point cloud planar feature set.

[0008] In one embodiment of this application, the step of merging the first model point cloud planar feature set and the first scene point cloud planar feature set to obtain the corresponding second model point cloud planar feature set and second scene point cloud planar feature set includes: merging the first model point cloud planar feature set and the first scene point cloud planar feature set respectively; wherein, merging the first model point cloud planar feature set involves: traversing all planes of the first model point cloud planar feature set, merging point cloud data belonging to the same plane in the first model point cloud planar feature set, updating the plane after merging the point cloud data, and thus obtaining the second model point cloud planar feature set; merging the first scene point cloud planar feature set... The plane merging process involves: traversing all planes in the first scene point cloud plane feature set, merging point cloud data belonging to the same plane in the first scene point cloud plane feature set, updating the plane after merging the point cloud data, and then obtaining the second scene point cloud plane feature set; the step of extracting line features from the model point cloud and the scene point cloud based on the model point cloud plane feature set and the scene point cloud plane feature set to obtain the corresponding model point cloud line feature set and scene point cloud line feature set includes: determining and extracting the corresponding model point cloud line feature set and scene point cloud line feature set based on the intersecting lines between planes in the second model point cloud plane feature set and the intersecting lines between planes in the second scene point cloud plane feature set, respectively.

[0009] In one embodiment of this application, the step of determining and extracting the corresponding model point cloud line feature set and scene point cloud line feature set based on the intersecting lines between planes in the second model point cloud plane feature set and the intersecting lines between planes in the second scene point cloud plane feature set, respectively, includes: determining a selected plane based on the second model point cloud plane feature set, traversing the remaining planes in the second model point cloud plane feature set, obtaining other planes intersecting with the selected plane, and thus obtaining the corresponding intersecting lines, determining the intersecting lines as model point cloud line features, and determining the model point cloud line feature set of the second model point cloud plane feature set based on the model point cloud line features; and determining a selected plane based on the second scene point cloud plane feature set, traversing the remaining planes in the second scene point cloud plane feature set, obtaining other planes intersecting with the selected plane, and thus obtaining the corresponding intersecting lines, determining the intersecting lines as scene point cloud line features, and determining the scene point cloud line feature set of the second scene point cloud plane feature set based on the scene point cloud line features.

[0010] In one embodiment of this application, after obtaining the corresponding intersecting lines, it is determined whether there are intersecting lines between two planar point clouds based on the actual position of the planar point clouds: if there are intersecting lines between the two planar point clouds, the intersecting lines are retained and used as line features; if there are no intersecting lines between the two planar point clouds, the intersecting lines are deleted.

[0011] In one embodiment of this application, determining whether two planar point clouds intersect by a straight line based on the actual position of the planar point cloud includes: obtaining a first planar point cloud and a second planar point cloud corresponding to the first plane and the second plane, respectively, based on the first plane and the second plane corresponding to the intersecting straight line; obtaining the distance from each point in the plane to the intersecting straight line; traversing each point of the first planar point cloud and each point of the second planar point cloud respectively; if the distance from a point to the intersecting straight line is less than a set threshold, the point is considered to be on the intersecting straight line; otherwise, the point is considered not to be on the intersecting straight line; if both the first planar point cloud and the second planar point cloud have more than a set number of points on the intersecting straight line, the first planar point cloud and the second planar point cloud are considered to have an intersecting straight line; otherwise, the first planar point cloud and the second planar point cloud are considered not to have an intersecting straight line.

[0012] In one embodiment of this application, the step of performing feature matching on the model point cloud and the scene point cloud based on the model point cloud planar feature set, the scene point cloud planar feature set, the model point cloud line feature set, and the scene point cloud line feature set to determine the feature matching relationship includes: for each planar feature in the second model point cloud planar feature set, obtaining the planar feature that matches it in the second scene point cloud planar feature set; for each model point cloud line feature in the model point cloud line feature set, obtaining the scene point cloud line feature that matches it in the scene point cloud line feature set; and determining the feature matching relationship based on the matching relationship of the planar features and the matching relationship of the line features.

[0013] In one embodiment of this application, determining the coordinate transformation relationship between the model point cloud and the scene point cloud based on the feature matching relationship for registration of the model point cloud and the scene point cloud includes: setting an initial coordinate transformation relationship; transforming the model point cloud to the scene point cloud according to the initial coordinate transformation relationship based on the feature matching relationship; constructing an error term in the feature matching relationship based on the error; establishing an objective function based on the error term; optimizing the objective function; determining the optimal coordinate transformation relationship; and registering the model point cloud and the scene point cloud according to the optimal coordinate transformation relationship.

[0014] In one embodiment of this application, after acquiring the 3D model of the scene and the scene point cloud, the method further includes: performing voxelization processing on the model point cloud and the scene point cloud respectively according to the spatial size of the model point cloud and the scene point cloud; then performing planar extraction on the point cloud data in each voxel of the voxelized model point cloud to obtain a first model point cloud planar feature set and point cloud data of each plane; and performing planar extraction on the point cloud data in each voxel of the voxelized scene point cloud to obtain a first scene point cloud planar feature set and point cloud data of each plane.

[0015] To address the aforementioned technical problems, another technical solution adopted in this application is: providing a point cloud data registration system, comprising: an acquisition module for acquiring a 3D model of a scene and a scene point cloud, and acquiring a model point cloud based on the vertex coordinates of the 3D model; a first extraction module for extracting planar features from the model point cloud and the scene point cloud to obtain a model point cloud planar feature set and a scene point cloud planar feature set, respectively; a second extraction module for extracting line features from the model point cloud and the scene point cloud based on the model point cloud planar feature set and the scene point cloud planar feature set to obtain corresponding model point cloud line feature sets and scene point cloud line feature sets; a matching module for performing feature matching on the model point cloud and the scene point cloud based on the model point cloud planar feature set, the scene point cloud planar feature set, the model point cloud line feature set, and the scene point cloud line feature set to determine a feature matching relationship; and a determination module for determining the coordinate transformation relationship between the model point cloud and the scene point cloud based on the feature matching relationship, so as to register the model point cloud and the scene point cloud.

[0016] To solve the above-mentioned technical problems, another technical solution adopted in this application is to provide an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method described above.

[0017] To solve the above-mentioned technical problems, another technical solution adopted in this application is to provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the method described above.

[0018] Unlike existing technologies, the point cloud data registration method provided in this application includes: acquiring a 3D model of a scene and a scene point cloud; acquiring a model point cloud based on the vertex coordinates of the 3D model; extracting planar features from the model point cloud and the scene point cloud to obtain a model point cloud planar feature set and a scene point cloud planar feature set, respectively; extracting line features from the model point cloud and the scene point cloud based on the model point cloud planar feature set and the scene point cloud planar feature set to obtain corresponding model point cloud line feature sets and scene point cloud line feature sets; performing feature matching on the model point cloud and the scene point cloud based on the model point cloud planar feature set, the scene point cloud planar feature set, the model point cloud line feature set, and the scene point cloud line feature set to determine a feature matching relationship; and determining the coordinate transformation relationship between the model point cloud and the scene point cloud based on the feature matching relationship to register the model point cloud and the scene point cloud. By using the planar and line features of the 3D model and the scene point cloud for registration, the registration speed can be improved while simultaneously increasing registration accuracy and efficiency. Attached Figure Description

[0019] Figure 1 This is a flowchart illustrating an embodiment of the point cloud data registration method of the present invention;

[0020] Figure 2 This is a flowchart illustrating an embodiment of step S1 of the present invention;

[0021] Figure 3 This is a flowchart illustrating an embodiment of step S2 of the present invention;

[0022] Figure 4 This is a flowchart illustrating a subsequent embodiment of step S2 of the present invention;

[0023] Figure 5 This is a flowchart illustrating an embodiment of step S23 of the present invention;

[0024] Figure 6 This is a flowchart illustrating an embodiment of step S3 of the present invention;

[0025] Figure 7 This is a flowchart illustrating an embodiment of the intersecting line determination step of the present invention;

[0026] Figure 8 This is a flowchart illustrating an embodiment of step S4 of the present invention;

[0027] Figure 9 This is a schematic diagram of an embodiment of the point cloud data registration system of the present invention;

[0028] Figure 10 This is a schematic diagram of the structure of an embodiment of the electronic device of the present invention;

[0029] Figure 11 This is a schematic diagram of the structure of an embodiment of the computer-readable storage medium of the present invention. Detailed Implementation

[0030] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be particularly noted that the following embodiments are for illustrative purposes only and do not limit the scope of the invention. Similarly, the following embodiments are only some, not all, embodiments of the present invention, and all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0031] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0032] Traditional point cloud registration methods mainly involve direct registration between point clouds, using registration techniques such as ICP. Matching points are determined by iteratively calculating the distance between points. However, in large-scale scenarios, when registering a large amount of point cloud data, the time consumption increases significantly, the registration efficiency is low, and there is a possibility of getting stuck in local optima, leading to inaccurate matching.

[0033] The applicant found in the research that when registering large amounts of point cloud data, extracting some key feature values ​​to represent the corresponding point cloud data relationships can significantly reduce the registration time, thereby improving registration efficiency and effectively avoiding getting trapped in local optima.

[0034] Therefore, a novel point cloud data registration method is proposed. This method acquires the model point cloud and scene point cloud of the 3D model of the scene, extracts planar features based on the model point cloud and scene point cloud, extracts line features based on the planar features, performs feature matching based on the planar features and line features, determines the feature matching relationship, and determines the coordinate transformation relationship between the model point cloud and scene point cloud based on the feature matching relationship, so as to register the model point cloud and scene point cloud.

[0035] Please see Figure 1 , Figure 1 This is a flowchart illustrating an embodiment of the point cloud data registration method of the present invention; it should be noted that if substantially the same result is obtained, the method of the present invention is not necessarily identical. Figure 1 The process sequence shown is limited to the following: Figure 1 As shown, the method includes the following steps:

[0036] S1. Obtain the 3D model and scene point cloud of the scene, and obtain the model point cloud based on the vertex coordinates of the 3D model;

[0037] See Figure 2 , Figure 2 This is a flowchart illustrating an embodiment of step S1, which includes:

[0038] S11. Obtain the 3D model of the scene and the scene point cloud;

[0039] A 3D model is a polygonal representation of an object, typically displayed using a computer or other video equipment. The displayed object can be a real-world entity or a fictional object; anything physically existing in nature can be represented by a 3D model. A scene can be a scene to be registered, or it can be used for verification of a registered scene. The 3D model is a 3D representation of the scene, generated using 3D modeling tools or other methods. A scene point cloud is standard point cloud data, which is a set of vectors in a 3D coordinate system containing geometric position information.

[0040] Specifically, for a scene, a corresponding 3D model is generated using 3D modeling tools, and the scene point cloud Q corresponding to the scene is obtained based on standard point cloud data.

[0041] S12. Obtain the vertex coordinates of the 3D model to construct the model point cloud.

[0042] Specifically, the coordinates of all vertices of the 3D model are obtained to get the vertex data that constitutes the 3D model, and the vertex data is used to form the model point cloud M.

[0043] In some embodiments, after acquiring the 3D model of the scene and the scene point cloud, the method further includes: performing voxelization on the model point cloud M and the scene point cloud Q according to their spatial sizes, and then performing planar extraction on the point cloud data in each voxel of the voxelized model point cloud to obtain a first model point cloud planar feature set and point cloud data for each plane; and performing planar extraction on the point cloud data in each voxel of the voxelized scene point cloud to obtain a first scene point cloud planar feature set and point cloud data for each plane.

[0044] In voxelization, a voxel is short for volume element. A solid containing voxels can be represented by stereo rendering or by extracting polygonal isosurfaces of a given threshold contour. It is the smallest unit of digital data in three-dimensional space segmentation. Some 3D displays use voxels to describe the resolution of the display, such as a display that can display 512×512×512 voxels.

[0045] Specifically, based on the size of the space occupied by the model point cloud M and the scene point cloud Q, the model point cloud M and the scene point cloud Q are represented by stereo rendering or by extracting polygonal isosurfaces of a given threshold contour, i.e., voxelization is performed.

[0046] S2. Extract planar features from the model point cloud and the scene point cloud to obtain the planar feature set of the model point cloud and the planar feature set of the scene point cloud, respectively.

[0047] See Figure 3 , Figure 3 This is a flowchart illustrating one embodiment of step S2, which includes:

[0048] S21. Extract planar features from the model point cloud to obtain the first model point cloud planar feature set;

[0049] In some embodiments, the point cloud data in each voxel of the voxelized model point cloud is extracted in a plane to obtain a first model point cloud planar feature set.

[0050] Specifically, the point cloud of the scene's 3D model is subjected to planar extraction using the RANSAC method to obtain a set of planar features S. M ={P1,P2,…,P m} and the point cloud data corresponding to each plane; where S M P represents all planar features extracted from the point cloud M of the model. i The i-th plane extracted from the point cloud can be represented by the following plane equation:

[0051] n i ·(pc i ) = 0

[0052] Where, n i =(nx i ,ny i ,nz i Let ) represent the normal vector of plane i, which is a unit vector, while c i =(cx i ,cy i ,nz i ) represents the center point of plane i, and p represents the coordinates of any point in three-dimensional space;

[0053] S22. Extract planar features from the scene point cloud to obtain the first scene point cloud planar feature set.

[0054] In some embodiments, the point cloud data in each voxel of the voxelized scene point cloud is extracted in a plane to obtain a first scene point cloud planar feature set.

[0055] Specifically, the scene point cloud is extracted into a plane using the RANSAC method to obtain a set of planar features S. Q ={P1,P2,…,P n} and the point cloud data corresponding to each plane; where S Q P represents all planar features extracted from the scene point cloud Q. i The i-th plane extracted from the point cloud can be represented by the following plane equation:

[0056] n i ·(pc i ) = 0

[0057] Where, n i =(nx i ,ny i ,nz i Let ) represent the normal vector of plane i, which is a unit vector, while c i =(cx i ,cy i ,nz i ) represents the center point of plane i, and p represents the coordinates of any point in three-dimensional space.

[0058] See Figure 4 , Figure 4 This is a flowchart illustrating an embodiment following step S2, specifically, step S22 is followed by step S23, which involves plane merging.

[0059] S23. Perform planar merging on the first model point cloud planar feature set and the first scene point cloud planar feature set to obtain the corresponding second model point cloud planar feature set and the second scene point cloud planar feature set.

[0060] Specifically, the planar set S extracted from the model point cloud M and the scene point cloud Q. M and S Q Each point cloud belonging to the same plane is merged, duplicate planes are removed, and the merged planes are updated to obtain a new planar feature set S'. M ={P1,P2,…,P k} and S' Q ={P1,P2,…,P h} and the point cloud corresponding to each plane.

[0061] See Figure 5 , Figure 5 This is a flowchart illustrating one embodiment of step S23, which includes:

[0062] S231. Traverse all planes of the first model point cloud plane feature set, merge the point cloud data belonging to the same plane in the first model point cloud plane feature set, update the merged plane, and then obtain the second model point cloud plane feature set.

[0063] Specifically, for the planar feature set S M One of the planes P i Traverse the planar feature set S M All remaining planes in the middle, when the following condition is satisfied:

[0064]

[0065] This indicates that plane P iIt is the same plane as the plane that satisfies the conditions, for example, plane P. i and plane P j Since they are on the same plane, we can then consider plane P. j Remove it, and at the same time, remove the part belonging to plane P. j Point and plane P i The points in the equation are merged. In the above formula, thresholds 1 and 2 are the set threshold values.

[0066] After traversing the planar feature set S M For each plane in the cloud, duplicate planes have been removed, and points on the same plane have been merged. The next step is to re-fit the merged point cloud with planes to obtain a new set of planar features S'. M That is, the planar feature set of the point cloud of the second model.

[0067] S232. Traverse all planes of the first scene point cloud planar feature set, merge the point cloud data belonging to the same plane in the first scene point cloud planar feature set, update the merged plane, and then obtain the second scene point cloud planar feature set.

[0068] Specifically, for the planar feature set S Q One of the planes uses a set of features S similar to a plane. M The processing method involves traversing the planar feature set S. Q For all remaining planes, merge the planes.

[0069] After traversing the planar feature set S Q For each plane in the cloud, duplicate planes have been removed, and points on the same plane have been merged. The next step is to re-fit the merged point cloud with planes to obtain a new set of planar features S'. Q That is, the planar feature set of the point cloud in the second scene.

[0070] S3. Extract line features from the model point cloud and the scene point cloud based on the model point cloud planar feature set and the scene point cloud planar feature set to obtain the corresponding model point cloud line feature set and scene point cloud line feature set.

[0071] Specifically, for the second model point cloud planar feature set S' M Second scene point cloud planar feature set S' Q By utilizing the lines intersecting between intersecting planes, line features are extracted from the model point cloud M and the scene point cloud Q, respectively, to obtain the corresponding line feature set L. M ={l1,l2,…,l a} and L Q ={l1,l2,…,l b}, corresponding to the model point cloud line feature set and the scene point cloud line feature set, respectively, where the model point cloud line feature set L M It is based on the second model point cloud planar feature set S' M The feature set L of the scene point cloud obtained by the intersection of the planes in the image. Q It is based on the planar feature set S' of the second scene point cloud Q The intersection of the planes in the middle, l i The i-th line feature is represented as follows:

[0072]

[0073] Where, d i =(u i ,v i ,w i ) indicates line feature l i direction, p i =(x i ,y i ,z i ) indicates line feature l i A little bit above.

[0074] See Figure 6 , Figure 6 This is a flowchart illustrating one embodiment of step S3, which includes:

[0075] S31. Based on the intersecting straight lines between planes in the second model point cloud plane feature set, determine and extract the corresponding model point cloud line feature set;

[0076] In some embodiments, a selected plane is determined based on the second model point cloud planar feature set, the remaining planes in the second model point cloud planar feature set are traversed, other planes that intersect with the selected plane are obtained, and the corresponding intersecting lines are obtained. The intersecting lines are determined as model point cloud line features, and the model point cloud line feature set of the second model point cloud planar feature set is determined based on the model point cloud line features.

[0077] For example: using the second model point cloud planar feature set S' M Taking the planar features in the example, for a certain plane P... i Traverse the second model point cloud planar feature set S' M For all remaining planes, find those that may intersect plane P. i All intersecting planes; where the condition for determining whether planes intersect is as follows:

[0078] |n i ·n j | <thres3

[0079] Where, n i and nj Representing plane P respectively i and plane P j The normal vector, thres3 is the set threshold.

[0080] Assume plane P i and plane P j If they intersect, then the intersecting lines are l. i Direction d i Obtained in the following way:

[0081]

[0082] The above formula represents the cross product of n1 and n2, and a point p on the line... i We can obtain the following by solving a system of plane equations:

[0083]

[0084] z i By choosing any value and solving the above system of equations, we can obtain the corresponding x. i and y i Thus, a point p on the straight line is obtained. i =(x i ,y i ,z i );

[0085] After obtaining the equation of the line, it is also necessary to determine whether the two planar point clouds actually intersect the line based on the actual position of the planar point cloud. If the two point clouds have intersecting lines, i.e. collinear regions, then the line feature is retained.

[0086] Find the planar feature set S' of the second model point cloud M All line features are used to obtain the model point cloud line feature set L. M .

[0087] S32. Based on the intersecting straight lines between planes in the second scene point cloud plane feature set, determine and extract the corresponding scene point cloud line feature set;

[0088] In some embodiments, a selected plane is determined based on the second scene point cloud planar feature set. The remaining planes in the second scene point cloud planar feature set are traversed to obtain other planes that intersect with the selected plane, thereby obtaining the corresponding intersecting lines. The intersecting lines are determined as scene point cloud line features, and the scene point cloud line feature set of the second scene point cloud planar feature set is determined based on the scene point cloud line features. Specifically, the processing procedure in step S31 is adopted to find the second scene point cloud planar feature set S'. Q All line features are used to obtain the scene point cloud line feature set L. Q .

[0089] In some embodiments, after obtaining the corresponding intersecting line, it is determined whether there is an intersecting line between two plane point clouds according to the positions of the actual plane point clouds: if there is an intersecting line between the two plane point clouds, the intersecting line is retained and used as the line feature; if there is no intersecting line between the two plane point clouds, the intersecting line is deleted.

[0090] Refer to Figure 7 , Figure 7 which is a schematic flowchart of an embodiment of step one for determining whether there is an intersecting line between two plane point clouds, including:

[0091] Step A: Based on the first plane and the second plane corresponding to the intersecting line, obtain the first plane point cloud and the second plane point cloud corresponding to the first plane and the second plane, and obtain the distance from each point in the plane to the intersecting line; for example: for plane P i and plane P j , their corresponding plane point clouds can be obtained. Assuming the plane point clouds are PC i and PC j respectively, then for each point p in the plane, its distance to the intersecting line can be calculated according to the following formula:

[0092] d = ||(p - p i ) × d i ||;

[0093] Step B: Traverse each point of the first plane point cloud and each point of the second plane point cloud respectively. When the distance from the point to the intersecting line is less than the set threshold, it is considered that the point is on the intersecting line; otherwise, it is considered that the point is not on the intersecting line; for example: traverse each point in PC i and PC j respectively, calculate its distance to the line according to the above formula. When d < thres4, it is considered that the point is on the intersecting line l i ;

[0094] Step C: When there are more than the set number of points in the first plane point cloud and the second plane point cloud on the intersecting line, it is considered that there is an intersecting line between the first plane point cloud and the second plane point cloud; otherwise, it is considered that there is no intersecting line between the first plane point cloud and the second plane point cloud; for example: only when there are a certain number of points in both PC i and PC j on the intersecting line l i is it considered that plane P i and plane P j intersect to form an intersecting line l i , otherwise it is considered that there is no intersecting line between the plane point clouds PC i and PC j .

[0095] S4. Based on the model point cloud planar feature set, the scene point cloud planar feature set, the model point cloud line feature set, and the scene point cloud line feature set, perform feature matching on the model point cloud and the scene point cloud to determine the feature matching relationship.

[0096] See Figure 8 , Figure 8 This is a flowchart illustrating one embodiment of step S4, which includes:

[0097] S41. For each planar feature in the second model point cloud planar feature set, obtain the planar feature that matches it in the second scene point cloud planar feature set.

[0098] Specifically, for the second model point cloud planar feature set S' M For each planar feature in the second scene point cloud planar feature set S' Q The method for finding the corresponding plane in the middle is as follows:

[0099]

[0100] Where, n a ·(pc a ) = 0 indicates that the second model point cloud planar feature set S' M Planar features in n b ·(pc b ) = 0 indicates that the second scene point cloud planar feature set S' Q In the context of planar features, two planes are considered to be matched if they satisfy the above relationship.

[0101] S42. For each model point cloud line feature in the model point cloud line feature set, obtain the scene point cloud line feature that matches it in the scene point cloud line feature set.

[0102] Specifically, for the model point cloud line feature set L M Each line feature in the scene point cloud line feature set L Q To find the corresponding line in the given information, the specific method is as follows:

[0103]

[0104] Among them, (d) a ,p a ) represents the feature set L of the model point cloud. M Line features in (d) b ,p b L represents the feature set of scene point cloud lines. Q In line features, when two line features satisfy the above relationship, the two lines are considered to be matched.

[0105] S43. Determine feature matching relationships based on the matching relationships of planar features and line features.

[0106] S5. Based on feature matching relationships, determine the coordinate transformation relationship between the model point cloud and the scene point cloud in order to register the model point cloud and the scene point cloud.

[0107] Specifically, an initial coordinate transformation relationship is set. Based on the feature matching relationship, the model point cloud is transformed into the scene point cloud according to the initial coordinate transformation relationship. An error term is constructed in the feature matching relationship based on the error. An objective function is established based on the error term. The objective function is then optimized to determine the optimal coordinate transformation relationship. The model point cloud and the scene point cloud are registered according to the optimal coordinate transformation relationship.

[0108] In some embodiments, based on the feature matching relationship determined in the above steps, the coordinate transformation between the model point cloud M and the point cloud Q can be solved, thereby achieving the registration of the model and the point cloud.

[0109] Suppose the coordinate transformation matrix to be found is (R,t), which represents the transformation from the model to the point cloud, where R represents the rotation matrix and t represents the translation matrix. The specific solution process for the coordinate transformation matrix is ​​as follows:

[0110] Based on the feature matching relationship, when the model point cloud M is transformed to the scene point cloud Q in the coordinate system using the coordinate transformation matrix (R,t), the matched features will satisfy the following relationship:

[0111] For planar features, we have:

[0112]

[0113] For line features, we have:

[0114]

[0115] Due to the presence of errors, the equations in the two formulas above are not exactly equal. Therefore, the two formulas are transformed into the following form:

[0116] For planar features, we have:

[0117]

[0118] For line features, we have:

[0119]

[0120] Based on the transformed formula, four error terms are constructed from the matched planar and line features, and then the objective function is constructed:

[0121]

[0122] By optimizing the objective function, an optimal transformation matrix can be obtained when the objective function is minimized. This transformation matrix is ​​the coordinate transformation matrix from the model to the point cloud.

[0123] Unlike existing technologies, this embodiment acquires a 3D model of the scene and a scene point cloud, and obtains the model point cloud based on the vertex coordinates of the 3D model. Planar features are extracted from the model point cloud and the scene point cloud to obtain planar feature sets for the model point cloud and the scene point cloud, respectively. Line features are extracted from the model point cloud and the scene point cloud based on their respective planar feature sets to obtain corresponding line feature sets for the model point cloud and the scene point cloud. Feature matching is performed on the model point cloud and the scene point cloud based on these planar feature sets, and feature matching relationships are determined. Based on these feature matching relationships, the coordinate transformation relationships between the model point cloud and the scene point cloud are determined for registration. Registration using the planar and line features of the 3D model and the scene point cloud improves registration speed and accuracy, increases registration efficiency, and avoids getting trapped in local optima.

[0124] Please see Figure 9 , Figure 9 This is a schematic diagram of an embodiment of the point cloud data registration system of the present invention. This system can perform the point cloud data registration steps in the above method. For related details, please refer to the detailed description in the above method; it will not be repeated here.

[0125] The system 600 includes: an acquisition module 610, a first extraction module 620, a second extraction module 630, a matching module 640, and a determination module 650. The acquisition module 610 acquires a 3D model of the scene and a scene point cloud, obtaining the model point cloud based on the vertex coordinates of the 3D model. The first extraction module 620 extracts planar features from the model point cloud and the scene point cloud, obtaining a planar feature set for the model point cloud and a planar feature set for the scene point cloud, respectively. The second extraction module 630 extracts line features from the model point cloud and the scene point cloud based on their planar feature sets, obtaining corresponding line feature sets for the model point cloud and the scene point cloud. The matching module 640 performs feature matching on the model point cloud and the scene point cloud based on their planar feature sets, planar feature sets, line feature sets, and line feature sets, determining the feature matching relationship. The determination module 650 determines the coordinate transformation relationship between the model point cloud and the scene point cloud based on the feature matching relationship, for registration of the model point cloud and the scene point cloud.

[0126] Please see Figure 10 , Figure 10This is a schematic diagram of an embodiment of the electronic device of the present invention. This electronic device can perform the point cloud data registration step in the above method.

[0127] The electronic device 700 includes a memory 720, a processor 710, and a computer program stored in the memory 720 and executable on the processor 710. When the processor 710 executes the computer program, it can perform the point cloud data registration steps in the above method. For related details, please refer to the detailed description in the above method, which will not be repeated here.

[0128] Please see Figure 11 , Figure 11 This is a schematic diagram of an embodiment of the computer-readable storage medium of the present invention. The computer-readable storage medium 800 stores a computer program 810, which, when executed by a processor, implements the point cloud data registration step in the above-described method. For related details, please refer to the detailed description in the above-described method; it will not be repeated here.

[0129] The above solution acquires the model point cloud and scene point cloud of the 3D model of the scene, extracts planar features based on the model point cloud and scene point cloud, extracts line features based on the planar features, performs feature matching based on the planar features and line features, determines the feature matching relationship, and determines the coordinate transformation relationship between the model point cloud and scene point cloud based on the feature matching relationship, so as to register the model point cloud and scene point cloud. In the process of registering a large amount of point cloud data, registering by using the planar and line features of the 3D model and scene point cloud can improve the registration speed and accuracy, improve the registration efficiency, and avoid getting trapped in local optima.

[0130] In the several embodiments provided by this invention, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, indirect coupling or communication connection of apparatuses or units, and may be electrical, mechanical, or other forms.

[0131] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0132] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0133] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0134] The above description is merely an embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A point cloud data registration method, characterized in that, include: Obtain a 3D model and scene point cloud of the scene; obtain the model point cloud based on the vertex coordinates of the 3D model, wherein the scene point cloud is a standard point cloud; Planar feature extraction is performed on the model point cloud and the scene point cloud to obtain a first model point cloud planar feature set and a first scene point cloud planar feature set, respectively. The first model point cloud planar feature set and the first scene point cloud planar feature set are then merged in a plane to obtain a corresponding second model point cloud planar feature set and a second scene point cloud planar feature set. Line features are extracted from the model point cloud and the scene point cloud based on the second model point cloud planar feature set and the second scene point cloud planar feature set, respectively, to obtain the corresponding model point cloud line feature set and scene point cloud line feature set. Specifically, intersecting lines between two planes in the second model point cloud planar feature set and the second scene point cloud planar feature set are obtained. After obtaining the corresponding intersecting lines, it is determined whether there are intersecting lines between the two planar point clouds based on the actual positions of the planar point clouds. That is, based on the first and second planes corresponding to the intersecting lines, the first and second plane point clouds corresponding to the first and second planes are obtained, and the distance from each point in the plane to the intersecting line is obtained. The algorithm iterates through each point in the first planar point cloud and each point in the second planar point cloud. If the distance from a point to the intersecting line is less than a set threshold, the point is determined to be on the intersecting line; otherwise, the point is determined not to be on the intersecting line. If both the first and second planar point clouds have more than a set number of points on the intersecting line, the first and second planar point clouds are determined to have an intersecting line; otherwise, the first and second planar point clouds are determined not to have an intersecting line. If the two planar point clouds have an intersecting line, the intersecting line is retained and used as a line feature; if the two planar point clouds do not have an intersecting line, the intersecting line is deleted. Based on the model point cloud planar feature set, the scene point cloud planar feature set, the model point cloud line feature set, and the scene point cloud line feature set, feature matching is performed on the model point cloud and the scene point cloud to determine the feature matching relationship. Based on the feature matching relationship, the coordinate transformation relationship between the model point cloud and the scene point cloud is determined in order to register the model point cloud and the scene point cloud.

2. The method according to claim 1, characterized in that, The planar feature extraction of the model point cloud is performed to obtain the first model point cloud planar feature set; And perform planar feature extraction on the scene point cloud to obtain a first scene point cloud planar feature set, including: The data in the model point cloud are randomly sampled to extract planar features, resulting in a first model point cloud planar feature set and point cloud data corresponding to each plane. The data in the scene point cloud are subjected to random sampling for planar feature extraction to obtain the first scene point cloud planar feature set and the point cloud data corresponding to each plane.

3. The method according to claim 2, characterized in that, The step of merging the first model point cloud planar feature set and the first scene point cloud planar feature set to obtain the corresponding second model point cloud planar feature set and second scene point cloud planar feature set includes: The first model point cloud planar feature set and the first scene point cloud planar feature set are respectively merged into a planar feature set. Merging the first model point cloud planar feature set involves: traversing all planes in the first model point cloud planar feature set, merging point cloud data belonging to the same plane in the first model point cloud planar feature set, updating the merged plane, and thus obtaining the second model point cloud planar feature set. Merging the first scene point cloud planar feature set involves: traversing all planes in the first scene point cloud planar feature set, merging point cloud data belonging to the same plane in the first scene point cloud planar feature set, updating the merged plane, and thus obtaining the second scene point cloud planar feature set. The step of extracting line features from the model point cloud and the scene point cloud based on the model point cloud planar feature set and the scene point cloud planar feature set to obtain the corresponding model point cloud line feature set and scene point cloud line feature set includes: Based on the intersecting lines between planes in the second model point cloud plane feature set and the intersecting lines between planes in the second scene point cloud plane feature set, respectively, the corresponding model point cloud line feature set and scene point cloud line feature set are determined and extracted.

4. The method according to claim 3, characterized in that, The step of determining and extracting the corresponding model point cloud line feature set and scene point cloud line feature set based on the intersecting lines between planes in the second model point cloud plane feature set and the intersecting lines between planes in the second scene point cloud plane feature set, respectively, includes: Based on the second model point cloud planar feature set, a selected plane is determined. The remaining planes in the second model point cloud planar feature set are traversed to obtain other planes that intersect with the selected plane, thereby obtaining the corresponding intersecting lines. The intersecting lines are determined as model point cloud line features. The model point cloud line feature set of the second model point cloud planar feature set is determined based on the model point cloud line features. Additionally, based on the second scene point cloud planar feature set, a selected plane is determined. The remaining planes in the second scene point cloud planar feature set are traversed to obtain other planes that intersect with the selected plane, thereby obtaining the corresponding intersecting lines. The intersecting lines are determined as scene point cloud line features. The scene point cloud line feature set of the second scene point cloud planar feature set is determined based on the scene point cloud line features.

5. The method according to claim 2, characterized in that, The step of performing feature matching on the model point cloud and the scene point cloud based on the model point cloud planar feature set, the scene point cloud planar feature set, the model point cloud line feature set, and the scene point cloud line feature set to determine the feature matching relationship includes: For each planar feature in the second model point cloud planar feature set, obtain the planar feature that matches it in the second scene point cloud planar feature set; for each model point cloud line feature in the model point cloud line feature set, obtain the scene point cloud line feature that matches it in the scene point cloud line feature set; determine the feature matching relationship based on the matching relationship of planar features and the matching relationship of line features.

6. The method according to claim 5, characterized in that, The step of determining the coordinate transformation relationship between the model point cloud and the scene point cloud based on the feature matching relationship, in order to register the model point cloud and the scene point cloud, includes: An initial coordinate transformation relationship is set. Based on the feature matching relationship, the model point cloud is transformed into the scene point cloud according to the initial coordinate transformation relationship. An error term is constructed in the feature matching relationship according to the error. An objective function is established based on the error term. The objective function is then optimized to determine the optimal coordinate transformation relationship. The model point cloud and the scene point cloud are registered according to the optimal coordinate transformation relationship.

7. The method according to claim 2, characterized in that, After acquiring the 3D model and point cloud of the scene, the process also includes: Based on the spatial size of the model point cloud and the scene point cloud, voxelization is performed on both. Then, planar extraction is performed on the point cloud data in each voxel of the voxelized model point cloud to obtain a first model point cloud planar feature set and point cloud data for each plane. Similarly, planar extraction is performed on the point cloud data in each voxel of the voxelized scene point cloud to obtain a first scene point cloud planar feature set and point cloud data for each plane.

8. A point cloud data registration system, characterized in that, include: The acquisition module is used to acquire a 3D model of the scene and a scene point cloud, and to acquire the model point cloud based on the vertex coordinates of the 3D model, wherein the scene point cloud is a standard point cloud; The first extraction module is used to extract planar features from the model point cloud and the scene point cloud to obtain a first model point cloud planar feature set and a first scene point cloud planar feature set, wherein the first model point cloud planar feature set and the first scene point cloud planar feature set are merged in a plane to obtain a corresponding second model point cloud planar feature set and a second scene point cloud planar feature set. The second extraction module is used to extract line features from the model point cloud and the scene point cloud based on the second model point cloud planar feature set and the second scene point cloud planar feature set, to obtain corresponding model point cloud line feature sets and scene point cloud line feature sets. Specifically, it acquires the intersecting straight lines between two planes in the second model point cloud planar feature set and the second scene point cloud planar feature set, respectively. After acquiring the corresponding intersecting straight lines, it determines whether there are intersecting straight lines between the two planar point clouds based on the actual positions of the planar point clouds. That is, based on the first and second planes corresponding to the intersecting straight lines, it acquires the first and second planar point clouds corresponding to the first and second planes, and acquires the line features from each point in the plane to the corresponding plane. The distance to the intersecting line; traversing each point of the first planar point cloud and each point of the second planar point cloud respectively, if the distance from a point to the intersecting line is less than a set threshold, it is determined that the point is on the intersecting line; otherwise, it is determined that the point is not on the intersecting line; if both the first and second planar point clouds have more than a set number of points on the intersecting line, it is determined that the first and second planar point clouds have an intersecting line; otherwise, it is determined that the first and second planar point clouds do not have an intersecting line; if the two planar point clouds have an intersecting line, the intersecting line is retained and used as a line feature; if the two planar point clouds do not have an intersecting line, the intersecting line is deleted. The matching module performs feature matching on the model point cloud and the scene point cloud based on the model point cloud planar feature set, the scene point cloud planar feature set, the model point cloud line feature set, and the scene point cloud line feature set, and determines the feature matching relationship; The determination module determines the coordinate transformation relationship between the model point cloud and the scene point cloud based on the feature matching relationship, so as to register the model point cloud and the scene point cloud.

9. An electronic device, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the steps of the method as claimed in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, A computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method as described in any one of claims 1 to 7.

Images