A multi-view heterogeneous point cloud registration method and device based on semantic segmentation

Abstract

A multi-view heterogeneous point cloud registration method and device based on semantic segmentation, the method comprising: collecting overhead and ground level view heterogeneous data of a real scene using different sensors, constructing a heterogeneous three-dimensional scene point cloud using a three-dimensional reconstruction algorithm and preprocessing to obtain a training data set by downsampling; importing the data set into a RandLA-Net algorithm for training; using a semantic segmentation algorithm to obtain point cloud semantic labels for different views, extracting specific label point clouds and performing point cloud filtering; performing stereo bounding box on the filtered label point clouds, indirectly obtaining two heterogeneous point cloud transformation matrices between the bounding boxes in two world coordinate systems, and realizing multi-view heterogeneous point cloud registration fusion. The present application can solve the problem of multi-view heterogeneous point cloud registration by using point cloud downsampling, semantic segmentation, point cloud filtering, stereo bounding box and other technologies, and can enhance the robustness of registration and improve the accuracy of registration.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of point cloud registration, and particularly relates to a multi-view heterogeneous point cloud registration method and device based on semantic segmentation, which can be used for registration and fusion of scenes reconstructed by different sensors in a multi-view situation. BACKGROUND

[0002] Point cloud registration aims to correctly register multiple point clouds in the same coordinate system to form a more complete point cloud, which is widely used in fields such as three-dimensional reconstruction, parameter evaluation, positioning and pose estimation, and also involves point cloud registration technology in emerging applications such as autonomous driving, robots and augmented reality.

[0003] According to the acquisition principle of the point cloud to be registered, the point cloud registration problem can be divided into two categories: homogenous point cloud registration and heterogeneous point cloud registration. At present, the research on homogenous point cloud registration is more extensive, and the registration methods based on traditional optimization, feature learning and end-to-end learning are usually used. The registration method based on traditional optimization (ICP algorithm and its optimization variant algorithm) can accurately register two point cloud data with complete correspondence without outliers through the corresponding search and transformation estimation stages. The feature learning method (3DMatch, PPFNet) uses a deep neural network to extract the features of the point cloud and estimates the transformation matrix based on the corresponding relationship of the features or directly regresses from the features; the end-to-end learning method (FMR, PRNet) inputs two point clouds into an end-to-end neural network, and the output is the transformation matrix for aligning the two point clouds; these two learning-based methods can well solve the low overlap rate homogenous point cloud registration. The aerial-view and eye-level view heterogeneous point cloud registration needs to register and fuse the heterogeneous point clouds in different world coordinate systems from different perspectives, such as the sky and the ground. Compared with homogenous point clouds, multi-view heterogeneous point clouds have characteristics such as large outliers, density differences and large rotation poses. In particular, the density difference of heterogeneous point clouds and the increase in the number of point clouds result in poor robustness of homogenous point cloud registration methods and significant performance degradation of the registration method, which makes it difficult to apply to the multi-view heterogeneous three-dimensional point cloud registration problem. Hybrid Fusion is a registration method based on traditional optimization, which uses traditional methods for fine registration based on coarse matching, and solves the precision problem of multi-view heterogeneous point cloud registration, but it is not applicable to multi-view heterogeneous point clouds with large initial poses. In addition, there is currently no large-scale multi-view heterogeneous dataset available, which hinders the development of neural networks in this field. SUMMARY

[0004] In view of the deficiencies of the prior art, the present application provides a multi-view heterogeneous point cloud registration method and device based on semantic segmentation, which aims to solve the problem of multi-view heterogeneous point cloud registration with large initial poses.

[0005] The retraining of the RandLA-Net model in the application obtains the semantic CLASS of a specific object in a large-scene multi-view heterogeneous point cloud, and extracts a specific label point cloud according to the semantic CLASS, filters the specific label point cloud through a vertical ground plane segmentation method and a density clustering filtering algorithm, performs a minimum circumscribed matrix box on the filtered label point cloud, and point clouds the minimum circumscribed matrix box, indirectly obtains the transformation matrix between two heterogeneous point clouds by registering the point cloud minimum circumscribed boxes in two world coordinate systems, and finally realizes multi-view heterogeneous point cloud registration and fusion.

[0006] The first aspect of the application relates to a multi-view heterogeneous point cloud registration method based on semantic segmentation, comprising the following steps:

[0007] Step 1: Collecting an outdoor multi-view heterogeneous point cloud data set to be registered, and pre-processing the data set:

[0008] Step 1-1: Cropping, coordinate system alignment and down-sampling are performed on each point cloud in the data set;

[0009] Step 1-2: Semantic annotation is performed on the pre-processed point cloud data to obtain a Heshan-3D data set;

[0010] Step 1-3: Constructing a RandLA-Net model;

[0011] Step 1-4: Training the RandLA-Net model using the Heshan-3D data set to obtain the parameters and weights of the point cloud samples, and taking the parameters and weights as the parameters and weights of the trained RandLA-Net model;

[0012] Step 1-5: Using the trained RandLA-Net model to perform semantic segmentation on the standardized scene point cloud to be registered, and predicting the semantic class Class of each point in the scene point cloud, defining Class car as a car class, Class building as a house class, and Class tree as a tree class;

[0013] Step 2: Extracting respective class point cloud data clusters according to the semantic segmentation classes Class car , Class building and Class tree ;

[0014] Step 3: Filtering the point cloud data clusters to remove points with semantic classification errors to obtain relatively clean point cloud class clusters:

[0015] Step 3-1: Vertical ground segmentation filtering, RANSAC fitting ground plane and setting appropriate ground plane distance threshold threshold, filtering out all point clouds within the threshold;

[0016] Step 3-2: Using DBSCAN density clustering filtering to screen out clean point cloud class clusters from the wrong classified point cloud clusters;

[0017] Step 4: Making the minimum bounding box for the filtered point cloud class cluster, and point cloud processing the minimum bounding box;

[0018] Step 5: Using pose estimation algorithm to obtain the rotation matrix R and translation matrix t between the two world coordinate system minimum bounding boxes;

[0019] Step 6: The R and t obtained in step 5 constitute the transformation matrix T between the two source point clouds to be registered, and the T obtained by the minimum bounding box indirectly realizes the registration and fusion of the overhead and horizontal point clouds.

[0020] Preferably, the multi-view heterogeneous point cloud to be registered in step 1 is obtained by two different sensors from two angles; multi-view refers to the data collection angle, which is ground level view collection angle and air view collection angle respectively; heterogeneous refers to the sensor used for data collection, which is ground laser radar and RGB camera carried by air unmanned aerial vehicle respectively; the multi-view heterogeneous point cloud to be registered P = {p1, p2,..., p n}, Q = {q1, q2,..., q m} is obtained by sensors, and is obtained by subsequent reconstruction algorithm.

[0021] Preferably, the point cloud filtering method in step 3 is to combine vertical ground plane segmentation and density clustering filtering to screen out points with wrong semantic classification, so as to obtain relatively clean point cloud class clusters; specifically, according to the plane formula: Ax+By+Cz+D=0, at least three points are needed for plane fitting, taking P building as an example, three points are randomly selected from P building to calculate the plane parameters A, B, C and D, and the remaining point cloud coordinates are substituted into the plane formula to calculate the error, which is compared with the threshold error, and if the error is less than the threshold error, the point is considered as a point on the plane, so as to fit the ground plane; according to the distance formula: , the distance between each point outside the plane and the fitted plane is calculated and compared with the set height threshold, and the points within the threshold are considered as noise points with a certain height from the ground plane, and then DBSCAN density clustering is used to remove the remaining noise points with wrong semantic classification, so as to obtain the target point cloud P' building of Class building , and the same method can be used to obtain P'car , P' tree , Q' building , Q' car , Q' tree .

[0022] Preferably, since the collected data is obtained by two different sensors, the density difference brings serious challenges to the pose estimation algorithm, and step 4 is used to solve the challenge, that is, the minimum circumscribed matrix frame is made for the filtered point cloud, and then the circumscribed frame is point clouded, and is respectively represented as P frame ={p1, p2,..., p k}, Q frame ={q1, q2,..., q l}, wherein k, l represent the number of point clouds of the framed point cloud; P frame and Q frame are selected as new point cloud clusters to be matched, and the rotation matrix R * and the translation matrix t * between the framed point clouds can be calculated according to the formula: R * , , wherein N is the number of matching points between P frame and Q frame .

[0023] Preferably, the pose estimation algorithm in step 5 is a 4PCS algorithm.

[0024] Preferably, the transformation matrix T in step 6 is obtained from the rotation matrix R * and the translation matrix t * obtained in step 5, and T can make min, so that the initial point clouds to be matched P={p1, p2,..., p n}, Q={q1, q2,..., q m} are matched and fused.

[0025] In the present application, the two different-view heterogeneous point clouds are P={p1, p2,..., p n}, Q={q1, q2,..., q m}, the category point cloud clusters extracted from the semantic classification results are P tree ={p1, p2,..., p n}, P building ={p1, p2,..., p n}, P car ={p1, p2,..., p n}, Q tree ={q1, q2,..., q m}, Qbuilding = {q1, q2,..., q m}, Q car = {q1, q2,..., q m}, where n, m represent the total number of points in the two point cloud clusters respectively.

[0026] According to the plane formula: Ax + By + Cz + D = 0, plane fitting needs at least three points, taking P building as an example, three points are randomly selected from P building to calculate the plane parameters A, B, C, D, and the remaining point cloud coordinates are substituted into the plane formula to calculate the error, which is then compared with the threshold error. If the error is less than the threshold error, the point is considered to be on the plane, and the ground plane is fitted.

[0027] According to the distance formula: The distance between each point outside the plane and the fitted plane is calculated and compared with the set height threshold. Points within the threshold are considered to be a certain height away from the ground plane. Based on this principle, most of the noise points with incorrect semantic classification can be effectively removed. Then, DBSCAN density clustering is used to remove the remaining noise points with incorrect semantic classification, thereby obtaining the target point cloud P' building Class building , using the same method, P' car , P' tree , Q' building , Q' car , and Q' tree can be obtained.

[0028] To effectively solve the challenge of density difference between heterogeneous point clouds, the point cloud of the minimum bounding box obtained in step 4 is point cloudized, respectively P frame = {p1, p2,..., p k}, Q frame = {q1, q2,..., q l}, where k, l represent the number of point clouds of the point cloudized box.

[0029] P frame and Q frame are selected as new point cloud clusters to be matched, and the rotation matrix R * and translation matrix t * between the point cloudized boxes can be calculated according to the formula: R * , where N is the number of matching points between P frame and Q frame .

[0030] According to the rotation matrix R * and translation matrix t *Can constitute the final pose transformation matrix T can be made Minimum, so that the initial to be registered point cloud P = {p1, p2,..., p n}, Q = {q1, q2,..., q m} registration fusion.

[0031] The second aspect of the application relates to a multi-view heterogeneous point cloud registration device based on semantic segmentation, comprising a memory and one or more processors, the memory has executable code stored therein, and the one or more processors execute the executable code to implement a multi-view heterogeneous point cloud registration method based on semantic segmentation of the application.

[0032] The third aspect of the application relates to a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements a multi-view heterogeneous point cloud registration method based on semantic segmentation of the application.

[0033] Compared with the prior art, the application has the following beneficial effects:

[0034] 1. The multi-view heterogeneous point cloud registration method and device based on semantic segmentation first apply semantic segmentation technology to the registration of overhead-view and flat-view heterogeneous point clouds, and compared with the prior art which takes the entire complete point cloud as the registration object, the application can extract specific objects by using semantic segmentation technology, thereby effectively filtering out background and outlier point clouds and reducing the interference of outlier point cloud noise and other irrelevant point clouds.

[0035] 2. Compared with the overhead-view and flat-view heterogeneous point cloud fine registration based only on coarse matching, the application can effectively register heterogeneous point clouds with large initial poses.

[0036] 3. The application adopts ground plane segmentation and density clustering filtering to effectively filter out misclassified point clouds in semantic segmentation, reduce the error of estimated rigid transformation matrix, and enhance the robustness of registration.

[0037] 4. The application adopts the rotation matrix and translation matrix between the minimum bounding boxes of specific objects after filtering to indirectly implement the registration of overhead-view and flat-view heterogeneous point clouds, which can effectively solve the challenge caused by the density difference of heterogeneous point clouds. At the same time, compared with the prior art which takes the entire point cloud as the processing object, the indirect processing of the minimum bounding box of the specific object after filtering can realize the processing of very few point clouds, greatly reducing the calculation speed of rigid transformation estimation. BRIEF DESCRIPTION OF DRAWINGS

[0038] Figure 1 The flowchart of the method of the application;

[0039] Figure 2The overall framework diagram of the present application;

[0040] Figure 3 The view angle and registration scene diagram used in the present application;

[0041] Figure 4 The original point cloud P diagram of one of the view angles (overhead view angle) of the present application;

[0042] Figure 5 The original point cloud Q diagram of one of the view angles (flat view angle) of the present application;

[0043] Figure 6 The original relative pose relationship diagram of the original point cloud P and the original point cloud Q of the present application;

[0044] Figure 7 The registration fusion completion diagram of the original point cloud P and the original point cloud Q of the present application;

[0045] Figure 8 The registration fusion completion effect diagram of the original point cloud P and the original point cloud Q of the present application.

[0046] Figure 9 The local registration fusion diagram of the control group of the present application.

[0047] Figure 10 The global registration fusion diagram of the control group of the present application. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be described clearly and completely below in combination with the drawings and specific embodiments, but the implementation manner of the present application is not limited thereto.

[0049] Embodiment 1

[0050] This embodiment relates to a ground landscape three-dimensional reconstruction method using a multi-view heterogeneous point cloud registration method based on semantic segmentation of the present application, as shown in the flowchart of the present application, Figure 1 including the input of the flat view-overhead view angle heterogeneous point cloud to be registered, the data set making and preprocessing, obtaining the semantic segmentation result and extracting the point cloud clusters of each category according to the semantic category label, performing vertical ground plane segmentation filtering and density clustering filtering on the extracted point cloud clusters to obtain clean point cloud clusters, performing minimum bounding box on the clean point cloud and estimating the rotation matrix and translation matrix between the boxes, indirectly obtaining the transformation matrix of the point cloud to be registered by using the rotation matrix and translation matrix between the boxes, and finally realizing the flat view-overhead view angle heterogeneous point cloud registration fusion. In order to further facilitate the understanding of the multi-view heterogeneous point cloud registration method, Figure 2 the overall framework of the registration process is shown.

[0051] The specific steps are as follows:

[0052] Step 1: Collecting outdoor multi-view heterogeneous point cloud data sets to be registered, and preprocessing the data sets (such as Figure 3 As shown in the figure, the point clouds to be registered are obtained from the ground laser radar at the eye-level view and the aerial unmanned aerial vehicle with the RBG camera at the top view, respectively.

[0053] Step 1-1: Cropping, coordinate system alignment, and down-sampling are performed on each point cloud in the data set;

[0054] Step 1-2: Semantic labeling is performed on the preprocessed point cloud data to obtain the Heshan-3D data set;

[0055] Step 1-3: Constructing a RandLA-Net model;

[0056] Step 1-4: Training the RandLA-Net model using the Heshan-3D data set to obtain the parameters and weights of the point cloud samples, and taking the parameters and weights as the parameters and weights of the trained RandLA-Net model;

[0057] Step 1-6: Using the trained RandLA-Net model to perform semantic segmentation on the standardized point cloud of the scene to be registered, and predicting the semantic class Class of each point in the scene point cloud, defining Class car as the car class, Class building as the house class, and Class tree as the tree class;

[0058] Step 2: Extracting the respective class point cloud data clusters according to the semantic segmentation classes Class car , Class building , and Class tree ;

[0059] Step 3: Filtering the point cloud data clusters to remove points with semantic classification errors to obtain relatively clean point cloud class clusters:

[0060] Step 3-1: Vertical ground segmentation filtering, RANSAC fitting the ground plane and setting an appropriate ground plane distance threshold threshold, and filtering all point clouds within the threshold;

[0061] Step 3-2: Using DBSCAN density clustering filtering to filter out clean point cloud class clusters from the incorrectly classified point cloud clusters;

[0062] Step 4: Performing minimum bounding box on the filtered point cloud class clusters, and point cloudizing the minimum bounding box;

[0063] Step 5: Use the pose estimation algorithm to obtain the rotation matrix R and translation matrix t between the minimum bounding boxes of the two world coordinate systems;

[0064] Step 6: R and t obtained in step 5 constitute the transformation matrix T between the two heterogeneous point clouds to be registered. T obtained through the minimum bounding box indirectly realizes the registration fusion of the top-down and top-down point clouds.

[0065] Step 7: Use the registration and fusion results of the top-down and top-down point clouds in step 6 to perform 3D reconstruction of the ground landscape.

[0066] In one embodiment of the present invention, steps 1 to 6 include:

[0067] Assume that two different point clouds from different perspectives are P = {p1, p2, ..., p n}( Figure 4 ), Q={q1,q2,...,q m}( Figure 5 ), let the category point cloud cluster extracted by semantic classification be P tree ={p1,p2,...,p n}, P building ={p1,p2,...,p n}, P car ={p1,p2,...,p n}, Q tree ={q1,q2,...,q m}, Q building ={q1,q2,...,q m}, Q car ={q1,q2,...,q m}, where n and m represent the difference in the total number of point clouds in the two point cloud clusters;

[0068] According to the plane formula: Ax+By+Cz+D=0, plane fitting requires at least three points. building For example, from P building Three points are randomly selected to calculate the plane parameters A, B, C, and D. The remaining point cloud coordinates are substituted into the plane formula to calculate the error, which is then compared with the threshold error. If the point error is less than the threshold error, it is considered a point on the plane, thus fitting the ground plane.

[0069] According to the distance formula: The distance between each point outside the plane and the fitting plane is calculated and compared with the set height threshold. The points within the threshold are regarded as noise points at a certain height from the ground plane. According to this principle, most of the noise points with incorrect semantic classification can be effectively removed. Then, DBSCAN density clustering is used to remove the remaining noise points with incorrect semantic classification again, so as to obtain Classbuilding Target point cloud P' of the class building , the same method can be used to obtain P' car , P' tree , Q' building , Q' car , Q' tree ;

[0070] In order to effectively solve the challenge brought by the density difference of heterogeneous point clouds, the minimum bounding box of the point cloud obtained in step 4 is converted into point cloud, which are P frame ={p1,p2,...,p k}, Q frame ={q1,q2,...,q l}, where k and l represent the number of point clouds in the box after point clouding;

[0071] P frame and Q frame Select as a new point cloud cluster to be matched, according to the formula: R * , The rotation matrix R between the point cloud frames can be calculated * and the translation matrix t * , where N is P frame With Q frame Match points between;

[0072] According to the rotation matrix R obtained in step 5 * and the translation matrix t * , which can form the final pose transformation matrix

[0073] Figure 6 is the original relative pose relationship between the original point cloud P and the original point cloud Q, and the pose transformation matrix T can make

[0074] Minimum, to achieve the original point cloud P and the original point cloud Q registration fusion ( Figure 7 ). Figure 8 Schematic diagram of the registration and fusion effect of the original point cloud P and the original point cloud Q of the present invention.

[0075] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Those skilled in the art will understand that the present invention includes, but is not limited to, the drawings and the contents described in the above specific embodiments. Any modifications that do not deviate from the functional and structural principles of the present invention are intended to be included within the scope of the claims.

[0076] In order to verify the effectiveness of the method of the present application, the present application is compared with the method based on traditional optimization (ICP) from the aspects of efficiency and registration accuracy. Since the number of point clouds of the outdoor heterogeneous point cloud data to be registered (original point cloud P and original point cloud Q) is large, the calculation amount of the method based on traditional optimization is too large, which greatly affects the efficiency of the verification work. Therefore, the present application down-samples the original point cloud P and Q. Through experimental demonstration, the present application only needs 7.53516s to complete registration, and the method based on traditional optimization needs 62.7094s to register the point cloud to be registered, and there is a large rotation error and translation error.

[0077]

[0078] Table 1 Registration rotation error and registration time

[0079]

[0080]

[0081] Table 2 Registration translation error

[0082] Note: The index calculation formula is as follows:

[0083] Suppose the transformation matrix is The translation matrix of the transformation matrix is t, the three-axis rotation error of the Ground Truth (GT) of the transformation matrix is ax', ay', az', and the translation matrix t' of the GT

[0084] The rotation angle calculation formula of the three axes is:

[0085] ay = -arcsin (R 31 );

[0086]

[0087] Rotation error calculation formula:

[0088] x-axis rotation error = |ax-ax'|,

[0089] y-axis rotation error = |ay-ay'|,

[0090] z-axis rotation error = |az-az'|;

[0091] Translation error calculation formula: |t-t'|.

[0092] Example 2

[0093] The embodiment relates to a multi-view heterogeneous point cloud registration device based on semantic segmentation, comprising a memory and one or more processors, the memory stores executable codes, and the one or more processors execute the executable codes to implement a multi-view heterogeneous point cloud registration method based on semantic segmentation of the embodiment 1.

[0094] Embodiment 3

[0095] The embodiment relates to a computer readable storage medium, which stores a program, and the program is executed by a processor to implement the multi-view heterogeneous point cloud registration method based on semantic segmentation of the embodiment 1.

[0096] The content described in the embodiments of the present specification is only a list of implementation forms of the inventive concept, and the protection scope of the present application should not be regarded as being limited to the specific forms stated in the embodiments, and the protection scope of the present application also extends to equivalent technical means that can be thought of by those skilled in the art according to the inventive concept.

Claims

1. A multi-view heterogeneous point cloud registration method based on semantic segmentation, characterized in that, Comprising the following steps: Step 1: Collecting outdoor multi-view heterogeneous point cloud data sets to be registered, and preprocessing the data sets; the multi-view heterogeneous point cloud to be registered is a point cloud collected from two different sensors from two viewing angles; Multi-view refers to the data collection viewing angle, which is ground level viewing angle and aerial viewing angle respectively; Step 1-1: cropping, coordinate system alignment and down-sampling each point cloud in the data set; Step 1-2: performing semantic annotation on the preprocessed point cloud data to obtain Heshan-3D data set; Step 1-3: constructing a RandLA-Net model; Step 1-4: training the RandLA-Net model using the Heshan-3D data set to obtain parameters and weights of the point cloud sample, and taking the parameters and weights as the parameters and weights of the trained RandLA-Net model; Step 1-5: Using the trained RandLA-Net model to perform semantic segmentation on the standardized scene point cloud to be registered, predicting the semantic class Class of each point in the scene point cloud, defining Class car as the car category, Class building as the house category, Class tree as the tree category; Step 2: Extracting the respective class point cloud data clusters according to the semantic segmentation class Class car , Class building , Class tree , respectively. Step 3: filtering the point cloud data cluster to screen out points with semantic classification errors to obtain clean point cloud category clusters: Step 3-1: vertical ground segmentation filtering, RANSAC fitting ground plane and setting appropriate ground plane distance threshold threshold, filtering out all point clouds within the threshold; Step 3-2: using DBSCAN density clustering filtering to screen out clean point cloud category clusters from the incorrectly classified point cloud clusters; Step 4: performing minimum bounding box on the filtered point cloud category clusters, and point cloudizing the minimum bounding box; Step 5: using pose estimation algorithm to obtain the rotation matrix R and translation matrix t between the minimum bounding boxes of the two world coordinate systems; Step 6: the R and t obtained in step 5 constitute the transformation matrix T between the two heterogeneous point clouds to be registered, and the T obtained through the minimum bounding box indirectly realizes the registration and fusion of the overhead-view and ground-level-view point clouds.

2. The multi-view heterogeneous point cloud registration method based on semantic segmentation according to claim 1, characterized in that: The heterogeneous in step 1 refers to the sensors used for data acquisition, which are ground laser radar and RGB camera carried by unmanned aerial vehicle respectively; the multi-view heterogeneous point cloud to be registered P = {p1, p2,..., p n} and Q = {q1, q2,..., q m} are obtained by the data of sensors, and then reconstructed by reconstruction algorithm.

3. The multi-view heterogeneous point cloud registration method based on semantic segmentation according to claim 1, characterized in that: The point cloud filtering method described in step 3 is a combination of vertical plane segmentation and density clustering filtering to screen out points with semantic classification errors to obtain a relatively clean point cloud class cluster; specifically, according to the plane formula: Ax+By+Cz+D=0, plane fitting requires at least three points, and three points P building are randomly selected from P building to calculate the plane parameters A, B, C, and D, the remaining point cloud coordinates are sequentially substituted into the plane formula to calculate the error, which is then compared with the threshold error, and if the error is less than the threshold error, the point is considered to be on the plane, thereby fitting the ground plane; according to the distance formula: , the distance of each point outside the plane to the fitted plane is calculated and compared with the set height threshold, and the points within the threshold are considered to be a certain height away from the ground plane, and then DBSCAN density clustering is used to remove the remaining noise points with incorrect semantic classification, thereby obtaining the target point cloud P' building , P' car , P' tree , Q' building , Q' car , and Q' tree .

4. The multi-view heterogeneous point cloud registration method based on semantic segmentation according to claim 1, characterized in that: The data is collected by two different sensors, and the step 4 respectively makes the minimum circumscribed matrix frame for the filtered point cloud, and then makes the circumscribed frame point cloud, and respectively represents as P frame ={p1′,p2′,...,p k ′}, Q frame ={q1′,q2′,...,q l ′}, where k, l represent the point cloud number of the point cloud after the frame; P frame and Q frame are selected as new point cloud clusters to be matched.

5. The multi-view heterogeneous point cloud registration method based on semantic segmentation according to claim 1, characterized in that: The pose estimation algorithm described in step 5 is a 4PCS algorithm, according to the formula: The rotation matrix R between the point clouded frames is calculated * and the translation matrix t * , where N is the number of matching points between P frame and Q frame .

6. The multi-view heterogeneous point cloud registration method based on semantic segmentation according to claim 1, characterized in that: the transformation matrix described in step 6 is the rotation matrix R obtained from step 5 * and the translation matrix t * is obtained such that T minimizes the initial point clouds to be registered P = {p1, p2,..., p n}, Q = {q1, q2,..., q m} are fused by registration.

7. A device for registering multi-view heterogeneous point clouds based on semantic segmentation, characterized in that, The device comprises a memory and one or more processors, the memory stores executable code, and the one or more processors execute the executable code to implement the multi-view heterogeneous point cloud registration method based on semantic segmentation of any one of claims 1-6.

8. A computer-readable storage medium, characterized in that, A program is stored thereon, which is executed by a processor to implement the multi-view heterogeneous point cloud registration method based on semantic segmentation of any one of claims 1-6.

Images