A method for automatic registration of multi-view point clouds for interior scenes of finished houses

Abstract

An automatic multi-view point cloud registration method for finished house interior scenes includes the following steps: 1) Acquiring multi-station scanned point cloud data of the interior scene, and performing filtering and lightweight processing on the multi-station scanned point cloud data to obtain processed multi-station scanned point cloud data; 2) Performing semantic segmentation on the processed multi-station scanned point cloud data to obtain point cloud data of different structures; 3) Calculating the deflection angle of the ceiling edge line, and aligning the different structure point cloud data according to the deflection angle of the ceiling edge line; 4) Performing instance segmentation on the aligned different structure point cloud data to obtain multi-station interior point cloud data; 5) Performing pairwise registration on the multi-station interior point cloud data to obtain coarsely registered point cloud data; 6) Using a pose graph-based optimization-based fine registration algorithm to register the coarsely registered point cloud data, completing the fine registration of all point cloud data. This invention can quickly complete multi-view point cloud registration, greatly improving registration accuracy and efficiency.

Description

Technical Field

[0001] This invention relates to the fields of information technology and building safety management, specifically to a multi-view point cloud automatic registration method for interior scenes of finished houses. Background Technology

[0002] Finished houses require dimensional quality inspection before delivery. Traditional dimensional quality inspection methods rely on workers using measuring tapes and other equipment, which are inefficient and prone to subjective errors. With increasing attention being paid to automated dimensional quality inspection based on terrestrial laser scanning, the importance of automated multi-view registration is growing. Traditional target-based registration methods are inefficient and easily affected by environmental factors. Therefore, a multi-view point cloud automatic registration method for finished house interior scenes is needed to facilitate automated dimensional quality inspection and address the inefficiency of target-based methods.

[0003] In recent years, terrestrial laser scanners have gradually emerged in the field of indoor scene digitization. They can quickly capture 3D point cloud data of a scanned scene, allowing dimensional quality inspection indicators such as surface flatness and spatial dimension estimation to be calculated through data processing. From data acquisition to the generation of a dimensional quality inspection report, the intermediate steps include scan path planning, multi-view point cloud registration, 3D reconstruction, and dimensional quality inspection. Multi-view point cloud registration is the first step in data processing. It involves transforming the coordinate data of scanned point clouds obtained from multiple different sites into the same coordinate system. After registration, the global information of the entire finished building's indoor scene can be obtained and digitally archived, which can then be used for overall construction quality assessment and reverse engineering. In practical applications, multi-view point cloud registration mainly relies on manual targets such as target balls or checkerboard-shaped target paper. This method requires lifting and planning the target placement points, resulting in low efficiency, and the registration accuracy is easily affected by environmental factors. Summary of the Invention

[0004] The purpose of this invention is to provide an automatic registration method for multi-view point clouds in interior scenes of finished houses, including the following steps:

[0005] 1) Acquire multi-station scan point cloud data of indoor scenes, and perform filtering and lightweight processing on the multi-station scan point cloud data to obtain processed multi-station scan point cloud data.

[0006] 2) Perform semantic segmentation on the processed multi-station scan point cloud data to obtain point cloud data with different structures.

[0007] The point cloud data of different structures includes ceiling point cloud data.

[0008] 3) Calculate the deflection angle of the ceiling edge line based on the ceiling point cloud data, and perform axis alignment on the point cloud data of different structures based on the deflection angle of the ceiling edge line.

[0009] 4) Perform instance segmentation on the point cloud data of different structures after axis alignment to obtain instances of multi-station indoor point cloud data.

[0010] 5) Perform pairwise registration on instances of multi-station indoor point cloud data to obtain coarsely registered point cloud data.

[0011] 6) Use a pose graph-based fine registration algorithm to register the coarsely registered point cloud data, and complete the fine registration of all point cloud data.

[0012] Furthermore, the step of semantic segmentation of the processed multi-station scan point cloud data includes:

[0013] 2.1) Establish a dataset of indoor scanned point cloud data for different types of houses, and divide the indoor scanned point cloud data of different types of houses in the dataset into point cloud data with different structures to obtain a training set.

[0014] 2.2) Construct the network model.

[0015] 2.3) Train the network model using the training set to obtain the semantic segmentation network model.

[0016] 2.4) Input the processed multi-station scan point cloud data into the semantic segmentation network model to obtain point cloud data with different structures.

[0017] Furthermore, the point cloud data with different structures also includes ground point cloud data, wall point cloud data, window point cloud data, door point cloud data, and noise point cloud data.

[0018] Furthermore, the step of calculating the deflection angle of the ceiling edge line based on the ceiling point cloud data, and aligning the point cloud data of different structures according to the deflection angle of the ceiling edge line includes:

[0019] 3.1) Cluster the ceiling point cloud data using density clustering algorithm.

[0020] 3.2) Project point cloud data with different structures onto a two-dimensional plane to generate a binary image of the xoy plane.

[0021] 3.3) Detect the edge points of the binary image of the xoy plane and calculate the deflection angle of the ceiling edge line.

[0022] 3.4) Align the point cloud data of different structures according to the deflection angle of the ceiling edge line.

[0023] Furthermore, the step of projecting point cloud data with different structures onto a two-dimensional plane to generate a binary image of the xoy plane includes:

[0024] 3.2.1) Project the point cloud data onto a two-dimensional plane.

[0025] The coordinates (x, y) of the point cloud data projected onto the two-dimensional plane are:

[0026]

[0027]

[0028] In the formula, x img y img These are the x and y coordinates of the point cloud data projected onto a two-dimensional plane, respectively. res represents the resolution.

[0029] 3.2.2) The point cloud data projected onto the two-dimensional plane is translated to obtain the coordinates of the two-dimensional image point cloud, as shown below:

[0030] x img ′=x img -x min -a (3)

[0031] y img ′=y img -y min -b (4)

[0032] In the formula, x img ′、y img '' represents the x-coordinate and y-coordinate in the point cloud coordinates of the two-dimensional image, respectively. min y min The two-dimensional image coordinates x and x are respectively img y img The minimum value in the range. a and b are both preset values.

[0033] 3.2.3) Set preset values ​​a and b, and project the point cloud coordinates onto the middle position (imx, imy) of the image, that is:

[0034] imx = max(x) img ′)+2a (5)

[0035] imy = max(y img ′)+2b (6)

[0036] In the formula, max(x) img ′), max(y img ′) are the coordinates of the point cloud in the two-dimensional image, x and y respectively. img ′、y img The maximum value in '.

[0037] 3.2.4) Set the pixel at the coordinate of the two-dimensional image point cloud in the two-dimensional plane as g, and set the other locations as k to obtain the binary image in the xoy plane, where g and k are both integers.

[0038] Furthermore, the step of detecting edge points in the xoy plane binary image and calculating the deflection angle of the ceiling edge line includes:

[0039] 3.3.1) Dilate the xoy plane binary image using a morphological dilation algorithm to obtain a binary image.

[0040] The formula for calculating the dilation of the xoy plane binary image by the morphological dilation algorithm is as follows:

[0041]

[0042] In the formula, B is a sliding window. A is a binary image in the xoy plane. (x,y) are the point cloud coordinates of the two-dimensional image.

[0043] If the intersection of the sliding window B at the coordinates (x,y) of the two-dimensional image point cloud and the binary image A in the xoy plane is not empty, the corresponding coordinates (x,y) of the output binary image are assigned a value of 1; otherwise, the value is 0.

[0044] 3.3.2) Calculate the gradient at each point (x, y) in the binary image sequentially to obtain the edge image, as shown below:

[0045]

[0046] In the formula, G is the gradient of the binary image.

[0047] Wherein, the horizontal convolution matrix is ​​convolved with the binary image by the convolution function G. x The convolution function G of the vertical convolution matrix and the binary image. y As shown below:

[0048]

[0049]

[0050] In the formula, I is a binary image.

[0051] 3.3.3) Use a plane fitting algorithm to remove the background point cloud from the edge image to obtain the edge points.

[0052] 3.3.4) Transform the edge points to the parameter space and detect the edge lines through HOUGH transformation.

[0053] The HOUGH transformation is shown below:

[0054] rho=x*cos(theta)+y*sin(theta) (11)

[0055] In the formula, rho is the polar radius, and theta is the deflection angle of the ceiling edge line.

[0056] Furthermore, the calculation formula for the axis-aligned rotation transformation of point cloud data with different structures is as follows:

[0057] p′=Rp (12)

[0058] In the formula, p and p′ are the coordinates before and after the rotation transformation, respectively.

[0059] The rotation transformation matrix R is shown below:

[0060]

[0061] In the formula, theta is the deflection angle of the ceiling edge line.

[0062] Furthermore, the step of segmenting the point cloud data of different structures after axis alignment to obtain instances of multi-station indoor point cloud data includes:

[0063] 4.1) Use a plane fitting algorithm to fit the ceiling plane and the wall plane.

[0064] 4.2) Determine the normal vector of the wall plane based on its orientation and relative position to the scanning origin.

[0065] The scanning origin is the base point for aligning point cloud data with different structures.

[0066] 4.3) Classify the wall planes according to their normal vectors, and use density clustering algorithm to extract individual wall planes in each class.

[0067] 4.4) Statistical analysis of the neighborhood of the window point cloud data is performed using a statistical filter to obtain noise-removed window point cloud data.

[0068] 4.5) Use density clustering algorithm to extract individual windows.

[0069] 4.6) Use a support plane-based segmentation algorithm to segment the gate point cloud data.

[0070] Furthermore, the step of performing pairwise registration of instances of multi-station indoor point cloud data to obtain coarsely registered point cloud data includes:

[0071] 5.1) The pairwise registration algorithm based on the large plane and the pairwise registration algorithm based on the gate register the indoor point cloud data of each station with the subsequent point cloud data in sequence to obtain the corresponding registration score and transformation matrix.

[0072] The ceiling point cloud data is used as the source point cloud data, and the wall plane point cloud data adjacent to the ceiling point cloud data is used as the target point cloud data.

[0073] The registration score is shown below:

[0074] score=∑ obj∈{LP,w,d} overlap obj -∑ form∈{h,v} ω form *conflict form (14)

[0075] In the formula, obj is an element of the set {LP, w, d}. form is an element of the set {h, v}. overlap LP overlap w overlap d conflict h conflict v These are the evaluation functions for the large plane, window, door, horizontal direction, and vertical direction, respectively. ω form Represents the weight, ω h Represents the horizontal conflict weight, ω v This represents the vertical conflict weight.

[0076] Among them, the evaluation function overlap based on the pairwise registration algorithm of the large plane is... obj The calculation formula is as follows:

[0077]

[0078] In the formula, N overlap N represents the total number of overlapping points. tar N represents the number of points in the target point cloud. sor The number of points in the source point cloud.

[0079] The evaluation function of the gate-based pairwise registration algorithm is overlap. obj The calculation formula is as follows:

[0080]

[0081] 5.2) Determine whether the registration score is greater than the preset threshold. If so, define that the two stations corresponding to the registration score have a registration relationship.

[0082] 5.3) Define each station as a node. The registration relationship between two nodes represents an edge between the two nodes. The weight of the edge is the registration score between the two stations, resulting in one or more connected graphs.

[0083] 5.4) For each connected graph, the edges are pruned using the maximum spanning tree algorithm, and the node with the maximum proximity centrality is designated as the registration reference station. Other stations are transformed to the local coordinate system of the registration reference station according to the transformation matrix obtained by the shortest transformation path, thus obtaining coarse registration point cloud data.

[0084] The formula for calculating the proximity centrality is as follows:

[0085]

[0086] In the formula, f and u are nodes. C(u) is the closeness centrality of node u. d(f,u) represents the shortest distance between node u and node f in the connected graph. n-1 represents the number of nodes that node u can reach.

[0087] Furthermore, the step of registering the coarsely registered point cloud data using a pose graph-based optimization fine registration algorithm to complete the fine registration of all point cloud data includes:

[0088] 6.1) Using the coarse registration point cloud data of the first station as the fine registration reference station, the coarse registration point cloud data of each station is sequentially registered with the coarse registration point cloud data after the fine registration reference station through the ICP algorithm.

[0089] 6.2) Fine registration of adjacent stations generates adjacent edges, and fine registration of non-adjacent stations generates loop edges. Each edge corresponds to a fine registration score and a transformation matrix.

[0090] 6.3) Determine whether the fine registration score is greater than the preset threshold. If not, define the edge corresponding to the fine registration score as an invalid edge and exclude the invalid edge to obtain the registration adjacency relationship graph.

[0091] 6.4) Remove loop edges in the registration adjacency graph that contain incorrect registration relationships to obtain a finely registered adjacency graph.

[0092] 6.5) Based on the least squares optimization algorithm, the pose graph of the fine registration adjacency relationship graph is optimized to complete the fine registration of all point cloud data.

[0093] The technical effects of this invention are undeniable. The multi-view point cloud automatic registration method for finished house interior scenes disclosed in this invention can automatically register multi-view point clouds of finished house interior scenes. It can use a computer to calculate the transformation parameters of each scanning station to complete multi-view point cloud registration. This invention eliminates the need for manual target marking and can quickly complete multi-view point cloud registration, greatly improving registration accuracy and efficiency.

[0094] To address the requirement of targetless registration, this invention proposes an automatic multi-view point cloud registration method for finished house interior scenes. It uses a terrestrial laser scanner to perform rapid scanning at multiple locations inside the house and obtains global information of the finished house interior through a targetless multi-view point cloud registration algorithm. Attached Figure Description

[0095] Figure 1 This invention discloses a flowchart of an automated point cloud registration method for interior scenes of finished houses from multiple perspectives;

[0096] Figure 2 A schematic diagram of land laser scanning as an example of an embodiment of the present invention;

[0097] Figure 3 The result of radius filtering and lightweighting of scanned point cloud data for an embodiment of the present invention;

[0098] Figure 4 The result of semantic segmentation of the lightweight scan data used in an embodiment of the present invention;

[0099] Figure 5 The scanned point cloud data axis alignment result is an example of an embodiment of the present invention;

[0100] Figure 6 The vertical and horizontal conflict bounding boxes are illustrated as examples of embodiments of the present invention. Figure 6 (a) is a rendering of the vertical conflict bounding box; Figure 6 (b) is a rendering of the horizontal conflict bounding box;

[0101] Figure 7 A flowchart illustrating the pairwise registration process based on gate point cloud data, serving as an example of an implementation of this invention;

[0102] Figure 8 For the first set of finished house point cloud data registration in the embodiment of the present invention, the top and bottom views and cross-sections are obtained; Figure 8 (a) A top view of the first set of finished house point cloud data before registration; Figure 8 (b) A top view of the first set of finished house point cloud data after registration; Figure 8 (c) is a cross-section of the first set of finished house point cloud data after registration;

[0103] Figure 9 The second set of finished house point cloud data registration for the present invention includes front and rear top views and cross-sections; Figure 9 (a) A top view of the second set of finished house point cloud data before registration; Figure 9 (b) A top view of the second set of finished house point cloud data after registration; Figure 9 (c) is a cross-section of the second set of finished house point cloud data after registration;

[0104] Figure 10 The third set of finished house point cloud data registration front and back top views and cross-sections are examples of embodiments of the present invention. Figure 10 (a) Top view of the third set of finished house point cloud data before registration; Figure 10 (b) A top view of the third set of finished house point cloud data after registration; Figure 10 (c) is a cross-section of the point cloud data of the third set of finished houses after registration;

[0105] Figure 11 The fourth set of finished house point cloud data registration front and back top views and cross-sections are examples of embodiments of the present invention. Figure 11 (a) A top view of the fourth set of finished house point cloud data before registration; Figure 11 (b) A top view of the fourth set of finished house point cloud data after registration; Figure 11 (c) is a cross-section of the fourth set of finished house point cloud data after registration;

[0106] Figure 12 The fifth set of finished house point cloud data registration front and back top views and cross-sections are examples of embodiments of the present invention. Figure 12 (a) A top view of the fifth set of finished house point cloud data before registration; Figure 12 (b) A top view of the fifth set of finished houses after point cloud data registration; Figure 12 (c) is a cross-section of the fifth set of finished house point cloud data after registration. Detailed Implementation

[0107] The present invention will be further described below with reference to embodiments, but it should not be construed that the scope of the present invention is limited to the following embodiments. Various substitutions and modifications made based on ordinary technical knowledge and common practices in the art without departing from the above-described technical concept of the present invention should be included within the scope of protection of the present invention.

[0108] Example 1:

[0109] See Figures 1 to 12 An automatic registration method for multi-view point clouds in finished house interior scenes includes the following steps:

[0110] 1) Acquire multi-station scan point cloud data of indoor scenes, and perform filtering and lightweight processing on the multi-station scan point cloud data to obtain processed multi-station scan point cloud data.

[0111] 2) Perform semantic segmentation on the processed multi-station scan point cloud data to obtain point cloud data with different structures.

[0112] The point cloud data of different structures includes ceiling point cloud data.

[0113] 3) Calculate the deflection angle of the ceiling edge line based on the ceiling point cloud data, and perform axis alignment on the point cloud data of different structures based on the deflection angle of the ceiling edge line.

[0114] 4) Perform instance segmentation on the point cloud data of different structures after axis alignment to obtain instances of multi-station indoor point cloud data.

[0115] 5) Perform pairwise registration on instances of multi-station indoor point cloud data to obtain coarsely registered point cloud data.

[0116] 6) Use a pose graph-based fine registration algorithm to register the coarsely registered point cloud data, and complete the fine registration of all point cloud data.

[0117] Example 2:

[0118] A method for automatic registration of multi-view point clouds for interior scenes of finished houses, the main technical content of which is described in Example 1, further includes the step of semantic segmentation of the processed multi-station scan point cloud data, which includes:

[0119] 2.1) Establish a dataset of indoor scanned point cloud data for different types of houses, and divide the indoor scanned point cloud data of different types of houses in the dataset into point cloud data with different structures to obtain a training set.

[0120] 2.2) Construct the network model.

[0121] 2.3) Train the network model using the training set to obtain the semantic segmentation network model.

[0122] 2.4) Input the processed multi-station scan point cloud data into the semantic segmentation network model to obtain point cloud data with different structures.

[0123] Example 3:

[0124] A method for automatic registration of multi-view point clouds for interior scenes of finished houses, the main technical contents of which are described in any one of Embodiments 1 to 2. Furthermore, the point cloud data of different structures also includes ground point cloud data, wall point cloud data, window point cloud data, door point cloud data, and noise point cloud data.

[0125] Example 4:

[0126] A method for automatic registration of multi-view point clouds for interior scenes of finished houses, the main technical contents of which are described in any one of embodiments 1 to 3, further comprising the steps of calculating the deflection angle of the ceiling edge line based on the ceiling point cloud data, and aligning the point cloud data of different structures based on the deflection angle of the ceiling edge line, including:

[0127] 3.1) Cluster the ceiling point cloud data using density clustering algorithm.

[0128] 3.2) Project point cloud data with different structures onto a two-dimensional plane to generate a binary image of the xoy plane.

[0129] 3.3) Detect the edge points of the binary image of the xoy plane and calculate the deflection angle of the ceiling edge line.

[0130] 3.4) Align the point cloud data of different structures according to the deflection angle of the ceiling edge line.

[0131] Example 5:

[0132] A multi-view point cloud automatic registration method for interior scenes of finished houses, the main technical contents of which are described in any one of embodiments 1 to 4, further comprising the step of projecting point cloud data of different structures onto a two-dimensional plane to generate an xoy plane binary image, including:

[0133] 3.2.1) Project the point cloud data onto a two-dimensional plane.

[0134] The coordinates (x, y) of the point cloud data projected onto the two-dimensional plane are:

[0135]

[0136]

[0137] In the formula, x img y img These are the x and y coordinates of the point cloud data projected onto a two-dimensional plane, respectively. res represents the resolution.

[0138] 3.2.2) The point cloud data projected onto the two-dimensional plane is translated to obtain the coordinates of the two-dimensional image point cloud, as shown below:

[0139] x img ′=x img -x min -a (3)

[0140] y img ′=y img -y min -b (4)

[0141] In the formula, x img ′、y img '' represents the x-coordinate and y-coordinate in the point cloud coordinates of the two-dimensional image, respectively. min y min The two-dimensional image coordinates x and x are respectively img y img The minimum value in the range. a and b are both preset values.

[0142] 3.2.3) Set preset values ​​a and b, and project the point cloud coordinates onto the middle position (imx, imy) of the image, that is:

[0143] imx = max(x) img ′)+2a (5)

[0144] imy = max(y img ′)+2b (6)

[0145] In the formula, max(x) img ′), max(y img ′) are the coordinates of the point cloud in the two-dimensional image, x and y respectively. img ′、y img The maximum value in '.

[0146] 3.2.4) Set the pixel at the coordinate of the two-dimensional image point cloud in the two-dimensional plane as g, and set the other locations as k to obtain the binary image in the xoy plane, where g and k are both integers.

[0147] Example 6:

[0148] A multi-view point cloud automatic registration method for interior scenes of finished houses, the main technical contents of which are described in any one of embodiments 1 to 5, further comprising the step of detecting edge points of the xoy plane binary image and calculating the deflection angle of the ceiling edge line, including:

[0149] 3.3.1) Dilate the xoy plane binary image using a morphological dilation algorithm to obtain a binary image.

[0150] The formula for calculating the dilation of the xoy plane binary image by the morphological dilation algorithm is as follows:

[0151]

[0152] In the formula, B is a sliding window. A is a binary image in the xoy plane. (x,y) are the point cloud coordinates of the two-dimensional image.

[0153] If the intersection of the sliding window B at the coordinates (x,y) of the two-dimensional image point cloud and the binary image A in the xoy plane is not empty, the corresponding coordinates (x,y) of the output binary image are assigned a value of 1; otherwise, the value is 0.

[0154] 3.3.2) Calculate the gradient at each point (x, y) in the binary image sequentially to obtain the edge image, as shown below:

[0155]

[0156] In the formula, G is the gradient of the binary image.

[0157] Wherein, the horizontal convolution matrix is ​​convolved with the binary image by the convolution function G.x The convolution function G of the vertical convolution matrix and the binary image. y As shown below:

[0158]

[0159]

[0160] In the formula, I is a binary image.

[0161] 3.3.3) Use a plane fitting algorithm to remove the background point cloud from the edge image to obtain the edge points.

[0162] 3.3.4) Transform the edge points to the parameter space and detect the edge lines through HOUGH transformation.

[0163] The HOUGH transformation is shown below:

[0164] rho=x*cos(theta)+y*sin(theta) (11)

[0165] In the formula, rho is the polar radius, and theta is the deflection angle of the ceiling edge line.

[0166] Example 7:

[0167] A method for automatic registration of multi-view point clouds for interior scenes of finished houses, the main technical contents of which are described in any one of Embodiments 1 to 6. Furthermore, the calculation formula for the rotation transformation of point cloud data with different structures for axis alignment is as follows:

[0168] p′=Rp (12)

[0169] In the formula, p and p′ are the coordinates before and after the rotation transformation, respectively.

[0170] The rotation transformation matrix R is shown below:

[0171]

[0172] In the formula, theta is the deflection angle of the ceiling edge line.

[0173] Example 8:

[0174] A method for automatic registration of multi-view point clouds for interior scenes of finished houses, the main technical contents of which are described in any one of embodiments 1 to 7, further comprising the step of segmenting the point cloud data of different structures after axis alignment to obtain instances of multi-station indoor point cloud data, including:

[0175] 4.1) Use a plane fitting algorithm to fit the ceiling plane and the wall plane.

[0176] 4.2) Determine the normal vector of the wall plane based on its orientation and relative position to the scanning origin.

[0177] The scanning origin is the base point for aligning point cloud data with different structures.

[0178] 4.3) Classify the wall planes according to their normal vectors, and use density clustering algorithm to extract individual wall planes in each class.

[0179] 4.4) Statistical analysis of the neighborhood of the window point cloud data is performed using a statistical filter to obtain noise-removed window point cloud data.

[0180] 4.5) Use density clustering algorithm to extract individual windows.

[0181] 4.6) Use a support plane-based segmentation algorithm to segment the gate point cloud data.

[0182] Example 9:

[0183] A method for automatic registration of multi-view point clouds for interior scenes of finished houses, the main technical contents of which are described in any one of embodiments 1 to 8, further comprising the step of performing pairwise registration of instances of multi-station indoor point cloud data to obtain coarsely registered point cloud data, including:

[0184] 5.1) The pairwise registration algorithm based on the large plane and the pairwise registration algorithm based on the gate register the indoor point cloud data of each station with the subsequent point cloud data in sequence to obtain the corresponding registration score and transformation matrix.

[0185] The ceiling point cloud data is used as the source point cloud data, and the wall plane point cloud data adjacent to the ceiling point cloud data is used as the target point cloud data.

[0186] The registration score is shown below:

[0187] score=∑ obj∈{LP,w,d} overlap obj -∑ form∈{h,v} ω form *conflict form (14)

[0188] In the formula, obj is an element of the set {LP, w, d}. form is an element of the set {h, v}. overlap LP overlap w overlap d conflict h conflict v These are the evaluation functions for the large plane, window, door, horizontal direction, and vertical direction, respectively. ωform Represents the weight, ω h Represents the horizontal conflict weight, ω v This represents the vertical conflict weight.

[0189] Among them, the evaluation function overlap based on the pairwise registration algorithm of the large plane is... obj The calculation formula is as follows:

[0190]

[0191] In the formula, N overlap N represents the total number of overlapping points. tar N represents the number of points in the target point cloud. sor The number of points in the source point cloud.

[0192] The evaluation function of the gate-based pairwise registration algorithm is overlap. obj The calculation formula is as follows:

[0193]

[0194] 5.2) Determine whether the registration score is greater than the preset threshold. If so, define that the two stations corresponding to the registration score have a registration relationship.

[0195] 5.3) Define each station as a node. The registration relationship between two nodes represents an edge between the two nodes. The weight of the edge is the registration score between the two stations, resulting in one or more connected graphs.

[0196] 5.4) For each connected graph, the edges are pruned using the maximum spanning tree algorithm, and the node with the maximum proximity centrality is designated as the registration reference station. Other stations are transformed to the local coordinate system of the registration reference station according to the transformation matrix obtained by the shortest transformation path, thus obtaining coarse registration point cloud data.

[0197] The formula for calculating the proximity centrality is as follows:

[0198]

[0199] In the formula, f and u are nodes. C(u) is the closeness centrality of node u. d(f,u) represents the shortest distance between node u and node f in the connected graph. n-1 represents the number of nodes that node u can reach.

[0200] Example 10:

[0201] A multi-view point cloud automatic registration method for interior scenes of finished houses, the main technical contents of which are described in any one of embodiments 1 to 9, further comprising the step of registering the coarsely registered point cloud data using a pose graph-based optimization fine registration algorithm to complete the fine registration of all point cloud data, including:

[0202] 6.1) Using the coarse registration point cloud data of the first station as the fine registration reference station, the coarse registration point cloud data of each station is sequentially registered with the coarse registration point cloud data after the fine registration reference station through the ICP algorithm.

[0203] 6.2) Fine registration of adjacent stations generates adjacent edges, and fine registration of non-adjacent stations generates loop edges. Each edge corresponds to a fine registration score and a transformation matrix.

[0204] 6.3) Determine whether the fine registration score is greater than the preset threshold. If not, define the edge corresponding to the fine registration score as an invalid edge and exclude the invalid edge to obtain the registration adjacency relationship graph.

[0205] 6.4) Remove loop edges in the registration adjacency graph that contain incorrect registration relationships to obtain a finely registered adjacency graph.

[0206] 6.5) Based on the least squares optimization algorithm, the pose graph of the fine registration adjacency relationship graph is optimized to complete the fine registration of all point cloud data.

[0207] Example 11:

[0208] See Figures 1 to 12 An automatic registration method for multi-view point clouds in finished house interior scenes includes the following steps:

[0209] 1) Use a 3D laser scanner to acquire multi-station scan point cloud data of an indoor scene, and filter and lightweight process the multi-station scan point cloud data to obtain processed multi-station scan point cloud data.

[0210] The collected data may contain a large number of outdoor points in the point cloud, so we choose to filter out the scan points that are more than a preset range away from the scan center point; because the data volume of a single station is too large, we perform lightweight processing.

[0211] 2) Perform semantic segmentation on the processed multi-station scan point cloud data to obtain point cloud data with different structures.

[0212] The point cloud data of different structures includes ceiling point cloud data.

[0213] 3) Calculate the deflection angle of the ceiling edge line based on the ceiling point cloud data, and perform axis alignment on the point cloud data of different structures based on the deflection angle of the ceiling edge line.

[0214] 4) Perform instance segmentation on the point cloud data of different structures after axis alignment to obtain instances of multi-station indoor point cloud data.

[0215] 5) Perform pairwise registration on instances of multi-station indoor point cloud data to obtain coarsely registered point cloud data.

[0216] 6) Use a pose graph-based fine registration algorithm to register the coarsely registered point cloud data, and complete the fine registration of all point cloud data.

[0217] Example 12:

[0218] A method for automatic registration of multi-view point clouds for interior scenes of finished houses, the main technical content of which is described in Example 11, further includes the step of semantic segmentation of the processed multi-station scan point cloud data, which includes:

[0219] 2.1) Establish a dataset of indoor scanned point cloud data for different types of houses, and divide the indoor scanned point cloud data of different types of houses in the dataset into point cloud data with different structures to obtain a training set.

[0220] 2.2) Construct the network model.

[0221] 2.3) Train the network model using the training set to obtain the semantic segmentation network model.

[0222] 2.4) Input the processed multi-station scan point cloud data into the semantic segmentation network model to obtain point cloud data with different structures.

[0223] A point cloud semantic segmentation neural network was used to perform semantic segmentation on the point cloud data. To train the network model, a dataset was created from indoor scanned point cloud data of multiple houses with different floor plans. The scanned data for each station was manually segmented into point cloud data for the ceiling, floor, walls, windows, doors, and noise. Finally, the trained network model was used to perform semantic segmentation on the scanned point cloud data.

[0224] Example 13:

[0225] A method for automatic registration of multi-view point clouds for interior scenes of finished houses, the main technical contents of which are described in any one of embodiments 11 to 12. Furthermore, the point cloud data of different structures also includes ground point cloud data, wall point cloud data, window point cloud data, door point cloud data, and noise point cloud data.

[0226] Example 14:

[0227] A method for automatic registration of multi-view point clouds for interior scenes of finished houses, the main technical contents of which are described in any one of embodiments 11 to 13, further comprising the step of calculating the deflection angle of the ceiling edge line based on the ceiling point cloud data, and aligning the point cloud data of different structures according to the deflection angle of the ceiling edge line, including:

[0228] 3.1) Use density clustering algorithm to cluster the ceiling point cloud data, and determine whether the number of points in the ε neighborhood of each point in the ceiling point cloud data exceeds the preset threshold. If so, define these points as core points.

[0229] The basic idea of ​​the algorithm is to divide the original point cloud by continuously querying unvisited core points and classifying the points in the ε-neighborhood of the core points into one category.

[0230] 3.2) Project point cloud data with different structures onto a two-dimensional plane to generate a binary image of the xoy plane.

[0231] 3.3) Detect the edge points of the binary image of the xoy plane and calculate the deflection angle of the ceiling edge line.

[0232] 3.4) Align the point cloud data of different structures according to the deflection angle of the ceiling edge line.

[0233] The ceiling point cloud data is clustered using a density clustering algorithm. The point cloud data is then projected to generate an xoy plane binary image. The ceiling edge lines are extracted using a morphological dilation algorithm, an image edge detection algorithm, and a line detection algorithm.

[0234] Example 15:

[0235] A method for automatic registration of multi-view point clouds for interior scenes of finished houses, the main technical contents of which are described in any one of embodiments 11 to 14, further comprising the step of projecting point cloud data of different structures onto a two-dimensional plane to generate an xoy plane binary image, including:

[0236] 3.2.1) Project the point cloud data onto a two-dimensional plane.

[0237] The coordinates (x, y) of the point cloud data projected onto the two-dimensional plane are:

[0238]

[0239]

[0240] In the formula, x img y img These are the x and y coordinates of the point cloud data projected onto a two-dimensional plane, respectively. res represents the resolution.

[0241] 3.2.2) To avoid negative image coordinates, the point cloud data projected onto the two-dimensional plane is translated to obtain the two-dimensional image point cloud coordinates, as shown below:

[0242] x img ′=x img -x min -a (3)

[0243] y img ′=y img -y min -b (4)

[0244] In the formula, x img ′、y img '' represents the x-coordinate and y-coordinate in the point cloud coordinates of the two-dimensional image, respectively. min y min The two-dimensional image coordinates x and x are respectively img y img The minimum value in the image. a and b are preset values, used to project the point cloud to a suitable central position in the image.

[0245] 3.2.3) Set preset values ​​a and b, and project the point cloud coordinates onto the middle position (imx, imy) of the image, that is:

[0246] imx = max(x) img ′)+2a (5)

[0247] imy = max(y img ′)+2b (6)

[0248] In the formula, max(x) img ′), max(y img ′) are the coordinates of the point cloud in the two-dimensional image, x and y respectively. img ′、y img The maximum value in '.

[0249] 3.2.4) Set the pixel at the coordinate of the two-dimensional image point cloud in the two-dimensional plane to 255, and set it to 0 elsewhere to obtain the binary image of the xoy plane.

[0250] Example 16:

[0251] A multi-view point cloud automatic registration method for interior scenes of finished houses, the main technical contents of which are described in any one of embodiments 11 to 15, further comprising the step of detecting edge points of the xoy plane binary image and calculating the deflection angle of the ceiling edge line, including:

[0252] 3.3.1) Dilate the xoy plane binary image using a morphological dilation algorithm to obtain a binary image.

[0253] The formula for calculating the dilation of the xoy plane binary image by the morphological dilation algorithm is as follows:

[0254]

[0255] In the formula, B is a sliding window. A is a binary image in the xoy plane. (x,y) are the point cloud coordinates of the two-dimensional image.

[0256] If the intersection of the sliding window B at the coordinates (x,y) of the two-dimensional image point cloud and the binary image A in the xoy plane is not empty, the corresponding coordinates (x,y) of the output binary image are assigned a value of 1; otherwise, the value is 0.

[0257] 3.3.2) Calculate the gradient at each point (x, y) in the binary image sequentially to obtain the edge image, as shown below:

[0258]

[0259] In the formula, G is the gradient of the binary image.

[0260] Wherein, the horizontal convolution matrix is ​​convolved with the binary image by the convolution function G. x The convolution function G of the vertical convolution matrix and the binary image. y As shown below:

[0261]

[0262]

[0263] In the formula, I is a binary image.

[0264] 3.3.3) Use a plane fitting algorithm to remove the background point cloud from the edge image to obtain the edge points.

[0265] Background point cloud removal is achieved using a plane fitting algorithm. The algorithm assumes a model with at least D data points and a dataset much larger than D. T data points are randomly selected from the dataset to fit the model. Based on a predefined tolerance, data points that satisfy the model are called in-circle points, and those that do not are called outliers. This process is iterative. The effectiveness of the model depends on whether the number of in-circle points is sufficient; in-circle point data from effective models are retained for further processing.

[0266] 3.3.4) Transform the edge points to the parameter space and detect the edge lines through HOUGH transformation.

[0267] Line detection algorithms use the Hough transform to transform the image coordinate space to a parameter space. All lines intersecting at the same point in the parameter space have corresponding collinear points in the image coordinate space. Based on this property, given some edge points in the image coordinate space, the equations of the lines connecting these points can be determined using the Hough transform. In the image xy coordinate system, the line passing through the point (x...)... i ,y i The straight line y = kx i +b transforms to the parameter space kb:

[0268] b = -kx i +y

[0269] It is evident that points on a straight line in the xy coordinate space will intersect at a single point in the kb parameter space. For each edge point in the image, a straight line can be drawn in the kb space. At the intersection of these lines, a "vote" method is used. A two-dimensional accumulation array A(k,b) is established, which represents the intersection point (k,b) of n straight lines. i ,b i ) has A(k i ,b i =n, the threshold n0 for line detection detects lines that satisfy A(k) i ,b i () = All lines where n > n0.

[0270] The HOUGH transformation is shown below:

[0271] rho = x img ′*cos(theta)+y img ′*sin(theta) (11)

[0272] In the formula, rho is the polar radius, and theta is the deflection angle of the ceiling edge line.

[0273] Example 17:

[0274] A multi-view point cloud automatic registration method for finished house interior scenes, the main technical contents of which are described in any one of embodiments 11 to 16. Further, the output parameters of the edge lines detected by the line detection algorithm are the polar radius rho and the polar angle theta. The rotation angle of the axis-aligned point cloud data is θ, which is related to the polar angle theta by θ = theta. Let the coordinate values ​​of the point cloud data before the affine transformation in coordinate system p be (x, y), and take the scanning origin (0, 0) as the base point. The coordinate values ​​of the point cloud data after the rotation transformation in coordinate system p′ are p′. The calculation formula for the axis-aligned rotation transformation of point cloud data with different structures is as follows:

[0275] p′=Rp (12)

[0276] In the formula, p and p′ are the coordinates before and after the rotation transformation, respectively.

[0277] The rotation transformation matrix R is shown below:

[0278]

[0279] In the formula, theta is the deflection angle of the ceiling edge line.

[0280] Example 18:

[0281] A method for automatic registration of multi-view point clouds for interior scenes of finished houses, the main technical contents of which are described in any one of embodiments 11 to 17, further comprising the step of segmenting the point cloud data of different structures after axis alignment to obtain instances of multi-station indoor point cloud data, including:

[0282] 4.1) Use a plane fitting algorithm to fit the ceiling plane and the wall plane.

[0283] 4.2) Determine the normal vector of the wall plane based on its orientation and relative position to the scanning origin.

[0284] The scanning origin is the base point for aligning point cloud data with different structures.

[0285] 4.3) Classify the wall planes according to their normal vectors, and use density clustering algorithm to extract individual wall planes in each class.

[0286] 4.4) Statistical analysis of the neighborhood of the window point cloud data is performed using a statistical filter to obtain noise-removed window point cloud data.

[0287] 4.5) Use density clustering algorithm to extract individual windows.

[0288] 4.6) Use a support plane-based segmentation algorithm to segment the gate point cloud data.

[0289] The window data obtained from semantic segmentation may contain some sparse noise that affects post-processing. Statistical filters are used to perform statistical analysis on the neighborhood of each point, filtering out some outliers that do not meet the requirements based on the distance distribution characteristics from the point to all its neighbors.

[0290] In the instance segmentation process of wall point cloud data and door and window point cloud data, one of the characteristics of these instances is their normal vector. The normal vector is determined by the orientation of these instances and their relative position to the scanning origin. Instance segmentation of door point cloud data uses a support plane-based segmentation algorithm, the principle of which is as follows: traverse the wall point cloud data, construct an axis-aligned bounding box for each wall point cloud data piece, expand the bounding box along the normal vector direction away from the scanning origin, and extract the door point cloud data within the bounding box.

[0291] Example 19:

[0292] A method for automatic registration of multi-view point clouds for interior scenes of finished houses, the main technical contents of which are described in any one of embodiments 11 to 18, further comprising the step of performing pairwise registration of instances of multi-station indoor point cloud data to obtain coarsely registered point cloud data, including:

[0293] 5.1) The indoor point cloud data of each station is registered with the subsequent point cloud data in two stages, based on the pairwise registration algorithm of the large plane and the pairwise registration algorithm of the gate respectively, to obtain the corresponding registration score and transformation matrix.

[0294] The principle of the pairwise registration algorithm based on a large plane is as follows: Using the source point cloud and target point cloud as input, the ceiling point cloud data is first traversed, and the nearest neighboring walls are selected as candidate walls for correspondence matching during the registration process. A pair of ceilings and two pairs of mutually perpendicular walls from the source and target point cloud data, respectively, are used to calculate transformation parameters, including rotation matrices and translation vectors. The rotation matrix is ​​in the following form:

[0295]

[0296] Optimal rotation angle θ opt Calculated by the following formula

[0297]

[0298] The translation vector t = (t1, t2, t3) is obtained by solving the following system of equations:

[0299]

[0300] Where θ represents the rotation angle about the z-axis, and The normal vectors of the source point cloud's ceiling plane and the two walls. and This represents the normal vectors of the target point cloud to the ceiling plane and the two walls. and Represents the intercepts of the source point cloud after rotation across the three planes. and R(θ) represents the intercepts of the three planes of the target point cloud, and R(θ) represents the rotation matrix with θ as the independent variable. opt Let p be a point in the source point cloud, then the transformed point p′ is represented as follows:

[0301] p′=R(θ opt p+t

[0302] The ceiling point cloud data is used as the source point cloud data, and the wall plane point cloud data adjacent to the ceiling point cloud data is used as the target point cloud data.

[0303] The registration score is shown below:

[0304] score=∑ obj∈{LP,w,d} overlap obj -∑ form∈{h,v} ωform *conflict form (14)

[0305] In the formula, obj is an element of the set {LP, w, d}. form is an element of the set {h, v}. overlap LP overlap w overlap d conflict h conflict v These are the evaluation functions for the large plane, window, door, horizontal direction, and vertical direction, respectively. ω form Represents the weight, ω h Represents the horizontal conflict weight, ω v This represents the vertical conflict weight.

[0306] Among them, the evaluation function overlap based on the pairwise registration algorithm of the large plane is... obj The calculation formula is as follows:

[0307]

[0308] In the formula, N overlap N represents the total number of overlapping points. tar N represents the number of points in the target point cloud. sor The number of points in the source point cloud.

[0309] The evaluation function of the gate-based pairwise registration algorithm is overlap. obj The calculation formula is as follows:

[0310]

[0311] overlap LP The calculation principle is as follows: For each point in the target ceiling point cloud data and wall point cloud data, query the nearest neighbor point in the transformed ceiling point cloud data and wall point cloud data within a preset radius drad. If the query yields a result, the result is included in the overlapping point count. The total overlapping point count is N. overlap overlap LP The calculation is as follows:

[0312]

[0313] Where, N trans N represents the total number of points in the transformed ceiling and wall point cloud data. tar The total number of points in the target ceiling point cloud data and wall point cloud data.

[0314] overlap dThe calculation principle is as follows: Initialize overlap d =0, for each transformed gate point cloud data, query the target gate point cloud data with parallel normal vectors. For each point in the queried target gate point cloud data, query the nearest neighbor point belonging to the transformed gate point cloud data within a preset radius *drad*, and include this point in the overlapping point count. The total number of overlapping points is denoted as N. overlap Then the intermediate variable

[0315]

[0316] A preset threshold limit is set if there is overlap. temp If it is less than the limit, update the overlap. d =overlap d +overlap temp Otherwise update overlap d =overlap d +limit. overlap w The overlap of the representative window point cloud data is calculated using the same method as overlap. d .

[0317] conflict v The calculation principle is as follows:

[0318]

[0319] Where N sor N represents the number of points in the source point cloud. tar Indicates the number of points in the target point cloud. This indicates the number of points in the target's vertical collision bounding box within the transformed point cloud data of doors, windows, and walls. This represents the number of points in the transformed vertical conflict bounding box of the target door, window, and wall point cloud data. Both of these point counts are collectively referred to as the conflict point count. The principle for calculating the conflict point count is as follows:

[0320]

[0321] Among them, P WDW Let (x, y, z) represent the point cloud of doors, windows, and walls, where (x, y, z) represents a single point. and These represent the two diagonal points of the conflict box.

[0322]

[0323]

[0324] in and and Let r represent the minimum and maximum points of the ceiling point cloud data, respectively. c Let represent the preset constraint ratio, and h represent the preset extension length. Finally, to balance the contributions of the various components of the evaluation function, an amplification factor ω is introduced. v .

[0325] conflict h The calculation principle is as follows: First, construct several horizontal conflict bounding boxes for each door / window point cloud data from the source point cloud and each door / window point cloud data from the target point cloud. Each horizontal conflict bounding box requires two diagonal points, and the values ​​of these two diagonal points on the x-component are defined as follows:

[0326]

[0327]

[0328] in and represents the minimum and maximum values ​​of each gate point cloud data along the x-axis, respectively, and l represents the preset extension length of the horizontal conflict bounding box. c This indicates the shrinkage rate of the preset horizontal conflict bounding box along the illegal direction. This represents the normal vector of the gate point cloud data. This represents a unit vector pointing in the positive x-axis direction. The values ​​of the two diagonal points in the y and z components. and Defined in the same way. Wall point cloud data P is retrieved using horizontal conflict bounding boxes generated from diagonal points. wall A conflict point, or a set of conflict points, is defined as follows:

[0329]

[0330] To calculate conflict h , will P conflict Projected onto a 2D image along the normal direction, after morphological dilation, image edges are detected and corresponding rectangular bounding boxes are constructed. Assuming there are N pieces of door and window data from the source point cloud and the target point cloud, for the i-th door (or window) point cloud data, the area ratio is AR... i Defined as the area of ​​the largest rectangular bounding box and the projected area of ​​the horizontal conflict bounding box along the normal, a first transformation Finally, to balance the contributions of the evaluation function, an amplification factor ω is introduced. h After traversing all possible transformations, the transformation parameter with the highest score is selected as the correct transformation parameter between the two scanned point clouds.

[0331] The principle of the pairwise registration algorithm based on gates is as follows: An axis-aligned bounding box is constructed for the gate point cloud data, and transformation parameters are calculated using one top face and four side faces of the bounding box. In the evaluation function, the overlap in the pairwise registration algorithm based on large planes is considered. LP overlap w overlap d The calculation of overlap rate is based on

[0332]

[0333] Become

[0334]

[0335] 5.2) Determine whether the registration score is greater than the preset threshold. If so, define that the two stations corresponding to the registration score have a registration relationship.

[0336] The registration score threshold is set to 0.2.

[0337] 5.3) Define each station as a node. The registration relationship between two nodes represents an edge between the two nodes. The weight of the edge is the registration score between the two stations, resulting in one or more connected graphs.

[0338] 5.4) For each connected graph, the edges are pruned using the maximum spanning tree algorithm, and the node with the maximum proximity centrality is designated as the registration reference station. Other stations are transformed to the local coordinate system of the registration reference station according to the transformation matrix obtained by the shortest transformation path, thus obtaining coarse registration point cloud data.

[0339] The formula for calculating the proximity centrality is as follows:

[0340]

[0341] In the formula, f and u are nodes. C(u) is the closeness centrality of node u. d(f,u) represents the shortest distance between node u and node f in the connected graph. n-1 represents the number of nodes that node u can reach.

[0342] Example 20:

[0343] A multi-view point cloud automatic registration method for interior scenes of finished houses, the main technical contents of which are described in any one of embodiments 11 to 19, further comprising the step of registering the coarsely registered point cloud data using a pose graph-based fine registration algorithm to complete the fine registration of all point cloud data, including:

[0344] 6.1) Using the coarse registration point cloud data of the first station as the fine registration reference station, the coarse registration point cloud data of each station is sequentially registered with the coarse registration point cloud data after the fine registration reference station through the ICP algorithm.

[0345] 6.2) Fine registration of adjacent stations generates adjacent edges, and fine registration of non-adjacent stations generates loop edges. Each edge corresponds to a fine registration score and a transformation matrix.

[0346] 6.3) Determine whether the fine registration score is greater than the preset threshold. If not, define the edge corresponding to the fine registration score as an invalid edge and exclude the invalid edge to obtain the registration adjacency relationship graph.

[0347] The preset threshold is 0.25.

[0348] 6.4) Remove loop edges in the registration adjacency graph that contain incorrect registration relationships to obtain a finely registered adjacency graph.

[0349] 6.5) Based on the least squares optimization algorithm, the pose graph of the fine registration adjacency relationship graph is optimized to complete the fine registration of all point cloud data.

[0350] Example 21:

[0351] See Figures 1 to 12 An automatic registration method for multi-view point clouds in finished house interior scenes includes the following steps:

[0352] S101. Use a 3D laser scanner to obtain multi-station scan point cloud data;

[0353] S102. Filter and lightweight processing are performed on the multi-station scanning point cloud data in step S101 to obtain multi-station indoor point cloud data.

[0354] S103. Perform semantic segmentation on the multi-station indoor point cloud data obtained in step S102;

[0355] S104. For the ceiling point cloud data obtained in step S103, calculate the deflection angle between the ceiling edge line and the coordinate axis, and perform axis alignment operation on the scanned point cloud data.

[0356] S105. Perform instance segmentation on the axis-aligned semantic segmentation data obtained in steps S103 and S104.

[0357] S106. For the multi-station indoor point cloud data obtained in step S105, use a large plane-based registration algorithm to perform pairwise registration and determine the multi-station registration relationship.

[0358] S107. For the point cloud data obtained in step S106, use a gate-based registration algorithm and the registration process based on step S106 to complete the registration of all point cloud data to obtain coarsely registered point cloud data.

[0359] S108. For the coarse registration point cloud data obtained in step S107, use the fine registration algorithm based on pose graph optimization to complete the fine registration of all point cloud data.

[0360] Step S101 includes:

[0361] Multi-station point cloud data was obtained using a 3D laser scanner.

[0362] Step S102 includes:

[0363] The collected data may contain a large number of outdoor points in the point cloud, so we choose to filter out the scan points that are more than a preset range away from the scan center point; because the data volume of a single station is too large, we perform lightweight processing.

[0364] Step S103 includes:

[0365] A point cloud semantic segmentation neural network was used to perform semantic segmentation on the point cloud data. To train the network model, a dataset was created from indoor scanned point cloud data of multiple houses with different floor plans. The scanned data for each station was manually segmented into point cloud data for the ceiling, floor, walls, windows, doors, and noise. Finally, the trained network model was used to perform semantic segmentation on the scanned point cloud data.

[0366] Step S104 includes:

[0367] The ceiling point cloud data is clustered using a density clustering algorithm. The point cloud data is then projected to generate an xoy plane binary image. The ceiling edge lines are extracted using a morphological dilation algorithm, an image edge detection algorithm, and a line detection algorithm.

[0368] Step S105 includes:

[0369] For the single ceiling obtained in step S104, a plane fitting algorithm is used to fit the ceiling plane; for the wall point cloud data obtained in step S104, a plane fitting algorithm is used to fit the wall plane, and the wall planes are classified according to their normal vectors. A density clustering algorithm is used to extract individual wall planes in each class; for the window point cloud data obtained in step S104, a filter is used to remove sparsely distributed discrete points in space, and a density clustering algorithm is used to extract individual windows. The range of the window data in the xyz axis direction is calculated, and the window normal vector is determined according to the minimum range axis and the relative position relationship between the window data and the scanning origin; for the door point cloud data obtained in step S104, a segmentation algorithm based on the support plane is used for segmentation, and a density clustering algorithm is used to extract individual doors from each cluster of segmented door data.

[0370] Step S106 includes:

[0371] For the multi-station indoor point cloud data obtained in step S105, starting from the first station, a large-plane-based registration algorithm is used to register the scanned point cloud data of each station with the subsequent point cloud data in sequence. Each registration yields a registration score and a transformation matrix. Two stations with registration scores greater than a specified threshold are defined as having a registration relationship. Each station is considered as a node in graph theory. A registration relationship between two nodes represents an edge between them, with the weight of the edge being the registration score of the two stations. This results in one or more connected graphs. For each connected graph, the maximum spanning tree algorithm is used to prune edges, and the node with the maximum proximity centrality is designated as the registration reference station. Other stations are transformed to the local coordinate system of the registration reference station using the obtained transformation matrix based on the shortest transformation path.

[0372] Step S107 includes:

[0373] For the point cloud data obtained in step S106, a gate-based registration algorithm and a registration process based on step S106 are used to complete the registration of all point cloud data, resulting in coarsely registered point cloud data.

[0374] Step S108 includes:

[0375] The adjacency graph generated from the multi-view coarse registration results yields the adjacency relationships between each scanned data point. Starting with the first station's scanned data, it serves as the registration reference station. Each station's scanned data is then finely registered with the subsequent scanned data using an iterative nearest neighbor algorithm. Adjacent station registration generates adjacent edges, while non-adjacent station registration generates loop edges. Each edge represents a score and a transformation matrix. Invalid edges are excluded based on a pre-set score threshold. The global optimization process for the pose graph consists of two steps. The first step considers all edges in the original pose graph and excludes loop edges containing incorrect registration relationships. These loop edges have low scores and are excluded after the first optimization step. The second optimization step uses a least squares optimization algorithm to optimize the pose graph, ultimately obtaining the transformation matrix for all scanned data.

[0376] Example 22:

[0377] See Figures 1 to 12 An automatic registration method for multi-view point clouds in finished house interior scenes includes the following steps:

[0378] S101. Use a 3D laser scanner to obtain multi-station scan point cloud data;

[0379] S102. Filter and lightweight processing are performed on the multi-station scanning point cloud data in step S101 to obtain multi-station indoor point cloud data.

[0380] S103. Perform semantic segmentation on the multi-station indoor point cloud data obtained in step S102;

[0381] S104. For the ceiling point cloud data obtained in step S103, calculate the deflection angle between the ceiling edge line and the coordinate axis, and perform axis alignment operation on the scanned point cloud data.

[0382] S105. Perform instance segmentation on the axis-aligned semantic segmentation data obtained in steps S102 and S103.

[0383] S106. For the multi-station indoor point cloud data obtained in step S105, use a large plane-based registration algorithm to perform pairwise registration and determine the multi-station registration relationship.

[0384] S107. For the point cloud data obtained in step S106, use a gate-based registration algorithm and the registration process based on step S106 to complete the registration of all point cloud data to obtain coarsely registered point cloud data.

[0385] S108. For the coarse registration point cloud data obtained in step S107, use the fine registration algorithm based on pose graph optimization to complete the fine registration of all point cloud data.

[0386] Step S101 includes:

[0387] When using a terrestrial 3D laser scanner to collect point cloud data for a single-station indoor scene of a house, one or two stations are set up for scanning a room. The terrestrial 3D laser scanner is fixed on a tripod and leveled to obtain single-station scan point cloud data.

[0388] Step S102 includes:

[0389] When collecting data from a terrestrial 3D laser scanner, the point cloud may contain a large number of outdoor points. Therefore, points that are more than a preset distance from the scan center point are filtered out. Even after filtering, the number of scan points remains enormous, and directly inputting them into subsequent algorithms would reduce computational efficiency. Therefore, lightweight processing is performed on the scanned point cloud data.

[0390] Step S103 includes:

[0391] A point cloud semantic segmentation neural network was used to perform semantic segmentation on the point cloud data. To train the network model, a dataset was created from indoor scanned point cloud data of multiple houses with different floor plans. The scanned data for each station was manually segmented into point cloud data for the ceiling, floor, walls, windows, doors, and noise. Finally, the trained network model was used to perform semantic segmentation on the scanned point cloud data.

[0392] Step S104 includes:

[0393] Density clustering algorithm is used to cluster ceiling point cloud data. Points with similar spatial distances are grouped into one class. If the number of points in the ε-neighborhood of a point exceeds a preset threshold, that point is defined as a core point. The basic idea of ​​the algorithm is to divide the original point cloud by continuously querying unvisited core points and grouping the points in the ε-neighborhood of the core points into one class.

[0394] To project the point cloud data into a binary image in the xoy plane, a resolution res is set, and the point cloud coordinates (x, y) are mapped to the image coordinate system. First, the new coordinate values ​​in the two-dimensional image are calculated:

[0395]

[0396]

[0397] To avoid negative image coordinates, the data needs to be shifted:

[0398] x img ′=x img -x min -a

[0399] y img ′=y img -y min -b

[0400] Where x min and y min These are the coordinate values ​​x of the two-dimensional image. img and y img The minimum value in the image. The values ​​of a and b are set to project the point cloud to a suitable central position in the image. The height and width values ​​of the image array are set as follows:

[0401] imx = max(x) img ′)+2a

[0402] imy = max(y img ′)+2b

[0403] Where max(x) img ′) and max(y img ′) are the coordinate values ​​of the two-dimensional image x img ′ and y img The maximum value in ' is used. Finally, the point cloud data is filled into the image pixels. The pixel value is 255 where there are points in the image, and 0 where there are no points.

[0404] The principle of the morphological dilation algorithm is as follows: Dilate image A using a sliding window B as follows:

[0405]

[0406] The origin of the sliding window B is shifted to the position of the image pixel (x,y). If the intersection of B and A at pixel (x,y) is not empty (that is, at least one of the image values ​​of A corresponding to the element position of 1 in B is 1), then the corresponding pixel (x,y) of the output image is assigned the value 1; otherwise, it is assigned the value 0.

[0407] The principle of the edge detection algorithm based on the Sobel operator is as follows: The horizontal convolution matrix and the vertical convolution matrix are convolved with the grayscale image respectively to obtain G... x and G y If the gradient G is greater than a certain threshold, then the point (x, y) is considered an edge point.

[0408]

[0409]

[0410]

[0411] Background point cloud removal is achieved using a plane fitting algorithm. The algorithm assumes a model with at least D data points and a dataset much larger than D. T data points are randomly selected from the dataset to fit the model. Based on a predefined tolerance, data points that satisfy the model are called in-circle points, and those that do not are called outliers. This process is iterative. The effectiveness of the model depends on whether the number of in-circle points is sufficient; in-circle point data from effective models are retained for further processing.

[0412] Line detection algorithms use the Hough transform to transform the image coordinate space to a parameter space. All lines intersecting at the same point in the parameter space have corresponding collinear points in the image coordinate space. Based on this property, given some edge points in the image coordinate space, the equations of the lines connecting these points can be determined using the Hough transform. In the image xy coordinate system, the line passing through the point (x...)... i ,y i The straight line y = kx i +b transforms to the parameter space kb:

[0413] b = -kx i +y

[0414] It is evident that points on a straight line in the xy coordinate space will intersect at a single point in the kb parameter space. For each edge point in the image, a straight line can be drawn in the kb space. At the intersection of these lines, a "vote" method is used. A two-dimensional accumulation array A(k,b) is established, which represents the intersection point (k,b) of n straight lines. i ,b i ) has A(k i ,bi =n, the threshold n0 for line detection detects lines that satisfy A(k) i ,b i The standard HOUGH transform of a straight line uses the following parameterized equation:

[0415] rho=x*cos(theta)+y*sin(theta)

[0416] The output parameters of the edge lines detected by the line detection algorithm are the polar radius rho and the polar angle theta. The rotation angle of the axis-aligned point cloud data is θ, which is related to the polar angle theta by θ = theta. Let the coordinates of the point cloud data before the affine transformation in coordinate system p be (x, y), and with the scanning origin (0, 0) as the base point, the coordinates of the point cloud data after the rotation transformation in coordinate system p′ are p′. The rotation transformation formula can be expressed as:

[0417] p′=Rp

[0418] Where the rotation transformation matrix

[0419] Step S105 includes:

[0420] The window data obtained from semantic segmentation may contain some sparse noise that affects post-processing. Statistical filters are used to perform statistical analysis on the neighborhood of each point, filtering out some outliers that do not meet the requirements based on the distance distribution characteristics from the point to all its neighbors.

[0421] In the instance segmentation process of wall point cloud data and door and window point cloud data, one of the characteristics of these instances is their normal vector. The normal vector is determined by the orientation of these instances and their relative position to the scanning origin. Instance segmentation of door point cloud data uses a support plane-based segmentation algorithm, the principle of which is as follows: traverse the wall point cloud data, construct an axis-aligned bounding box for each wall point cloud data piece, expand the bounding box along the normal vector direction away from the scanning origin, and extract the door point cloud data within the bounding box.

[0422] Step S106 includes:

[0423] The principle of the pairwise registration algorithm based on a large plane is as follows: Using the source point cloud and target point cloud as input, the ceiling point cloud data is first traversed, and the nearest neighboring walls are selected as candidate walls for correspondence matching during the registration process. A pair of ceilings and two pairs of mutually perpendicular walls from the source and target point cloud data, respectively, are used to calculate transformation parameters, including rotation matrices and translation vectors. The rotation matrix is ​​in the following form:

[0424]

[0425] Optimal rotation angle θ opt Calculated by the following formula

[0426]

[0427] The translation vector t = (t1, t2, t3) is obtained by solving the following system of equations:

[0428]

[0429] Where θ represents the rotation angle about the z-axis, and The normal vectors of the source point cloud's ceiling plane and the two walls. and This represents the normal vectors of the target point cloud to the ceiling plane and the two walls. and Represents the intercepts of the source point cloud after rotation across the three planes. and R(θ) represents the intercepts of the three planes of the target point cloud, and R(θ) represents the rotation matrix with θ as the independent variable. opt Let p be a point in the source point cloud, then the transformed point p′ is represented as follows:

[0430] p′=R(θ opt p+t

[0431] Then, a score is calculated for the current transformation parameters using the following evaluation function:

[0432]

[0433] This evaluation function contains 5 terms, namely overlap LP overlap w overlap d ,conflict h and conflict v .

[0434] overlap LP The calculation principle is as follows: For each point in the target ceiling point cloud data and wall point cloud data, query the nearest neighbor point in the transformed ceiling point cloud data and wall point cloud data within a preset radius drad. If the query yields a result, the result is included in the overlapping point count. The total overlapping point count is N. overlap overlap LP The calculation is as follows:

[0435]

[0436] Where, N transN represents the total number of points in the transformed ceiling and wall point cloud data. tar The total number of points in the target ceiling point cloud data and wall point cloud data.

[0437] overlap d The calculation principle is as follows: Initialize overlap d =0, for each transformed gate point cloud data, query the target gate point cloud data with parallel normal vectors. For each point in the queried target gate point cloud data, query the nearest neighbor point belonging to the transformed gate point cloud data within a preset radius *drad*, and include this point in the overlapping point count. The total number of overlapping points is denoted as N. overlap Then the intermediate variable

[0438]

[0439] A preset threshold limit is set if there is overlap. temp If it is less than the limit, update the overlap. d =overlap d +overlap temp Otherwise update overlap d =overlap d +limit. overlap w The overlap of the representative window point cloud data is calculated using the same method as overlap. d .

[0440] conflict v The calculation principle is as follows:

[0441]

[0442] Where N sor N represents the number of points in the source point cloud. tar Indicates the number of points in the target point cloud. This indicates the number of points in the target's vertical collision bounding box within the transformed point cloud data of doors, windows, and walls. This represents the number of points in the transformed vertical conflict bounding box of the target door, window, and wall point cloud data. Both of these point counts are collectively referred to as the conflict point count. The principle for calculating the conflict point count is as follows:

[0443]

[0444] Among them, P WDW Let (x, y, z) represent the point cloud of doors, windows, and walls, where (x, y, z) represents a single point. and These represent the two diagonal points of the conflict box.

[0445]

[0446]

[0447] in and and Let r represent the minimum and maximum points of the ceiling point cloud data, respectively. c Let represent the preset constraint ratio, and h represent the preset extension length. Finally, to balance the contributions of the various components of the evaluation function, an amplification factor ω is introduced. v .

[0448] conflict h The calculation principle is as follows: First, construct several horizontal conflict bounding boxes for each door / window point cloud data from the source point cloud and each door / window point cloud data from the target point cloud. Each horizontal conflict bounding box requires two diagonal points, and the values ​​of these two diagonal points on the x-component are defined as follows:

[0449]

[0450]

[0451] in and represents the minimum and maximum values ​​of each gate point cloud data along the x-axis, respectively, and l represents the preset extension length of the horizontal conflict bounding box. c This indicates the shrinkage rate of the preset horizontal conflict bounding box along the illegal direction. This represents the normal vector of the gate point cloud data. This represents a unit vector pointing in the positive x-axis direction. The values ​​of the two diagonal points in the y and z components. and Defined in the same way. Wall point cloud data P is retrieved using horizontal conflict bounding boxes generated from diagonal points. wall A conflict point, or a set of conflict points, is defined as follows:

[0452]

[0453] To calculate conflict h , will P conflict Projected onto a 2D image along the normal direction, after morphological dilation, image edges are detected and corresponding rectangular bounding boxes are constructed. Assuming there are N pieces of door and window data from the source point cloud and the target point cloud, for the i-th door (or window) point cloud data, the area ratio is AR... i Defined as the area of ​​the largest rectangular bounding box and the projected area of ​​the horizontal conflict bounding box along the normal, a first transformation Finally, to balance the contributions of the evaluation function, an amplification factor ω is introduced.h After traversing all possible transformations, the transformation parameter with the highest score is selected as the correct transformation parameter between the two scanned point clouds.

[0454] The principle for calculating the closeness centrality of any node u in graph G is as follows:

[0455]

[0456] Where d(v,u) represents the shortest distance between node u and node v in graph G, and n-1 represents the number of nodes that u can reach.

[0457] S107 includes:

[0458] The principle of the pairwise registration algorithm based on gates is as follows: An axis-aligned bounding box is constructed for the gate point cloud data, and transformation parameters are calculated using one top face and four side faces of the bounding box. In the evaluation function, the overlap in the pairwise registration algorithm based on large planes is considered. LP overlap w overlap d The calculation of overlap rate is based on

[0459]

[0460] Become

[0461]

[0462] Step S108 includes:

[0463] The adjacency graph generated based on the multi-view coarse registration results yields the adjacency relationships between each scanned data point. Starting with the first station's scanned data, using it as the registration reference station, each station's scanned data is sequentially finely registered with the subsequent scanned data using the ICP algorithm. The ICP principle is as follows: For any point p in the source point cloud, find the nearest neighbor q in the target point cloud within a given distance threshold range to obtain the nearest neighbor set K; calculate the rigid transformation matrix T to minimize the total distance error function of the corresponding points. Then, perform a rigid transformation on the source point cloud and evaluate it; if the number of iterations exceeds the preset maximum number, stop the iteration; otherwise, return to the process of finding the nearest neighbor set. Adjacent station registration generates adjacent edges, and non-adjacent station registration generates loop edges. Each edge represents a score and a transformation matrix, and invalid edges are excluded based on a preset score threshold. The global optimization process of the pose graph is divided into two steps. The first step considers all edges on the original pose graph and excludes loop edges containing incorrect registration relationships. These loop edges have low scores and are excluded after the first step of optimization. The second step of optimization is to optimize the pose graph based on the least squares optimization algorithm, and finally obtain the transformation matrix of all scan data.

[0464] Example 23:

[0465] See Figures 1 to 12 An automatic registration method for multi-view point clouds in finished house interior scenes includes the following steps:

[0466] S101. Use a 3D laser scanner to obtain multi-station scan point cloud data;

[0467] In specific implementation, such as Figure 2 The scanner is fixed on a tripod and scans the indoor scene from 0° to 360° horizontally and from -60° to 90° vertically to obtain indoor scene point cloud data.

[0468] S102. Filter and lightweight processing are performed on the multi-station scanning point cloud data in step S101 to obtain multi-station indoor point cloud data.

[0469] In practice, point cloud data that are outside the preset radius from the scanning center are deleted. The preset radius is set to 10 meters, and the sampling rate is set to 2%.

[0470] S103. Perform semantic segmentation on the multi-station indoor point cloud data obtained in step S102;

[0471] In practice, the StraitifiedTransformer point cloud semantic segmentation neural network was used to perform semantic segmentation on the lightweight scanned point cloud data. To train the StraitifiedTransformer network, a dataset was created consisting of 91 scanned point cloud data points from 7 different house types. The dataset was organized in the manner of the S3DIS indoor scene dataset. Each scanned data point was manually segmented into ceiling, floor, walls, windows, doors, and noise. Subsequently, the dataset was organized in the same manner as the S3DIS indoor scene dataset and input into the StraitifiedTransformer network for training. Finally, the trained network model was used to perform semantic segmentation on the scanned point cloud data. The key parameters for training and testing were set as follows: batch size of 2, epochs of training of 100, base learning rate of 0.006, weight decay of 0.01, and dropout rate of 0.5.

[0472] S104. For the ceiling point cloud data obtained in step S103, calculate the deflection angle between the ceiling edge line and the coordinate axis, and perform axis alignment operation on the scanned point cloud data.

[0473] In practice, the DBSCAN density clustering algorithm is used to cluster the ceiling point cloud data, with a clustering distance of 0.2 meters. The density-clustered and normalized ceiling data will be mapped to a binary image on the xoy plane, with an image grid resolution of 0.01 meters and edge augmentation of 50 pixels.

[0474] S105. Perform instance segmentation on the axis-aligned semantic segmentation data obtained in steps S103 and S104.

[0475] In specific implementation, for ceiling instance segmentation, the RANSAC plane fitting algorithm was used to remove background point clouds, with a plane fitting distance threshold of 0.02 meters and 1000 iterations. For wall instance segmentation, the density clustering distance was set to 0.2 meters. For window instance segmentation, a statistical filter was used to remove sparsely distributed discrete points in the window data, with a neighborhood of 20, a standard deviation of 2, and a density clustering distance of 0.4 meters. For door instance segmentation, the support plane extension distance was 0.25 meters, and the subsequent density clustering distance was set to 0.2 meters.

[0476] S106. For the multi-station indoor point cloud data obtained in step S105, use a large plane-based registration algorithm to perform pairwise registration and determine the multi-station registration relationship.

[0477] In practice, the registration score threshold is set to 0.2. In the pairwise registration algorithm based on a large plane, the source point cloud and the target point cloud are used as inputs. First, the ceiling point cloud data is traversed, and the minimum point value in the horizontal direction of the ceiling point cloud data is determined. and maximum point Construction point and points Traverse the wall point cloud data, for the i-th wall point cloud data If point set

[0478]

[0479] If the number of points |Γ| is greater than a certain preset point threshold ptsRatio, then it is considered... This indicates that the wall surface is a candidate wall surface for matching the correspondence, where t represents the wall thickness and ε represents the tolerance. ptsRatio is set to 0.5, t to 0.1 meters, and ε to 0.1 meters. An evaluation function is used.

[0480]

[0481] Calculate a score for the transformation parameters of the current loop, where the amplification factor ω h and ω vThey were set to 0.8 and 20 respectively.

[0482] S107. For the point cloud data obtained in step S106, use a gate-based registration algorithm and the registration process based on step S106 to complete the registration of all point cloud data to obtain coarsely registered point cloud data.

[0483] In practice, axis-aligned bounding boxes are generated for the source gate point cloud data and the target gate point cloud data. The top surface of the bounding box corresponds to the ceiling in step S107, and the four sides correspond to the walls in step S107. The correspondence between each surface is established according to the same algorithm as in step S107, the transformation matrix is ​​calculated and evaluated, and the evaluation function remains unchanged except for the overlap rate, with the same parameters as in step S106.

[0484] S108. For the coarse registration point cloud data obtained in step S107, use the fine registration algorithm based on pose graph optimization to complete the fine registration of all point cloud data.

[0485] In practice, the ICP algorithm has a preset maximum number of iterations of 30; otherwise, it returns to the process of finding the nearest neighbor set. Registration between adjacent stations generates adjacent edges, and registration between non-adjacent stations generates loop edges. Each edge represents a score and a transformation matrix. Invalid edges are excluded based on a pre-set score threshold. The global optimization process for the pose graph consists of two steps. The first step considers all edges in the original pose graph and excludes loop edges containing incorrect registration relationships. These loop edges have low scores and are excluded after the first optimization step. The second optimization step uses the Levenberg-Marquardt least squares optimization algorithm to optimize the pose graph, ultimately obtaining the transformation matrix of all scanned data. The score threshold for the first step of pose graph optimization is set to 0.25.

[0486] The above experimental results demonstrate that the automatic registration method for multi-view point clouds of finished house interior scenes disclosed in this invention can complete the automatic registration of multi-view point clouds of finished house interior scenes. It can use a computer to calculate the transformation parameters of each scanning station to complete the multi-view point cloud registration. The method described in this invention is indeed effective.

[0487] The above are merely preferred embodiments of the present invention. It should be noted that those skilled in the art can make several modifications and improvements without departing from the present method, and such modifications and improvements should also be considered to fall within the scope of protection claimed in this application.

[0488] Table 1

[0489]

[0490] Table 2

[0491]

Claims

1. A method for automatic registration of multi-view point clouds for interior scenes of finished houses, characterized in that, Includes the following steps: 1) Acquire multi-station scan point cloud data of indoor scenes, and perform filtering and lightweight processing on the multi-station scan point cloud data to obtain processed multi-station scan point cloud data; 2) Perform semantic segmentation on the processed multi-station scan point cloud data to obtain point cloud data with different structures; The point cloud data with different structures includes ceiling point cloud data; 3) Based on the ceiling point cloud data, calculate the deflection angle of the ceiling edge line, and align the point cloud data of different structures according to the deflection angle of the ceiling edge line. The steps include: 3.1) Cluster the ceiling point cloud data using a density clustering algorithm; 3.2) Project point cloud data with different structures onto a two-dimensional plane to generate a binary image of the xoy plane; 3.3) Detect edge points in the xoy plane binary image and calculate the deflection angle of the ceiling edge line; 3.4) Align the point cloud data of different structures according to the deflection angle of the ceiling edge line; 4) Perform instance segmentation on the point cloud data of different structures after axis alignment to obtain instances of multi-station indoor point cloud data; 5) Perform pairwise registration on instances of multi-station indoor point cloud data to obtain coarsely registered point cloud data; 6) Use a pose-graph-based optimization-based fine registration algorithm to register the coarsely registered point cloud data, completing the fine registration of all point cloud data. The steps include: 6.1) Using the coarse registration point cloud data of the first station as the fine registration reference station, the coarse registration point cloud data of each station is sequentially registered with the coarse registration point cloud data after the fine registration reference station through the ICP algorithm. 6.2) Fine registration of adjacent stations generates adjacent edges, and fine registration of non-adjacent stations generates loop edges. Each edge corresponds to a fine registration score and a transformation matrix. 6.3) Determine whether the fine registration score is greater than the preset threshold. If not, define the edge corresponding to the fine registration score as an invalid edge and exclude the invalid edge to obtain the registration adjacency relationship graph. 6.4) Remove loop edges in the registration adjacency graph that contain incorrect registration relationships to obtain a finely registered adjacency graph; 6.5) Based on the least squares optimization algorithm, the pose graph of the fine registration adjacency relationship graph is optimized to complete the fine registration of all point cloud data.

2. The method for automatic registration of multi-view point clouds for interior scenes of finished houses according to claim 1, characterized in that, The step of semantic segmentation of the processed multi-station scan point cloud data includes: 2.1) Establish a dataset of indoor scanned point cloud data for different types of houses, and segment the indoor scanned point cloud data of different types of houses in the dataset into point cloud data with different structures to obtain a training set; 2.2) Constructing a network model; 2.3) Train the network model using the training set to obtain the semantic segmentation network model; 2.4) Input the processed multi-station scan point cloud data into the semantic segmentation network model to obtain point cloud data with different structures.

3. The method for automatic registration of multi-view point clouds for interior scenes of finished houses according to claim 1, characterized in that, The point cloud data with different structures also includes ground point cloud data, wall point cloud data, window point cloud data, door point cloud data, and noise point cloud data.

4. The method for automatic registration of multi-view point clouds for interior scenes of finished houses according to claim 1, characterized in that, The step of projecting point cloud data with different structures onto a two-dimensional plane to generate a binary image of the xoy plane includes: 3.2.1) Project the point cloud data onto a two-dimensional plane; The coordinates of the point cloud data The coordinates projected onto the two-dimensional plane are: （1） （2） In the formula, , The coordinates of the point cloud data are respectively The horizontal and vertical coordinates projected onto a two-dimensional plane; For resolution; 3.2.2) The point cloud data projected onto the two-dimensional plane is translated to obtain the coordinates of the two-dimensional image point cloud, as shown below: （3） （4） In the formula, , These are the x-coordinate and y-coordinate in the point cloud coordinates of the two-dimensional image, respectively. Two-dimensional image coordinates , The minimum value in; All are preset values; 3.2.3) Set preset values Projecting the point cloud coordinates onto the center of the image ( ),Right now: （5） （6） In the formula, , These are the coordinate values ​​of the point cloud in the two-dimensional image. , The maximum value in; 3.2.4) Set the pixel at the point cloud coordinate of the two-dimensional image in the two-dimensional plane as g, and set other locations as k to obtain the binary image in the xoy plane, where g and k are both integers.

5. The method for automatic registration of multi-view point clouds for interior scenes of finished houses according to claim 1, characterized in that, The step of detecting edge points in the xoy plane binary image and calculating the deflection angle of the ceiling edge line includes: 3.3.1) Dilate the xoy plane binary image using a morphological dilation algorithm to obtain a binary image; The formula for calculating the dilation of the xoy plane binary image by the morphological dilation algorithm is as follows: （7） In the formula, B is the sliding window; A is the binary image in the xoy plane; () represents the coordinates of the point cloud in the two-dimensional image; When sliding window B is in the coordinates of the two-dimensional image point cloud The intersection of point () with the binary image A in the xoy plane is not empty, and the coordinates of the output binary image are... The value at the () position is assigned as 1, otherwise it is assigned as 0; 3.3.2) Calculate the values ​​of each point in the binary image sequentially. The gradient at point () is used to obtain the edge image, as shown below: （8） In the formula, G is the gradient of the binary image; Wherein, the convolution function between the horizontal convolution matrix and the binary image... The convolution function of the vertical convolution matrix and the binary image As shown below: （9） （10） In the formula, It is a binary image; 3.3.3) Use a plane fitting algorithm to remove the background point cloud from the edge image to obtain the edge points; 3.3.4) Transform the edge points to the parameter space and obtain the edge lines by HOUGH transformation; The HOUGH transform is shown below: （11） In the formula, Polar radius; This is the deflection angle of the ceiling edge line.

6. The method for automatic registration of multi-view point clouds for interior scenes of finished houses according to claim 1, characterized in that, The calculation formula for axis-aligned rotation transformation of point cloud data with different structures is as follows: （12） In the formula, , These are the coordinates before and after the rotation transformation, respectively. Wherein, the rotation transformation matrix As shown below: （13） In the formula, This is the deflection angle of the ceiling edge line.

7. The method for automatic registration of multi-view point clouds for interior scenes of finished houses according to claim 1, characterized in that, The step of segmenting the point cloud data with different structures after axis alignment to obtain instances of multi-station indoor point cloud data includes: 4.1) Use a plane fitting algorithm to fit the ceiling plane and wall plane; 4.2) Determine the normal vector of the wall plane based on its orientation and relative position to the scanning origin; The scanning origin is the base point for axial alignment of point cloud data with different structures; 4.3) Classify the wall planes according to their normal vectors, and use density clustering algorithm to extract individual wall planes in each class; 4.4) Statistical analysis of the neighborhood of the window point cloud data is performed using a statistical filter to obtain noise-removed window point cloud data; 4.5) Extract individual windows using a density clustering algorithm; 4.6) Use a support plane-based segmentation algorithm to segment the gate point cloud data.

8. The method for automatic registration of multi-view point clouds for interior scenes of finished houses according to claim 1, characterized in that, The step of performing pairwise registration of instances of multi-station indoor point cloud data to obtain coarsely registered point cloud data includes: 5.1) The pairwise registration algorithm based on the large plane and the pairwise registration algorithm based on the gate are used to register the indoor point cloud data of each station with the subsequent point cloud data in sequence to obtain the corresponding registration score and transformation matrix. The ceiling point cloud data is used as the source point cloud data, and the wall plane point cloud data that is adjacent to the ceiling point cloud data is used as the target point cloud data. The registration score As shown below: （14） In the formula, For set One of the elements; For set One of the elements; , , , , These are evaluation functions for the large plane, window, door, horizontal direction, and vertical direction, respectively. Represents weight, Represents horizontal conflict weights. Represents vertical conflict weights; Among them, the evaluation function of the pairwise registration algorithm based on the large plane. The calculation formula is as follows: （15） In the formula, This represents the total number of overlapping points; The target point cloud point count; The number of points in the source point cloud; Evaluation function of gate-based pairwise registration algorithm The calculation formula is as follows: （16） 5.2) Determine whether the registration score is greater than the preset threshold. If so, define that the two stations corresponding to the registration score have a registration relationship. 5.3) Define each station as a node. The registration relationship between two nodes represents an edge between the two nodes. The weight of the edge is the registration score between the two stations, resulting in one or more connected graphs. 5.4) For each connected graph, the edges are pruned using the maximum spanning tree algorithm, and the node with the maximum proximity centrality is designated as the registration reference station. Other stations are transformed to the local coordinate system of the registration reference station according to the transformation matrix obtained by the shortest transformation path, thus obtaining coarse registration point cloud data. The formula for calculating the proximity centrality is as follows: （17） In the formula, All are nodes; For nodes Proximity to centrality; Represents a node and nodes The shortest distance in a connected graph; Represents a node The number of nodes that can be reached.

Images