A method for extracting line features from building 3D point clouds based on deep learning

Abstract

The present invention discloses a method for extracting line features of building three-dimensional point clouds based on deep learning, including: firstly, using a ground fixed station laser scanner to obtain laser point cloud data of buildings in a survey area, then slicing the acquired three-dimensional point cloud of buildings, projecting the sliced ​​point cloud to a two-dimensional occupancy probability image, and calculating the PCA1 based on the weight center decomposition of each pixel according to the density of the sliced ​​point cloud to generate a sub-image; then manually marking the building contour line and making a training sample data set; then constructing a line feature extraction neural network LALine‑Net, obtaining the neural network model parameters through network training and model evaluation; applying new data to perform model prediction, extracting the line features of the laser point cloud of buildings in the survey area; finally, refitting and optimizing the extracted line features to obtain the final building contour line features. The present invention can quickly extract the urban building contour line from the laser point cloud, has better integrity and geometric accuracy, and can meet the needs of building mapping based on laser three-dimensional point clouds.

Description

Technical Field

[0001] The present invention relates to the fields of remote sensing science and technology and surveying and mapping engineering technology, and in particular to a method for extracting line features of three-dimensional point clouds of buildings based on deep learning. Background Art

[0002] In recent years, laser scanning technology has been widely used in all walks of life. In the field of architectural surveying and mapping, laser point cloud technology provides a powerful means for architectural mapping. Laser point cloud technology can capture every detail of a building and generate high-density three-dimensional point clouds, which can be used to create accurate architectural models. High-precision modeling, structural detection, design assistance and historical building protection can improve the accuracy and efficiency of work and reduce errors and risks. However, the amount of laser scanning point cloud data is large, and line feature extraction is difficult. Manually drawing various types of building plans, building elevations and sections is labor-intensive, time-consuming and labor-intensive. Automated line feature extraction still faces many problems. Due to the missing point cloud data caused by point cloud noise and object occlusion, the existing point cloud line feature automatic extraction method is difficult to ensure the integrity of the extracted line segments, and the geometric accuracy is still low.

[0003] In response to the above problems, the present invention proposes a method for extracting line features of building three-dimensional point clouds based on deep learning. Summary of the invention

[0004] The purpose of the present invention is to solve the shortcomings existing in the prior art, and to propose a method for extracting building three-dimensional point cloud line features based on deep learning, which can realize the extraction based on building contour lines to meet the needs of architectural mapping. It is of great significance to promote the application of drone photogrammetry technology and improve the efficiency and accuracy of urban planning, environmental protection and disaster assessment.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] A method for extracting line features of a three-dimensional point cloud of a building based on deep learning comprises the following steps:

[0007] Step S1: preprocessing of building point cloud data, slicing the building point cloud, projecting the sliced ​​point cloud into an occupancy probability image according to the input pixel size, calculating the PCA1 principal component of each pixel center according to the point cloud density, and generating a PCA1 image;

[0008] Step S2: Create a deep learning training sample dataset, annotate line features according to the two-dimensional image of the projected slice, construct a square fishing net of a specific size for image cutting, segment the raster and vector layers through the fishing net, and create training sample images and annotations, including slicing the occupancy probability image and PCA1 image into small training images, and generating corresponding vector annotation files in JSON format;

[0009] Step S3: In the model training stage, the training sample images and the annotation files are preprocessed, and the training samples are substituted into the line feature extraction neural network LALine-Net for training to obtain the neural network model parameters;

[0010] Step S4: In the model prediction stage, the input laser point cloud data is automatically projected and divided into blocks, the line features are extracted using the line feature extraction neural network LALine-Net, and the initial building outline vector line is obtained through line feature coordinate transformation and fusion;

[0011] Step S5: Line feature refitting and optimization. For the initial building outline vector line, determine the point set that falls on each edge, use the weighted iteration method to fit new straight line parameters, and correct the line segment intersection points based on the adjacency relationship of the line segments.

[0012] Preferably, in step S1, the specific implementation steps of the building point cloud data preprocessing are:

[0013] Step S11: Slice the building point cloud and project the sliced ​​point cloud into an occupancy probability image according to the input pixel size;

[0014] Step S12: traverse each occupied pixel, search for neighbors through the weight center, obtain the feature vector through eigenvalue decomposition, convert it into an angle value as the pixel value, and generate a PCA1 image.

[0015] Preferably, in step S3, the specific implementation steps of model training are:

[0016] Step S31: First, a line feature extraction neural network LALine-Net for line feature extraction of laser point cloud slicing is designed. The line feature extraction neural network LALine-Net includes a backbone network and two branch networks. The backbone network uses U-Net to extract image features. The structure includes 5 encoding structures and 4 decoding structures. Each encoding structure includes 2 convolutional layers and 1 pooling layer, and each decoding structure includes 1 deconvolution and 2 convolutions. The two branch networks take into account global and local features, one of which includes 2 convolutions and 1 normalization layer, and outputs a distance field; the other includes 2 convolutions and 1 fully connected layer, and outputs an angle field; the loss function is the sum of the normalized distance field and the angle field loss;

[0017] Step S32: Use PyTorch to complete the encoding of the deep neural network. The size of the point cloud slice image input by the network is 320*320. After being processed by multiple convolutional layers, the Adam optimization algorithm is used to reduce the loss value. When the loss value is reduced to a given threshold range, the training ends and the optimal line feature extraction neural network LALine-Net model parameters are obtained;

[0018] Step S33: Evaluate the model training results and further improve the accuracy of the model by adjusting training parameters and enhancing training samples.

[0019] Preferably, in step S4, the specific implementation steps of the model prediction are:

[0020] Step S41: automatically completing image projection and block division for the input laser point cloud data, and recording the coordinates of the upper left corner point of each block image;

[0021] Step S42: using the trained line feature extraction neural network LALine-Net to process the image and finally obtain a high-precision line feature extraction result;

[0022] Step S43: Obtain the initial building outline vector line through line feature coordinate transformation and fusion.

[0023] Preferably, in step S5, the specific implementation steps of line feature refitting and optimization are:

[0024] Step S51: for the initial building outline vector line, determine the point set that falls on each edge, and if the distance from the point to the line is less than a given threshold, add the point set PL to the line;

[0025] Step S52: For each line point set PL, use the weighted iteration method to fit new line parameters;

[0026] Step S53: Correct the intersection of the line segments according to the adjacency relationship of the line segments.

[0027] Compared with the prior art, the present invention has the following beneficial effects:

[0028] 1. The present invention can quickly extract the urban building contour lines from the laser point cloud, has better integrity and geometric accuracy, and can meet the needs of building mapping based on laser three-dimensional point cloud.

[0029] 2. The present invention can realize the extraction based on building contours to meet the needs of architectural mapping. It is of great significance to promote the application of drone photogrammetry technology and improve the efficiency and accuracy of urban planning, environmental protection and disaster assessment.

[0030] 3. The present invention can meet the requirements of roof parameter extraction and model construction based on drone images. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 is a flow chart of the present invention;

[0032] Figure 2 This is a schematic diagram of a laser scanning point cloud of a residential building in a city in an embodiment of the present invention;

[0033] Figure 3 This is a schematic diagram of the projection image of the point cloud slice and local details of the present invention;

[0034] Figure 4 A schematic diagram of a training sample image and annotated line elements of the present invention;

[0035] Figure 5 This is a schematic diagram of the structure of the image line feature extraction neural network LALine-Net of the present invention;

[0036] Figure 6 This is a schematic diagram of line feature extraction using a trained deep neural network in the present invention. DETAILED DESCRIPTION

[0037] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings, so that those skilled in the art can better understand the advantages and features of the present invention, thereby making a clearer definition of the protection scope of the present invention. The embodiments described in the present invention are only a part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present invention.

[0038] Reference Figure 1-Figure 6 , a method for extracting line features of building three-dimensional point clouds based on deep learning, comprising the following steps:

[0039] Step S1: preprocessing of building point cloud data, slicing the building point cloud, projecting the sliced ​​point cloud into an occupancy probability image according to the input pixel size, calculating the PCA1 principal component of each pixel center according to the point cloud density, and generating a PCA1 image;

[0040] Step S2: Create a deep learning training sample dataset, annotate line features according to the two-dimensional image of the projected slice, construct a square style (fishnet) of a specific size for image cutting, segment the raster and vector layers through the fishnet, and create training sample images and annotations, including slicing the occupancy probability image and PCA1 image into small training images, and generating corresponding vector annotation files in JSON format;

[0041] Step S3: In the model training stage, the training sample images and the annotation files are preprocessed, and the training samples are substituted into the line feature extraction neural network LALine-Net for training to obtain the neural network model parameters;

[0042] Step S4: In the model prediction stage, the input laser point cloud data is automatically projected and divided into blocks, the line features are extracted using the line feature extraction neural network LALine-Net, and the initial building outline vector line is obtained through line feature coordinate transformation and fusion;

[0043] Step S5: Line feature refitting and optimization. For the initial building outline vector line, determine the point set that falls on each edge, use the weighted iteration method to fit new straight line parameters, and correct the line segment intersection points based on the adjacency relationship of the line segments.

[0044] Specifically, in step S1, the specific implementation steps of building point cloud data preprocessing are:

[0045] Step S11: Figure 2 The input building point cloud data is shown in FIG. The building point cloud is sliced ​​and the sliced ​​point cloud is projected into an occupancy probability image according to the input pixel size, such as Figure 3 As shown;

[0046] Step S12: traverse each occupied pixel, search for neighbors through the weight center, obtain the feature vector through eigenvalue decomposition, convert it into an angle value as the pixel value, and generate a PCA1 image.

[0047] Specifically, in step S2, the specific implementation steps of preparing the training sample data set are:

[0048] Step S21: training sample annotation, annotating line elements according to the two-dimensional image of the projected slice;

[0049] Step S22: creating a deep learning training sample data set and constructing a square grid (fishing net) of a specific size for image cutting;

[0050] Step S23: Create a training sample image by segmenting the grid using a fishnet, and cut the occupancy probability image and the PCA1 image into small training images;

[0051] Step S24: Segment the vector annotation layer through the fishing net to generate a vector annotation file in the corresponding JSON format, such as Figure 4 Shown are tiled images and labeled line features from the training samples.

[0052] Specifically, in step S3, the specific implementation steps of model training are:

[0053] Step S31: First, a line feature extraction neural network LALine-Net for laser point cloud slicing line feature extraction is designed. The structure of the constructed line feature extraction neural network LALine-Net is as follows: Figure 5As shown in the figure, the network includes a backbone network and two branch networks. The backbone network uses U-Net to extract image features. The structure includes 5 encoding structures and 4 decoding structures. Each encoding structure includes 2 convolutional layers and 1 pooling layer. Each decoding structure includes 1 deconvolution and 2 convolutions. The two branch networks take into account both global and local features. One of them includes 2 convolutions and 1 normalization layer, and outputs the distance field; the other includes 2 convolutions and 1 fully connected layer, and outputs the angle field; the loss function is the sum of the normalized distance field and the angle field loss;

[0054] Step S32: The point cloud slice image input by the network has a resolution of 0.015m and an image size of 320*320. After being processed by multiple convolutional layers, the Adam optimization algorithm is used to reduce the loss value. When the loss value is reduced to a given threshold range, the training ends and the optimal line feature extraction neural network model parameters are obtained;

[0055] Step S33: Evaluate the model training results and further improve the accuracy of the model by adjusting the training parameters and enhancing the training samples.

[0056] Specifically, in step S4, the specific implementation steps of the model prediction are:

[0057] Step S41: automatically completing image projection and block division for the input laser point cloud data, and recording the coordinates of the upper left corner point of each block image;

[0058] Step S42: Process the image using the trained line feature extraction neural network LALine-Net. Figure 6 As shown, finally the high-precision line feature extraction result is obtained;

[0059] Step S43: Obtain the initial building outline vector line through line feature coordinate transformation and fusion.

[0060] Specifically, in step S5, the specific implementation steps of line feature refitting and optimization are:

[0061] Step S51: for the initial building outline vector line, determine the point set that falls on each edge, and if the distance from the point to the line is less than a given threshold, add the point set PL to the line;

[0062] Step S52: For each line point set PL, use the weighted iteration method to fit new line parameters;

[0063] Step S53: Correct the intersection of the line segments according to the adjacency relationship of the line segments.

[0064] In summary, the present invention can realize the extraction based on building contours to meet the needs of architectural mapping, which is of great significance for promoting the application of drone photogrammetry technology and improving the efficiency and accuracy of urban planning, environmental protection and disaster assessment.

[0065] The description and practice disclosed in the present invention are easy to think and understand for ordinary technicians in the technical field, and several improvements and modifications can be made without departing from the principles of the present invention. Therefore, modifications or improvements made without departing from the spirit of the present invention should also be regarded as the protection scope of the present invention.

Claims

1. A method for extracting line features from a three-dimensional point cloud of a building based on deep learning, characterized in that: The following steps are involved: Step S1: preprocessing of building point cloud data, slicing the building point cloud, projecting the sliced ​​point cloud into an occupancy probability image according to the input pixel size, calculating the PCA1 principal component of each pixel center according to the point cloud density, and generating a PCA1 image; Step S2: Create a deep learning training sample dataset, annotate line features according to the two-dimensional image of the projected slice, construct a square fishing net of a specific size for image cutting, segment the raster and vector layers through the fishing net, and create training sample images and annotations, including slicing the occupancy probability image and PCA1 image into small training images, and generating corresponding vector annotation files in JSON format; Step S3: In the model training stage, the training sample images and the annotation files are preprocessed, and the training samples are substituted into the line feature extraction neural network LALine-Net for training to obtain the neural network model parameters; Step S4: In the model prediction stage, the input laser point cloud data is automatically projected and divided into blocks, the line features are extracted using the line feature extraction neural network LALine-Net, and the initial building outline vector line is obtained through line feature coordinate transformation and fusion; Step S5: Line feature refitting and optimization: for the initial building outline vector line, determine the point set that falls on each edge, use the weighted iteration method to fit new line parameters, and correct the line segment intersection points according to the adjacency relationship of the line segments; In step S1, the specific implementation steps of building point cloud data preprocessing are: Step S11: Slice the building point cloud and project the sliced ​​point cloud into an occupancy probability image according to the input pixel size; Step S12: traverse each occupied pixel, search for neighbors through the weight center, obtain the feature vector through eigenvalue decomposition, convert it into an angle value as the pixel value, and generate a PCA1 image.

2. The method for extracting line features of a three-dimensional point cloud of a building based on deep learning according to claim 1, characterized in that: In step S3, the specific implementation steps of model training are: Step S31: First, a line feature extraction neural network LALine-Net for line feature extraction of laser point cloud slicing is designed. The line feature extraction neural network LALine-Net includes a backbone network and two branch networks. The backbone network uses U-Net to extract image features. The structure includes 5 encoding structures and 4 decoding structures. Each encoding structure includes 2 convolutional layers and 1 pooling layer, and each decoding structure includes 1 deconvolution and 2 convolutions. The two branch networks take into account global and local features, one of which includes 2 convolutions and 1 normalization layer, and outputs a distance field; the other includes 2 convolutions and 1 fully connected layer, and outputs an angle field; the loss function is the sum of the normalized distance field and the angle field loss; Step S32: Use PyTorch to complete the encoding of the deep neural network. The size of the point cloud slice image input by the network is 320*320. After being processed by multiple convolutional layers, the Adam optimization algorithm is used to reduce the loss value. When the loss value is reduced to a given threshold range, the training ends and the optimal line feature extraction neural network LALine-Net model parameters are obtained; Step S33: Evaluate the model training results and further improve the accuracy of the model by adjusting training parameters and enhancing training samples.

3. The method for extracting line features of a three-dimensional point cloud of a building based on deep learning according to claim 1, characterized in that: In step S4, the specific implementation steps of the model prediction are: Step S41: automatically completing image projection and block division for the input laser point cloud data, and recording the coordinates of the upper left corner point of each block image; Step S42: using the trained line feature extraction neural network LALine-Net to process the image and finally obtain a high-precision line feature extraction result; Step S43: Obtain the initial building outline vector line through line feature coordinate transformation and fusion.

4. The method for extracting line features of a three-dimensional point cloud of a building based on deep learning according to claim 1, characterized in that: In step S5, the specific implementation steps of line feature refitting and optimization are as follows: Step S51: for the initial building outline vector line, determine the point set that falls on each edge, and if the distance from the point to the line is less than a given threshold, add the point set PL to the line; Step S52: For each line point set PL, use the weighted iteration method to fit new line parameters; Step S53: Correct the intersection of the line segments according to the adjacency relationship of the line segments.

Images