A semantic segmentation method for outdoor scenes based on RandLA-Net based on local feature enhancement

Abstract

The present invention discloses a RandLA‑Net outdoor scene semantic segmentation method based on local feature enhancement. First, the spatial geometric features of the point cloud are extracted by encoding the coordinates, relative positions and normal vector direction changes of the center point and the neighboring points, and the semantic features are extracted by stacking the center point features and the neighborhood point features, and the geometric features and the semantic features are connected to obtain the enhanced features. Secondly, the weight of each feature is learned through the attention mechanism, the weight is multiplied by the feature and the weighted features are summed to obtain the aggregated features. Then, the receptive field of each point is increased by expanding the residual block, and the network segmentation accuracy is improved. Finally, the network structure of the encoding layer and the decoding layer is used to realize the semantic segmentation of the outdoor scene. The present invention has a good segmentation effect for large-scale scenes, has a certain improvement in segmentation accuracy, and has good robustness.

Description

Technical Field

[0001] The invention belongs to the technical field of point cloud scene segmentation and relates to a RandLA-Net outdoor scene semantic segmentation method based on local feature enhancement. Background Art

[0002] 3D scene understanding and analysis has always been an important research topic in the field of computer vision. In recent years, with the increasing maturity of 3D data acquisition technologies such as 3D laser technology and depth cameras, 3D point cloud processing methods have also developed rapidly. Among them, semantic segmentation of point clouds, as the core technology for point cloud data processing and analysis, is of great significance for the mining of deeper spatial information. It plays an important role in the fields of autonomous driving, intelligent robots, medical image segmentation, etc., and has broad application prospects.

[0003] Point cloud scene semantic segmentation refers to dividing the scene into several areas with different semantic categories by extracting point cloud features. For large-scale outdoor scene data, it takes a long time to divide the scene using traditional semantic segmentation methods, and the segmentation efficiency is low. With the development of computer hardware storage capacity and GPU computing power, scene semantic segmentation methods based on deep learning have become more mature. PointNet is the first network that directly processes point cloud classification and segmentation. Subsequently, many people have made improvements in different directions based on this network, but these improved networks are limited to the semantic segmentation of small-scale scenes. In 2020, the introduction of the RandLA-Net network specifically for semantic segmentation of large outdoor scenes broke this limitation.

[0004] The RandLA-Net network uses two modules, random sampling and local feature aggregation, for semantic segmentation of point clouds, which reduces the computation time of deep learning point cloud processing and ensures the extraction of local features, achieving significant segmentation results in large scene semantic segmentation. Although the network has a good representation ability for local features, it does not consider the normal vector relationship between a point and its neighboring points, resulting in insufficient local feature extraction. Summary of the invention

[0005] The purpose of the present invention is to provide a RandLA-Net outdoor scene semantic segmentation method based on local feature enhancement, which solves the problem of low semantic segmentation accuracy caused by insufficient local features of the RandLA-Net network.

[0006] The technical solution adopted by the present invention is a RandLA-Net outdoor scene semantic segmentation method based on local feature enhancement, which is specifically implemented according to the following steps:

[0007] Step 1, data preprocessing: input the original point cloud data, use the voxel grid filtering method to downsample the original scene to obtain the preprocessed point cloud data;

[0008] Step 2, normal vector calculation: use principal component analysis method to estimate the surface normal vector of the preprocessed point cloud;

[0009] Step 3, data set encapsulation: the original point cloud includes seven-dimensional information: x, y, z, r, g, b, cls, where x, y, z represent the three-dimensional coordinate information of the point cloud, r, g, b represent the color information of the point cloud, and cls represents the category information of the point cloud. The preprocessed point cloud surface normal vector is added to the preprocessed point cloud data, saved as a ply format file, and a kdtree file and a projection file are generated;

[0010] Step 4, optimize the RandLA-Net network encoder and perform feature extraction: input the encapsulated ply format point cloud data into the optimized RandLA-Net network encoder to extract point cloud features;

[0011] Step 5, use the RandLA-Net network decoder for feature mapping: input the extracted point cloud features into the RandLA-Net network decoder, map the point cloud features to each point to obtain the mapped point cloud features;

[0012] Step 6, semantic segmentation: The mapped point cloud features are processed through three fully connected layers and one dropout layer of the RandLA-Net network to obtain the category score of each point cloud, realizing large-scale scene semantic segmentation.

[0013] The present invention is also characterized in that:

[0014] Step 1 is as follows:

[0015] Step 1.1, calculate the center point of each voxel grid

[0016] Draw a voxel grid on the original point cloud data, assuming that the current input point is p i (x i ,y i ,z i ), the voxel grid side length L grid , then the center point coordinates W(X cen ,Y cen ,Z cen )for,

[0017]

[0018] The position of each voxel grid in space is obtained according to the side length of the voxel grid and the coordinates of the center point of the voxel grid;

[0019] Step 1.2, calculate the centroid of each voxel grid

[0020] Assuming that the centroid of the grid is G(X,Y,Z), then for a voxel grid containing n points, the centroid of each voxel grid is expressed as:

[0021]

[0022] Step 1.3, the centroid point of each voxel grid replaces all points in the corresponding voxel grid, and the remaining points other than the centroid point are filtered out to obtain the preprocessed point cloud data.

[0023] Step 2 is as follows:

[0024] Step 2.1, first establish a kd tree, organize the preprocessed point cloud into ordered points, and find the k nearest neighbors of each point;

[0025] Step 2.2: According to the least squares principle, the k-neighborhood points of each point are fitted into a local plane P. The plane formula is shown in formula (3):

[0026]

[0027] In formula (3), represents the normal vector of the local fitting plane P, p k represents any neighboring point of point p, and d represents the distance from the fitting plane P to the origin of the coordinate system;

[0028] Step 2.3, calculate the covariance matrix M:

[0029]

[0030] In formula (4), M represents the 3*3 covariance matrix, p k represents the current point, p0 represents the centroid of the k-neighborhood points of point p;

[0031] Step 2.4, use singular value decomposition to perform eigenvalue decomposition on the covariance matrix M, and according to the relationship between the matrix and the eigenvector, separate M and The relationship between them is expressed as:

[0032]

[0033] In formula (5), λ represents the eigenvalue matrix corresponding to the matrix M, It represents the eigenvector matrix corresponding to the eigenvalue matrix. The eigenvalue matrix consists of three eigenvalues. When the eigenvalue is the smallest, the corresponding eigenvector is the surface normal vector of the preprocessed point cloud.

[0034] Step 3 is as follows:

[0035] Step 3.1, add the preprocessed point cloud surface normal vector to the preprocessed point cloud data and encapsulate it into x, y, z, r, g, b, n x ,n y ,n z ,cls is a ply format file with ten-dimensional features, and is divided into three parts for training data, verification data, and test data, where n x 、n y 、n z Respectively represent the surface normal vector information of the point cloud;

[0036] Step 3.2, when training the network, search for N points around each point as an input sample. To facilitate the search of the nearest points, use step 2.1 to build the kd tree and save it in binary form to the file "point cloud name_kdtree.pkl", i.e., the kdtree file.

[0037] In step 3.3, the serial number of the nearest point of the downsampled point in the original point cloud is stored as a projection file.

[0038] Step 4 is as follows:

[0039] Step 4.1, optimize the RandLA-Net network encoder into 5 encoding layers, each encoding layer includes a random sampling module and a dilated residual block, the dilated residual block includes two local feature extraction modules, two feature aggregation modules, point coordinates after data encapsulation and ten-dimensional point features, and the feature dimensions of each encoding layer output are [32, 128, 256, 512, 1024];

[0040] Step 4.2, local feature extraction module extracts local features: explicitly encodes the relative position of each point in space after preprocessing, obtains the geometric features of the point cloud space, convolves the features of each point in the point cloud with the features of its neighboring points, extracts the semantic features of the point cloud, connects the geometric features and semantic features of the point cloud, and obtains the local features of the point;

[0041] Step 4.3, feature aggregation module extracts global features: a shared function consisting of a multi-layer perceptron MLP network and softmax is used to learn the weight corresponding to the local features of each point, i.e., the attention score. The attention score of the local feature of each point is multiplied by the corresponding local feature. Finally, all weighted features are summed to obtain the global features of the aggregated points.

[0042] Step 4.4, dilate the residual block to increase the receptive field of the point: stack the two local feature extraction modules, the two feature aggregation modules, the coordinates of the point after data encapsulation, and the ten-dimensional features together to form an expanded residual block, increase the global feature dimension of the point, extract the global features of the deep points, and obtain the point cloud features.

[0043] Step 4.2 is as follows:

[0044] Step 4.2.1, take the preprocessed point as the center point and encode the relative position of the point cloud: by splicing the three-dimensional coordinates of the center point, the three-dimensional coordinates of the neighborhood points, the relative coordinates of the center point and the neighborhood points, the Euclidean distance between the center point and the neighborhood points, and the angle θ between the normal vector of the center point and the neighborhood points, use the MLP multi-layer perceptron to extract high-dimensional features and obtain the geometric features f of the point cloud space Gi ;

[0045] The angle θ is calculated as shown in (6):

[0046]

[0047] Geometric characteristics Gi The encoding formula is shown in (7)

[0048]

[0049] In formula (7), p i represents the center point coordinates, represents the coordinates of the neighborhood points, represents the Euclidean distance between the center point and the neighboring points, It is a concatenation operation;

[0050] Step 4.2.2, the ten-dimensional features of the center point x, y, z, r, g, b, n x ,n y ,n z ,cls obtains the center point feature f through MLP c , according to the neighborhood point index obtained by the K nearest neighbor algorithm, the neighborhood point features corresponding to each center point are further obtained The center point feature is convolved with the neighborhood point feature to obtain the spatial semantic feature f Si , f Si The encoding formula is shown in (8):

[0051]

[0052] Step 4.2.3, connect the geometric features obtained in step 4.2.1 with the semantic features obtained in step 4.2.2 to obtain the enhanced local features f i k , as shown in formula (9),

[0053]

[0054] Finally, a new set of local features is obtained

[0055] Step 4.3 is as follows:

[0056] Step 4.3.1, learn the local features f of the point through a shared function g i k The corresponding weight, i.e. the attention score The shared function g is composed of MLP and softmax, as shown in formula (10). In formula (10), W is the learnable parameter of the shared MLP;

[0057]

[0058] Step 4.3.2: The attention score The corresponding local feature f i k Multiply and sum the K weighted features to get the global feature of the point, as shown in the following formula:

[0059]

[0060] Step 5 is as follows:

[0061] The point cloud features are mapped using the RandLA-Net network decoder, each decoder includes 5 decoding layers, each decoding layer includes an upsampling module and an MLP, the upsampling module upsamples the center point and extracts the corresponding point cloud features through the MLP, and maps the point cloud features to each point to obtain the mapped point cloud features. The beneficial effects of the present invention are:

[0062] The present invention discloses a RandLA-Net outdoor scene semantic segmentation method based on local feature enhancement, improves RandLA-Net from two perspectives of data processing and network framework, optimizes the problem of semantic segmentation error caused by insufficient local feature extraction of the RandLA-Net network, more efficiently extracts deep features of point clouds, has good segmentation effect for large-scale outdoor scenes, has certain improvements in effectiveness and accuracy, has good robustness, and enriches the method system of deep learning for semantic segmentation in large-scale point cloud scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 This is a network architecture diagram of the RandLA-Net outdoor scene semantic segmentation method based on local feature enhancement;

[0064] Figure 2 It is an organizational chart of the newly designed local feature extraction module of the RandLA-Net outdoor scene semantic segmentation method based on local feature enhancement;

[0065] Figure 3 It is an organizational chart of the feature aggregation module of the RandLA-Net outdoor scene semantic segmentation method based on local feature enhancement;

[0066] Figure 4 It is a dilated residual block organization diagram of the RandLA-Net outdoor scene semantic segmentation method based on local feature enhancement. DETAILED DESCRIPTION

[0067] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0068] Example

[0069] This embodiment provides a RandLA-Net outdoor scene semantic segmentation method based on local feature enhancement. Figure 1 As shown, the specific steps are as follows:

[0070] Step 1, data preprocessing: input the original point cloud data, use the voxel grid filtering method to downsample the original scene to obtain the preprocessed point cloud data;

[0071] Step 1.1, calculate the center point of each voxel grid

[0072] Draw a voxel grid on the original point cloud data, assuming that the current input point is p i (x i ,y i ,z i ), the voxel grid side length L grid , then the center point coordinates W(X cen ,Y cen ,Z cen )for,

[0073]

[0074] The position of each voxel grid in space is obtained according to the side length of the voxel grid and the coordinates of the center point of the voxel grid;

[0075] Step 1.2, calculate the centroid of each voxel grid

[0076] Assuming that the centroid of the grid is G(X,Y,Z), then for a voxel grid containing n points, the centroid of each voxel grid is expressed as:

[0077]

[0078] Step 1.3, the centroid point of each voxel grid replaces all points in the corresponding voxel grid, and the remaining points other than the centroid point are filtered out to obtain the preprocessed point cloud data.

[0079] Step 2, normal vector calculation: use the principal component analysis (PCA) method to estimate the surface normal vector of the preprocessed point cloud;

[0080] Step 2.1, first establish a kd tree (K-dimension tree, kd-tree), organize the preprocessed point cloud into ordered points, and find the k nearest neighbors of each point;

[0081] Step 2.2: According to the least squares principle, the k-neighborhood points of each point are fitted into a local plane P. The plane formula is shown in formula (3):

[0082]

[0083] In formula (3), represents the normal vector of the local fitting plane P, p k represents any neighboring point of point p, and d represents the distance from the fitting plane P to the origin of the coordinate system;

[0084] Step 2.3, calculate the covariance matrix M:

[0085]

[0086] In formula (4), M represents the 3*3 covariance matrix, p k represents the current point, p0 represents the centroid of the k-neighborhood points of point p;

[0087] Step 2.4, use singular value decomposition (SVD) to decompose the covariance matrix M into eigenvalues, and according to the relationship between the matrix and the eigenvector, decompose M and The relationship between them is expressed as:

[0088]

[0089] In formula (5), λ represents the eigenvalue matrix corresponding to the matrix M, It represents the eigenvector matrix corresponding to the eigenvalue matrix. The eigenvalue matrix consists of three eigenvalues. When the eigenvalue is the smallest, the corresponding eigenvector is the surface normal vector of the preprocessed point cloud.

[0090] Step 3, data set encapsulation: the original point cloud includes seven-dimensional information: x, y, z, r, g, b, cls, where x, y, z represent the three-dimensional coordinate information of the point cloud, r, g, b represent the color information of the point cloud, and cls represents the category information of the point cloud. The preprocessed point cloud surface normal vector is added to the preprocessed point cloud data, saved as a ply format file, and a kdtree file and a projection file are generated;

[0091] Step 3.1, add the preprocessed point cloud surface normal vector to the preprocessed point cloud data and encapsulate it into x, y, z, r, g, b, n x ,n y ,n z ,cls is a ply format file with ten-dimensional features, and is divided into three parts for training data, verification data, and test data, where n x 、n y 、n z Respectively represent the surface normal vector information of the point cloud;

[0092] Step 3.2, when training the network, instead of inputting all preprocessed point cloud data into the network at once, we search for N points around each point as input samples. To facilitate the search of the nearest points, we build a kd tree using step 2.1 and save it in binary form to the file "point cloud name_kdtree.pkl", i.e., the kdtree file.

[0093] Step 3.3, since the data has been pre-processed by downsampling, the semantic labels obtained by the point cloud semantic segmentation network model are the processed point cloud data. These semantic labels need to be restored to the original point cloud. At this time, a projection file is needed. Therefore, the serial number of the nearest point to the downsampled point in the original point cloud is stored as a projection file.

[0094] Step 4, optimize the RandLA-Net network encoder and perform feature extraction: input the encapsulated ply format point cloud data into the optimized RandLA-Net network encoder to extract point cloud features;

[0095] Step 4.1, optimize the RandLA-Net network encoder into 5 encoding layers, each encoding layer includes a random sampling module and an expansion residual block, and the expansion residual block includes two local feature extraction modules, two feature aggregation modules, point coordinates after data encapsulation, and ten-dimensional point features. The feature dimensions of each encoding layer output are [32, 128, 256, 512, 1024];

[0096] Step 4.2: local feature extraction module extracts local features: Figure 2As shown in the figure, the relative position of each point in the space after preprocessing is explicitly encoded to obtain the geometric features of the point cloud space, the features of each point in the point cloud are convolved with the features of its neighborhood points to extract the semantic features of the point cloud, and the geometric features and semantic features of the point cloud are connected to obtain the local features of the point;

[0097] Step 4.2.1, take the preprocessed point as the center point and encode the relative position of the point cloud: by splicing the three-dimensional coordinates of the center point, the three-dimensional coordinates of the neighborhood points, the relative coordinates of the center point and the neighborhood points, the Euclidean distance between the center point and the neighborhood points, and the angle θ between the normal vector of the center point and the neighborhood points, use the MLP multi-layer perceptron to extract high-dimensional features and obtain the geometric features f of the point cloud space Gi ;

[0098] The angle θ is calculated as shown in (6):

[0099]

[0100] Geometric characteristics Gi The encoding formula is shown in (7)

[0101]

[0102] In formula (7), p i represents the center point coordinates, represents the coordinates of the neighborhood points, represents the Euclidean distance between the center point and the neighboring points, It is a concatenation operation;

[0103] Step 4.2.2, the ten-dimensional features of the center point x, y, z, r, g, b, n x ,n y ,n z ,cls obtains the center point feature f through MLP c , according to the neighborhood point index obtained by the K-NearestNeighbor (KNN) algorithm, the neighborhood point features corresponding to each center point are further obtained The center point feature is convolved with the neighborhood point feature to obtain the spatial semantic feature f Si , f Si The encoding formula is shown in (8):

[0104]

[0105] Step 4.2.3, connect the geometric features obtained in step 4.2.1 with the semantic features obtained in step 4.2.2 to obtain the enhanced local features f i k , as shown in formula (9),

[0106]

[0107] Finally, a new set of local features is obtained

[0108] Step 4.3, the feature aggregation module extracts global features: Figure 3 As shown in the figure, a shared function composed of a multilayer perceptron (MLP) network and softmax is used to learn the weight corresponding to the local feature of each point, that is, the attention score. The attention score of the local feature of each point is multiplied by the corresponding local feature. Finally, all weighted features are summed to obtain the global feature of the aggregated point.

[0109] Step 4.3.1, learn the local features f of the point through a shared function g i k The corresponding weight, i.e. the attention score The shared function g is composed of MLP and softmax, as shown in formula (10). In formula (10), W is the learnable parameter of the shared MLP;

[0110]

[0111] Step 4.3.2: The attention score The corresponding local feature f i k Multiply and sum the K weighted features to get the global feature of the point, as shown in the following formula:

[0112]

[0113] Step 4.4, dilate the residual block to increase the receptive field of the point: large-scale point cloud data will undergo continuous random downsampling at each layer, resulting in the random discarding of features of some points. Therefore, it is very necessary to increase the receptive field of each point to ensure that the overall geometric details of the input point cloud can be preserved, such as Figure 4 As shown in the figure, two local feature extraction modules, two feature aggregation modules, the coordinates of the points after data encapsulation, and the ten-dimensional features are stacked together to form an expanded residual block, which increases the global feature dimension of the points, extracts the global features of deep points, and obtains point cloud features.

[0114] Step 5, use the RandLA-Net network decoder for feature mapping: input the extracted point cloud features into the RandLA-Net network decoder, map the point cloud features to each point to obtain the mapped point cloud features;

[0115] Step 5 is as follows:

[0116] The RandLA-Net network decoder is used to map the point cloud features. Each decoder includes 5 decoding layers. Each decoding layer includes an upsampling module and an MLP. The upsampling module upsamples the center point and extracts the corresponding point cloud features through the MLP, and maps the point cloud features to each point to obtain the mapped point cloud features.

[0117] Step 6, semantic segmentation: The mapped point cloud features are processed through three fully connected layers and one dropout layer of the RandLA-Net network to obtain the category score of each point cloud, realizing large-scale scene semantic segmentation.

[0118] It can be seen from the above method that the present invention is a RandLA-Net outdoor scene semantic segmentation method based on local feature enhancement. First, the spatial geometric features of the point cloud are extracted by encoding the coordinates, relative positions and normal vector direction changes of the center point and the neighboring points, and the semantic features are extracted by stacking the center point features and the neighborhood point features, and the geometric features and the semantic features are connected to obtain the enhanced features. Secondly, the weight of each feature is learned by the attention mechanism, the weight is multiplied by the feature and the weighted features are summed to obtain the aggregated features. Then, the receptive field of the point is increased by expanding the residual block to improve the network segmentation accuracy. Finally, the network structure of the encoding layer-decoding layer is used to realize the semantic segmentation of the outdoor scene. The present invention adopts an end-to-end method, the technical route is complete, the segmentation accuracy has been improved to a certain extent, and it has good robustness, enriching the method system of deep learning for semantic segmentation in large-scale point cloud scenes.

Claims

1. A RandLA-Net outdoor scene semantic segmentation method based on local feature enhancement, characterized in that: The specific steps are as follows: Step 1, data preprocessing: input the original point cloud data, use the voxel grid filtering method to downsample the original scene to obtain the preprocessed point cloud data; Step 2, normal vector calculation: use principal component analysis method to estimate the surface normal vector of the preprocessed point cloud; Step 3, Dataset Packaging: The original point cloud includes Seven-dimensional information, among which, Respectively represent the three-dimensional coordinate information of the point cloud, Represents the color information of the point cloud, Indicates the category information of the point cloud, adds the preprocessed point cloud surface normal vector to the preprocessed point cloud data, saves it into a ply format file, and generates a kdtree file and a projection file; Step 4, optimize the RandLA-Net network encoder and perform feature extraction: input the encapsulated ply format point cloud data into the optimized RandLA-Net network encoder to extract point cloud features; The step 4 is specifically as follows: Step 4.1, optimize the RandLA-Net network encoder into 5 encoding layers, each encoding layer includes a random sampling module and a dilated residual block, the dilated residual block includes two local feature extraction modules, two feature aggregation modules, point coordinates after data encapsulation and ten-dimensional point features, and the feature dimensions of each encoding layer output are [32, 128, 256, 512, 1024]; Step 4.2, local feature extraction module extracts local features: explicitly encodes the relative position of each point in space after preprocessing, obtains the geometric features of the point cloud space, convolves the features of each point in the point cloud with the features of its neighboring points, extracts the semantic features of the point cloud, connects the geometric features and semantic features of the point cloud, and obtains the local features of the point; Step 4.3, feature aggregation module extracts global features: a shared function consisting of a multi-layer perceptron MLP network and softmax is used to learn the weight corresponding to the local features of each point, i.e., the attention score. The attention score of the local feature of each point is multiplied by the corresponding local feature. Finally, all weighted features are summed to obtain the global features of the aggregated points. Step 4.4, dilate the residual block to increase the receptive field of the point: stack the two local feature extraction modules, the two feature aggregation modules, the coordinates of the point after data encapsulation, and the ten-dimensional features together to form an dilated residual block, increase the global feature dimension of the point, extract the global features of the deep point, and obtain the point cloud features; Step 5, use the RandLA-Net network decoder for feature mapping: input the extracted point cloud features into the RandLA-Net network decoder, map the point cloud features to each point to obtain the mapped point cloud features; Step 6, semantic segmentation: The mapped point cloud features are processed through three fully connected layers and one dropout layer of the RandLA-Net network to obtain the category score of each point cloud, realizing large-scale scene semantic segmentation.

2. According to claim 1, a RandLA-Net outdoor scene semantic segmentation method based on local feature enhancement is characterized in that: The step 1 is specifically as follows: Step 1.1, calculate the center point of each voxel grid Draw a voxel grid on the original point cloud data, assuming that the current input point is , the voxel grid side length , then the center point coordinates of the voxel grid for, (1) The position of each voxel grid in space is obtained according to the side length of the voxel grid and the coordinates of the center point of the voxel grid; Step 1.2, calculate the centroid of each voxel grid Assume that the center of gravity of the grid is , then for The centroid of each voxel grid is represented by: (2)； Step 1.3, the centroid point of each voxel grid replaces all points in the corresponding voxel grid, and the remaining points other than the centroid point are filtered out to obtain the preprocessed point cloud data.

3. According to claim 2, a RandLA-Net outdoor scene semantic segmentation method based on local feature enhancement is characterized in that: The step 2 is specifically as follows: Step 2.1, first establish a kd tree, organize the preprocessed point cloud into ordered points, and find the Neighboring points; Step 2.2, according to the least squares principle, Neighborhood points are fitted into a local plane , the plane formula is shown in formula (3): (3) In formula (3), Represents the local fitting plane The normal vector of Indicate point Any neighbor point of Represents the fitting plane The distance to the origin of the coordinate system; Step 2.3, calculate the covariance matrix : (4) In formula (4), represents the 3*3 covariance matrix, Indicates the current point, Indicate point of The centroid of the neighborhood points; Step 2.4, use singular value decomposition to decompose the covariance matrix Perform eigenvalue decomposition and, based on the relationship between the matrix and the eigenvector, and The relationship between them is expressed as: (5) In formula (5), Representation Matrix The corresponding eigenvalue matrix is, It represents the eigenvector matrix corresponding to the eigenvalue matrix. The eigenvalue matrix consists of three eigenvalues. When the eigenvalue is the smallest, the corresponding eigenvector is the surface normal vector of the preprocessed point cloud.

4. According to claim 3, a RandLA-Net outdoor scene semantic segmentation method based on local feature enhancement is characterized in that: The step 3 is specifically as follows: Step 3.1, add the preprocessed point cloud surface normal vector to the preprocessed point cloud data and encapsulate it into The ply format file of ten-dimensional features is divided into three parts for training data, verification data and test data. Respectively represent the surface normal vector information of the point cloud; Step 3.2, when training the network, search for N points around each point as an input sample. To facilitate the search of the nearest points, use step 2.1 to build the kd tree and save it in binary form to the file "point cloud name_kdtree.pkl", i.e., the kdtree file. In step 3.3, the serial number of the nearest point of the downsampled point in the original point cloud is stored as a projection file.

5. According to claim 1, a RandLA-Net outdoor scene semantic segmentation method based on local feature enhancement is characterized in that: The step 4.2 is specifically as follows: Step 4.2.1, take the preprocessed point as the center point and encode the relative position of the point cloud: by splicing the 3D coordinates of the center point, the 3D coordinates of the neighborhood points, the relative coordinates of the center point and the neighborhood points, the Euclidean distance between the center point and the neighborhood points, and the angle between the normal vector of the center point and the neighborhood points , use MLP multi-layer perceptron to extract high-dimensional features and obtain the geometric features of point cloud space ; Among them, the angle The calculation formula is shown in (6): (6) Geometric features The encoding formula is shown in (7) (7) In formula (7), represents the center point coordinates, represents the coordinates of the neighborhood points, represents the Euclidean distance between the center point and the neighboring points, It is a concatenation operation; Step 4.2.2, the ten-dimensional features of the center point Get the center point feature through MLP , according to the neighborhood point index obtained by the K nearest neighbor algorithm, the neighborhood point features corresponding to each center point are further obtained , convolve the center point features with the neighborhood point features to obtain the spatial semantic features , The encoding formula is shown in (8): (8) Step 4.2.3, connect the geometric features obtained in step 4.2.1 with the semantic features obtained in step 4.2.2 to obtain the enhanced local features , as shown in formula (9), (9) Finally, a new set of local features is obtained .

6. The RandLA-Net outdoor scene semantic segmentation method based on local feature enhancement according to claim 5, characterized in that: The step 4.3 is specifically as follows: Step 4.3.1: Learn the local features of the points through a shared function g The corresponding weight, i.e. the attention score , the shared function g is composed of MLP and softmax, as shown in formula (10), where W is the learnable parameter of the shared MLP; (10)； Step 4.3.2: The attention score The corresponding local features Multiply and sum the K weighted features to get the global feature of the point, as shown in the following formula: (11)。 7. The RandLA-Net outdoor scene semantic segmentation method based on local feature enhancement according to claim 6 is characterized in that: The step 5 is specifically as follows: The RandLA-Net network decoder is used to map the point cloud features. Each decoder includes 5 decoding layers. Each decoding layer includes an upsampling module and an MLP. The upsampling module upsamples the center point and extracts the corresponding point cloud features through the MLP, and maps the point cloud features to each point to obtain the mapped point cloud features.

Images