Three-dimensional point cloud data segmentation method and system based on multi-scale point features

Abstract

The invention provides a three-dimensional point cloud data segmentation method and system based on multi-scale point features, and relates to the technical field of point cloud processing, and the method comprises the steps: obtaining original point cloud data, constructing a three-dimensional Hash index table, carrying out the self-adaptive hierarchical division through a local maximum value of a density distribution curve, and obtaining a local maximum value of a density distribution curve; noise point elimination is carried out based on a region density count value and a geometric distribution coefficient, an enhanced feature vector containing a main feature direction, a main curvature value and the like is constructed for non-eliminated points, a local geometric descriptor is constructed by adopting four spherical search regions with increasing radiuses, a three-branch space attention network is input to extract spatial features, and the spatial features are extracted; and fused feature data are obtained through a multi-level geometric feature extraction network and a bidirectional feature propagation network. And obtaining a final segmentation result through segmentation energy function optimization and a region merging strategy.

Description

Technical Field

[0001] The present invention relates to the technical field of point cloud processing, and in particular to a three-dimensional point cloud data segmentation method and system based on multi-scale point features. Background Art

[0002] With the rapid development and popularization of 3D scanning technology, point cloud data has become an important data form for describing and analyzing 3D objects. Point cloud is a discrete sampling set composed of a large number of points with 3D coordinates, which is obtained through laser scanning, depth camera, multi-view stereo matching, etc. Compared with 2D images, point cloud data can truly record the 3D spatial information of objects.

[0003] Since point cloud data lacks regular topological structure and semantic information, directly analyzing and processing raw point clouds faces many challenges. Point cloud segmentation is one of the key tasks in point cloud processing, which aims to divide the point cloud into multiple non-intersecting subsets so that the points in the same subset belong to the same physical entity or have the same semantic attributes.

[0004] Traditional point cloud segmentation methods are mainly based on geometric features and clustering algorithms, which have problems such as insensitivity to point cloud density and distribution, lack of multi-scale feature representation, and susceptibility to noise and outlier interference;

[0005] Therefore, a solution is urgently needed to solve the problems existing in the prior art. Summary of the invention

[0006] The embodiments of the present invention provide a three-dimensional point cloud data segmentation method and system based on multi-scale point features, which can at least solve some of the problems existing in the prior art.

[0007] A first aspect of an embodiment of the present invention provides a three-dimensional point cloud data segmentation method based on multi-scale point features, comprising:

[0008] Obtain original point cloud data, construct a three-dimensional hash index table and map the original point cloud data to the three-dimensional hash index table, calculate the neighborhood density value of the point in the original point cloud data and construct a point cloud density histogram, use the least squares method to fit the point cloud density histogram to obtain a density distribution curve, extract the local maximum value in the density distribution curve as a point cloud stratification reference value, adaptively stratify the original point cloud data based on the point cloud stratification reference value to obtain a point cloud sub-region set, calculate the regional density count value and geometric distribution coefficient of each point in the point cloud sub-region set, identify and eliminate noise points in the point cloud sub-region based on a pre-set regional density threshold and geometric distribution coefficient threshold, and for points that are not eliminated, construct an enhanced feature vector by calculating the main feature direction, the main curvature value, the Gaussian curvature value and the shape index, and determine the point cloud boundary feature by calculating the local surface direction entropy value;

[0009] For each point in the enhanced feature vector, four spherical search areas with radii that are doubly increasing in sequence are constructed, a local geometric descriptor is constructed by calculating the eigenvalues ​​of the covariance matrix of the points in the spherical search area, the local geometric descriptor and the point cloud boundary features are input into a three-branch spatial attention network, the spatial distribution information, neighborhood structure information and boundary shape information of the points are extracted respectively, and feature weighted fusion is performed through a self-attention mechanism to obtain spatial features, the spatial features are added to a multi-level geometric feature extraction network, the curvature change features, normal vector change features and shape change features are extracted in sequence, multi-scale feature fusion is performed using a feature pyramid structure to obtain geometric features, a bidirectional feature propagation network is constructed to calculate the temporal dependency of the geometric features to obtain propagation features, and the propagation features are jump-connected and residual-connected with the enhanced feature vector to obtain fused feature data;

[0010] Calculate the feature matching loss value, spatial continuity loss value and category distribution loss value, construct a segmentation energy function, calculate the gradient value and energy loss value of the segmentation energy function based on the variational inference criterion, stop the iteration and output the initial segmentation label when the energy loss value is less than a preset energy loss threshold or the energy loss change rate of three consecutive iterations is less than a preset loss change threshold, perform connectivity analysis on the points based on the initial segmentation label, assign points that are spatially connected and have the same initial segmentation label to the same segmentation area, determine the area to be processed, merge the areas based on feature similarity and boundary continuity, and repeat the merging to obtain the final segmentation result.

[0011] In an optional embodiment,

[0012] Acquire original point cloud data, construct a three-dimensional hash index table and map the original point cloud data to the three-dimensional hash index table, calculate the neighborhood density value of the midpoint of the original point cloud data and construct a point cloud density histogram, use the least squares method to fit the point cloud density histogram to obtain a density distribution curve, extract the local maximum value in the density distribution curve as a point cloud stratification reference value, and adaptively stratify the original point cloud data based on the point cloud stratification reference value to obtain a point cloud sub-region set including:

[0013] Acquire original point cloud data, divide the three-dimensional space into cubic grids according to a preset side length value, calculate the corresponding grid index based on the three-dimensional coordinates of the point in the original point cloud data, store the original point cloud data in the position of the grid index, construct a three-dimensional hash index table, take the point in the original point cloud data as the center, construct a first spherical area according to a preset first radius, count the number of points falling into the first spherical area as the neighborhood density value of the point, divide the neighborhood density value into a plurality of equal-width intervals, count the number of points in each of the equal-width intervals to construct a point cloud density histogram;

[0014] Based on the median value of each density interval in the point cloud density histogram and the corresponding number of points, a density distribution curve is obtained by least square fitting, the density distribution curve is numerically differentiated, and a local maximum point on the density distribution curve is detected based on a sign change of the numerical differentiation, and the density value corresponding to the local maximum point is used as a reference value for point cloud stratification;

[0015] For each point cloud layer reference value, a density tolerance value is set and points whose density values ​​fall within an interval centered on the point cloud layer reference value and within a range of the density tolerance value are searched in the original point cloud data, and the searched points are added to the same point cloud sub-region to obtain a point cloud sub-region set.

[0016] In an optional embodiment,

[0017] Calculate the regional density count value and geometric distribution coefficient of each point in the point cloud sub-region set, identify and remove noise points in the point cloud sub-region based on the preset regional density threshold and geometric distribution coefficient threshold, and for the points that are not removed, calculate the main feature direction, main curvature value, Gaussian curvature value and shape index, construct an enhanced feature vector and determine the point cloud boundary features by calculating the local surface direction entropy value, including:

[0018] Taking the points in the point cloud sub-region set as the center, constructing a second spherical region according to a preset second radius, counting the number of points that fall into the second spherical region and belong to the same point cloud sub-region as the regional density count value, calculating the eigenvalues ​​of the covariance matrix of the points in the second spherical region and calculating the eigenvalue ratio as the geometric distribution coefficient;

[0019] Determine whether the area density count value of each point in the point cloud sub-area set is less than a preset area density threshold or whether the geometric distribution coefficient is greater than a preset geometric distribution coefficient threshold, mark the points that meet any condition as noise points and remove them from the original point cloud data, and obtain point cloud data after noise is filtered out;

[0020] Calculating a local point cloud covariance matrix for each point in the point cloud data after noise is filtered out, determining a main feature direction based on an eigenvector of the local point cloud covariance matrix, calculating a main curvature value based on a least squares fit of the local point cloud covariance matrix, calculating a Gaussian curvature value and a shape index based on the main curvature value, and combining the three-dimensional coordinates of the point, the main feature direction, the main curvature value, the Gaussian curvature value and the shape index to construct an enhanced feature vector;

[0021] Taking the point in the point cloud data after noise filtering as the center, construct a third spherical area according to a preset third radius, calculate the entropy value of the direction vector in the third spherical area, and determine the boundary feature points of the point cloud based on a preset entropy value threshold.

[0022] In an optional embodiment,

[0023] For each point in the enhanced feature vector, four spherical search areas with radii that are doubly increasing in sequence are constructed, and a local geometric descriptor is constructed by calculating the eigenvalues ​​of the covariance matrix of the points in the spherical search area. The local geometric descriptor and the point cloud boundary features are input into a three-branch spatial attention network, and the spatial distribution information, neighborhood structure information and boundary shape information of the points are extracted respectively, and feature weighted fusion is performed through a self-attention mechanism to obtain spatial features. The spatial features are added to a multi-level geometric feature extraction network, and curvature change features, normal vector change features and shape change features are extracted in sequence. Multi-scale feature fusion is performed using a feature pyramid structure to obtain geometric features, and a bidirectional feature propagation network is constructed to calculate the temporal dependency of the geometric features to obtain propagation features. The propagation features are jump-connected and residual-connected with the enhanced feature vector to obtain fused feature data, including:

[0024] Taking each point in the enhanced feature vector as the center, four concentric spherical regions are constructed, wherein the radii of the four concentric spherical regions are in a two-fold increasing relationship, and the radius of the first spherical region is a preset initial search radius;

[0025] Extract all points in the four concentric spherical regions, calculate the covariance matrix corresponding to each point in the four concentric spherical regions, perform eigenvalue decomposition on the covariance matrix to obtain three eigenvalues ​​and an eigenvector corresponding to each eigenvalue, arrange the eigenvalues ​​in order of size, use the eigenvectors corresponding to the first two eigenvalues ​​as the main eigenvector, and construct the three eigenvalues ​​and the main eigenvector as a local geometric descriptor;

[0026] The local geometric descriptor and the point cloud boundary feature are input into a preset three-branch spatial attention network, the first branch of the three-branch spatial attention network performs a convolution operation on the local geometric descriptor to extract spatial distribution features, the second branch performs a graph convolution operation on the local geometric descriptor to extract neighborhood structure features, the third branch performs a convolution operation on the point cloud boundary feature to extract boundary shape features, and the spatial distribution features, the neighborhood structure features and the boundary shape features are weightedly fused through a self-attention mechanism to obtain spatial features;

[0027] The spatial feature is input into a multi-level geometric feature extraction network, the multi-level geometric feature extraction network includes a plurality of feature extraction modules, each feature extraction module includes three parallel branches, the first parallel branch obtains a curvature change feature by calculating the difference of local curvature of the point cloud, the second parallel branch obtains a normal vector change feature by calculating the deviation degree of the local normal vector of the point cloud, and the third parallel branch obtains a shape change feature by calculating the difference of the local geometric descriptor, and the curvature change feature, the normal vector change feature and the shape change feature are multi-scale fused through a feature pyramid structure to obtain a geometric feature;

[0028] The geometric features are input into a bidirectional feature propagation network, which adopts a bidirectional recurrent neural network structure. The geometric features are transmitted forward and backward in a point cloud sequence to model the contextual information and temporal evolution law of the point cloud, thereby obtaining propagation features. The propagation features and the enhanced feature vectors are jump-connected and residual-connected to obtain fused feature data.

[0029] In an optional embodiment,

[0030] The propagation feature and the enhanced feature vector are skip-connected and residual-connected to obtain fused feature data including:

[0031] Taking the current layer as the starting point, perform average pooling downsampling on the propagation features and enhanced feature vectors in the first 50 layers of the current layer, and concatenate the propagation features of the first 50 layers and the enhanced feature vectors of the first 50 layers after downsampling with the propagation features of the current layer and the enhanced feature vectors of the current layer in the feature dimension to obtain multi-scale jump features;

[0032] The multi-scale jump feature is multiplied by the preset query matrix weight, key matrix weight and value matrix weight respectively, and the corresponding bias term is added to obtain the query matrix, key matrix and value matrix, the query matrix is ​​multiplied by the transpose of the key matrix, and divided by the square root of the dimension of the key matrix, and the attention weight matrix is ​​obtained by the softmax function;

[0033] The attention weight matrix is ​​multiplied by the value matrix to obtain an attention feature, the attention feature is concatenated with the multi-scale jump feature, a transformation feature is obtained by combining a linear transformation operation, the transformation feature is element-wise added to the enhanced feature vector of the current layer to obtain a residual feature, the residual feature is multiplied by a preset residual gating weight matrix, the enhanced feature vector of the current layer is multiplied by a preset enhanced gating weight matrix, a residual gating signal and an enhanced gating signal are obtained by combining a gated bias term and passing through a sigmoid function;

[0034] Multiply the residual gating signal by the residual feature, multiply the enhanced gating signal by the enhanced feature vector of the current layer, add the two products to obtain the initial fused feature, multiply the initial fused feature by the preset self-attention weight matrix, determine the self-attention weight vector in combination with the self-attention bias term, multiply the self-attention weight vector by the initial fused feature to obtain the selection feature, repeat the calculation to obtain the selection feature corresponding to each layer, and splice the selection features corresponding to each layer on the feature dimension to obtain the fused feature.

[0035] In an optional embodiment,

[0036] Calculate the feature matching loss value, spatial continuity loss value and category distribution loss value, construct a segmentation energy function, calculate the gradient value and energy loss value of the segmentation energy function based on the variational inference criterion, stop the iteration and output the initial segmentation label when the energy loss value is less than the preset energy loss threshold or the energy loss change rate of three consecutive iterations is less than the preset loss change threshold, perform connectivity analysis on the points based on the initial segmentation label, attribute points that are connected in space and have the same initial segmentation label to the same segmentation area, determine the area to be processed and merge the areas based on feature similarity and boundary continuity, and repeat the merging to obtain the final segmentation result including:

[0037] Add the fused feature to a preset classifier to obtain a category prediction result, determine the category label corresponding to the fused feature based on the category prediction result, extract the time series adjacent samples corresponding to the fused feature as adjacent features, count the category prediction results to obtain category distribution, use the cross entropy loss function to calculate the matching degree between the fused feature data and the category label to obtain a feature matching loss value, use the paired potential function to calculate the spatial continuity of the adjacent features to obtain a spatial continuity loss value, and use the information entropy function to calculate the overall consistency of the category distribution to obtain a category distribution loss value;

[0038] The feature matching loss value, the spatial continuity loss value and the category distribution loss value are weighted and summed according to a preset weight coefficient to construct a segmentation energy function, and the gradient value and energy loss value of the segmentation energy function are calculated based on the variational inference criterion. When the energy loss value is less than a preset energy loss threshold or the energy loss change rate of three consecutive iterations is less than a preset energy loss change rate, the iteration is stopped and an initial segmentation label is output;

[0039] Based on the initial segmentation label, connectivity analysis is performed on the points, points that are spatially connected and have the same initial segmentation label are assigned to the same segmentation area, the number of points and the boundary curvature of each segmentation area are calculated, and the area with a number of points less than a preset number threshold or a boundary curvature greater than a preset boundary curvature threshold is marked as a to-be-processed area. For each to-be-processed area, the feature similarity and boundary continuity between the current to-be-processed area and the adjacent area are calculated, and the merge score corresponding to each adjacent area is calculated based on the feature similarity and the boundary continuity. The to-be-processed area is merged into the adjacent area with the highest merge score, and the merging is repeated until there is no area that can be merged, thereby obtaining the final segmentation result.

[0040] In an optional embodiment,

[0041] The spatial continuity of the adjacent features is calculated using a pairwise potential function to obtain the spatial continuity loss as shown in the following formula:

[0042]

[0043] Among them, L spatial represents the spatial continuity loss, N represents the total number of points, and f i represents the fusion feature corresponding to the i-th point, f j represents the fusion feature corresponding to the jth point, p i represents the spatial coordinates of the i-th point, p j represents the spatial coordinates of the jth point, σ f The scale parameter representing the feature similarity, σ p The scale parameter representing the spatial distance, ||f i -f j || 2 represents the distance between point i and point j in the feature space, ||p i -p j || 2 Represents the distance between point i and point j in space.

[0044] A second aspect of an embodiment of the present invention provides a three-dimensional point cloud data segmentation system based on multi-scale point features, comprising:

[0045] The first unit is used to obtain original point cloud data, construct a three-dimensional hash index table and map the original point cloud data to the three-dimensional hash index table, calculate the neighborhood density value of the point in the original point cloud data and construct a point cloud density histogram, use the least squares method to fit the point cloud density histogram to obtain a density distribution curve, extract the local maximum value in the density distribution curve as a point cloud stratification reference value, adaptively stratify the original point cloud data based on the point cloud stratification reference value to obtain a point cloud sub-region set, calculate the regional density count value and geometric distribution coefficient of each point in the point cloud sub-region set, identify and eliminate noise points in the point cloud sub-region based on a preset regional density threshold and geometric distribution coefficient threshold, and for points that are not eliminated, construct an enhanced feature vector by calculating the main feature direction, the main curvature value, the Gaussian curvature value and the shape index, and determine the point cloud boundary feature by calculating the local surface direction entropy value;

[0046] The second unit is used to construct four spherical search areas with radii that are doubly increasing in sequence for each point in the enhanced feature vector, construct a local geometric descriptor by calculating the eigenvalues ​​of the covariance matrix of the points in the spherical search area, input the local geometric descriptor and the point cloud boundary feature into a three-branch spatial attention network, extract the spatial distribution information, neighborhood structure information and boundary shape information of the point respectively, and perform feature weighted fusion through a self-attention mechanism to obtain spatial features, add the spatial features to a multi-level geometric feature extraction network, extract curvature change features, normal vector change features and shape change features in sequence, use a feature pyramid structure to perform multi-scale feature fusion to obtain geometric features, construct a bidirectional feature propagation network to calculate the temporal dependency of the geometric features to obtain propagation features, and perform jump connections and residual connections between the propagation features and the enhanced feature vector to obtain fused feature data;

[0047] The third unit is used to calculate the feature matching loss value, the spatial continuity loss value and the category distribution loss value, construct a segmentation energy function, calculate the gradient value and the energy loss value of the segmentation energy function based on the variational inference criterion, stop the iteration and output the initial segmentation label when the energy loss value is less than a preset energy loss threshold or the energy loss change rate of three consecutive iterations is less than a preset loss change threshold, perform connectivity analysis on the points based on the initial segmentation label, attribute points that are spatially connected and have the same initial segmentation label to the same segmentation area, determine the area to be processed, merge the areas based on feature similarity and boundary continuity, and repeat the merging to obtain the final segmentation result.

[0048] According to a third aspect of the embodiments of the present invention,

[0049] An electronic device is provided, comprising:

[0050] processor;

[0051] a memory for storing processor-executable instructions;

[0052] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0053] According to a fourth aspect of the embodiments of the present invention,

[0054] A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.

[0055] In the present invention, by constructing a three-dimensional hash index and a point cloud density histogram, the original point cloud is adaptively hierarchically divided, which effectively solves the problems of uneven distribution of point cloud data and large scale differences, and provides a good data basis for subsequent feature extraction and regional segmentation. The regional density counting and the geometric distribution coefficient are combined to accurately identify and remove noise points and outliers in the point cloud, improve the quality and reliability of the point cloud data, and construct an enhanced feature vector integrating spatial distribution, neighborhood structure and boundary shape. The multi-scale and multi-modal point cloud features are extracted through a three-branch spatial attention network, and the representation ability and discriminability of the features are enhanced. A segmentation energy function including feature matching, spatial continuity and category distribution is constructed. The iterative optimization and adaptive convergence of the point cloud labels are realized through variational inference, which effectively balances data-driven and prior constraints, and improves the accuracy and consistency of the segmentation results. In summary, the present invention constructs a complete set of adaptive hierarchical point cloud segmentation methods, which can adapt to point cloud segmentation tasks of different scenes and objects, has stronger robustness, generalization and adaptability, and has broad application prospects in the fields of smart cities, autonomous driving, and cultural relics protection. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 A schematic diagram of a flow chart of a three-dimensional point cloud data segmentation method based on multi-scale point features according to an embodiment of the present invention;

[0057] Figure 2 Schematic diagram of the structure of a three-dimensional point cloud data segmentation system based on multi-scale point features according to an embodiment of the present invention. DETAILED DESCRIPTION

[0058] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0059] The technical solution of the present invention is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0060] Figure 1 FIG. 1 is a flow chart of a method for segmenting three-dimensional point cloud data based on multi-scale point features according to an embodiment of the present invention. Figure 1 As shown, the method includes:

[0061] S1. Obtain original point cloud data, construct a three-dimensional hash index table and map the original point cloud data to the three-dimensional hash index table, calculate the neighborhood density value of the point in the original point cloud data and construct a point cloud density histogram, use the least squares method to fit the point cloud density histogram to obtain a density distribution curve, extract the local maximum value in the density distribution curve as a point cloud stratification reference value, adaptively stratify the original point cloud data based on the point cloud stratification reference value to obtain a point cloud sub-region set, calculate the regional density count value and geometric distribution coefficient of each point in the point cloud sub-region set, identify and eliminate noise points in the point cloud sub-region based on a pre-set regional density threshold and geometric distribution coefficient threshold, and for points that are not eliminated, construct an enhanced feature vector by calculating the main feature direction, main curvature value, Gaussian curvature value and shape index, and determine the point cloud boundary feature by calculating the local surface direction entropy value;

[0062] The three-dimensional hash index table is an index structure for efficiently storing and searching three-dimensional point cloud data. It improves query efficiency by dividing the space into discrete grid units and mapping the point cloud data using a hash algorithm. The neighborhood density value refers to the density of points within a certain range around a point in the point cloud data, which is used to measure the density of the point in the local area. The point cloud density histogram is a histogram formed based on the neighborhood density values ​​of each point in the point cloud data, which is used to represent the distribution of point clouds at different density levels. The point cloud stratification reference value refers to the value used to classify point clouds by density or other features when processing point cloud data. The reference standard for stratification helps distinguish different areas or structures in the point cloud. The density count value refers to the number of points in the point cloud data within a specified area or voxel, which is used to indicate the density of points in the area. The geometric distribution coefficient is a coefficient that describes the uniformity of the distribution of point cloud data in space, and is usually used to evaluate the spatial distribution characteristics of point clouds. The Gaussian curvature is a mathematical quantity that describes the local curvature of a surface, especially in point cloud surface analysis, it is used to quantify the curvature of the surface. The local surface directional entropy is used to measure the uncertainty and complexity of the local surface direction of the point cloud, and is often used to describe and classify surface features.

[0063] In an optional embodiment,

[0064] Acquire original point cloud data, construct a three-dimensional hash index table and map the original point cloud data to the three-dimensional hash index table, calculate the neighborhood density value of the midpoint of the original point cloud data and construct a point cloud density histogram, use the least squares method to fit the point cloud density histogram to obtain a density distribution curve, extract the local maximum value in the density distribution curve as a point cloud stratification reference value, and adaptively stratify the original point cloud data based on the point cloud stratification reference value to obtain a point cloud sub-region set including:

[0065] Acquire original point cloud data, divide the three-dimensional space into cubic grids according to a preset side length value, calculate the corresponding grid index based on the three-dimensional coordinates of the point in the original point cloud data, store the original point cloud data in the position of the grid index, construct a three-dimensional hash index table, take the point in the original point cloud data as the center, construct a first spherical area according to a preset first radius, count the number of points falling into the first spherical area as the neighborhood density value of the point, divide the neighborhood density value into a plurality of equal-width intervals, count the number of points in each of the equal-width intervals to construct a point cloud density histogram;

[0066] Based on the median value of each density interval in the point cloud density histogram and the corresponding number of points, a density distribution curve is obtained by least square fitting, the density distribution curve is numerically differentiated, and a local maximum point on the density distribution curve is detected based on a sign change of the numerical differentiation, and the density value corresponding to the local maximum point is used as a reference value for point cloud stratification;

[0067] For each point cloud layer reference value, a density tolerance value is set and points whose density values ​​fall within an interval centered on the point cloud layer reference value and within a range of the density tolerance value are searched in the original point cloud data, and the searched points are added to the same point cloud sub-region to obtain a point cloud sub-region set.

[0068] The cube grid is a grid structure that divides three-dimensional space into uniform cube units, and is commonly used for segmentation, downsampling and feature extraction of point cloud data. The density tolerance value is used to measure the tolerance range of density changes in a local area of ​​the point cloud, and is usually used in clustering or segmentation algorithms to determine whether an area can be considered as a similar density area.

[0069] Get the original point cloud data. Assume that the acquired point cloud data contains 10,000 points, each of which contains three-dimensional coordinates (x, y, z) and other attribute information of the point, such as color, reflection intensity, etc. Example data points: (1.2, 2.5, 3.7), (2.1, 3.2, 1.5), ..., (10.5, 9.8, 7.2).

[0070] Set the side length to 1. Divide the 3D space into a grid of cubes with a side length of 1. The index of each grid is determined by its position in the x, y, and z directions. For example, the point with coordinates (2.3, 4.5, 1.8) falls into the grid with index (2, 4, 1).

[0071] Build a 3D hash index table. Store point cloud data in the corresponding grid index location. For example, the point (2.3, 4.5, 1.8) is stored in the grid with index (2, 4, 1). The hash table can quickly find points in a specific grid.

[0072] Calculate the neighborhood density of the point. Set the first radius value to 0.5. Take point (2.3, 4.5, 1.8) as an example, and construct the first spherical region with the point as the center and a radius of 0.5. Count the number of points that fall into the spherical region. Assuming it is 5, the neighborhood density value of the point is 5. Perform the same operation on each point in the point cloud data and calculate its neighborhood density value.

[0073] Construct a point cloud density histogram. Divide the neighborhood density values ​​into multiple equal-width intervals, such as [0, 2), [2, 4), [4, 6), [6, 8), [8, 10]. Count the number of points in each interval. For example, suppose the number of points in the interval [0, 2) is 1000, the number of points in the interval [2, 4) is 3000, the number of points in the interval [4, 6) is 4000, the number of points in the interval [6, 8) is 1500, and the number of points in the interval [8, 10] is 500. The point cloud density histogram is composed of these intervals and the corresponding number of points.

[0074] Fit the density distribution curve. Take the median of each density interval (for example, the median of [0, 2) is 1, the median of [2, 4) is 3, and so on) and the corresponding number of points as input, and use the least squares method to fit the density distribution curve. This curve describes the density distribution of the point cloud data.

[0075] Extract the reference value of the point cloud layer. Perform numerical differentiation on the density distribution curve, and detect the local maximum points on the density distribution curve by observing the sign change of the numerical differentiation. For example, suppose that local maximum points are detected at density values ​​of 3 and 7. Then 3 and 7 are used as the reference values ​​of the point cloud layer.

[0076] Adaptively divide the point cloud into layers. For the point cloud layer reference value 3, set the density tolerance value to 0.5. Search for points whose density values ​​fall within the interval [2.5, 3.5] in the original point cloud data. Add the found points to the same point cloud sub-region. Similarly, for the point cloud layer reference value 7, set the density tolerance value to 0.5, search for points whose density values ​​fall within the interval [6.5, 7.5], and add the found points to another point cloud sub-region. Finally, a set of point cloud sub-regions is obtained.

[0077] In this embodiment, by dividing the point cloud data into multiple sub-areas, each sub-area can be processed independently, thereby improving the parallelism and efficiency of point cloud processing. By dividing the space into cubic grids and establishing a hash index, the complexity of spatial search is greatly reduced, the efficiency of subsequent density calculation and neighborhood search is improved, and traversal search of the entire point cloud is avoided. The noise caused by histogram discretization is eliminated through density distribution curve fitting, and the spherical area statistical density is used to ensure the isotropy of density calculation. In summary, this embodiment realizes adaptive density stratification of point cloud data, provides a reliable density stratification basis for subsequent point cloud segmentation and feature extraction, and helps to improve the accuracy and robustness of the point cloud processing algorithm.

[0078] In an optional embodiment,

[0079] Calculate the regional density count value and geometric distribution coefficient of each point in the point cloud sub-region set, identify and remove noise points in the point cloud sub-region based on the preset regional density threshold and geometric distribution coefficient threshold, and for the points that are not removed, calculate the main feature direction, main curvature value, Gaussian curvature value and shape index, construct an enhanced feature vector and determine the point cloud boundary features by calculating the local surface direction entropy value, including:

[0080] Taking the points in the point cloud sub-region set as the center, constructing a second spherical region according to a preset second radius, counting the number of points that fall into the second spherical region and belong to the same point cloud sub-region as the regional density count value, calculating the eigenvalues ​​of the covariance matrix of the points in the second spherical region and calculating the eigenvalue ratio as the geometric distribution coefficient;

[0081] Determine whether the area density count value of each point in the point cloud sub-area set is less than a preset area density threshold or whether the geometric distribution coefficient is greater than a preset geometric distribution coefficient threshold, mark the points that meet any condition as noise points and remove them from the original point cloud data, and obtain point cloud data after noise is filtered out;

[0082] Calculating a local point cloud covariance matrix for each point in the point cloud data after noise is filtered out, determining a main feature direction based on an eigenvector of the local point cloud covariance matrix, calculating a main curvature value based on a least squares fit of the local point cloud covariance matrix, calculating a Gaussian curvature value and a shape index based on the main curvature value, and combining the three-dimensional coordinates of the point, the main feature direction, the main curvature value, the Gaussian curvature value and the shape index to construct an enhanced feature vector;

[0083] Taking the point in the point cloud data after noise filtering as the center, construct a third spherical area according to a preset third radius, calculate the entropy value of the direction vector in the third spherical area, and determine the boundary feature points of the point cloud based on a preset entropy value threshold.

[0084] The local point cloud covariance matrix is ​​a matrix obtained by calculating the covariance between the coordinates of each point in the local neighborhood of the point cloud, and is used to describe the geometric shape and directionality of the point cloud in the local area. The main characteristic direction refers to the main direction obtained by decomposing the eigenvalues ​​of the local covariance matrix of the point cloud, which represents the main direction of the point cloud distribution in the area.

[0085] Get the original point cloud data, which contains a large number of spatial three-dimensional points, each of which has a three-dimensional coordinate (x, y, z). For example, a point cloud data contains 10,000 points, and the coordinate value of each point is expressed in meters, distributed in a space range of 10mx10mx5m.

[0086] The point cloud data is divided into several sub-regions. The segmentation method can be based on the spatial division method. For example, the point cloud data is divided into a grid of 1mx1mx1m, and the points in each grid constitute a point cloud sub-region. In this example, the point cloud can be divided into 10x10x5=500 sub-regions.

[0087] For each point cloud sub-region, calculate the regional density count value and geometric distribution coefficient of each point. Take point P as an example and set the second radius to 0.1m. Construct a spherical region with point P as the center and a radius of 0.1m. Count the number of points that fall into the spherical region and belong to the same point cloud sub-region as the regional density count value of point P. Assume that the regional density count value of point P is 20.

[0088] Calculate the covariance matrix of all points in the second spherical region where point P is located. Perform eigenvalue decomposition on the covariance matrix to obtain three eigenvalues ​​λ1, λ2, and λ3, and sort them by size, λ1≥λ2≥λ3. Calculate the eigenvalue ratio λ1 / λ3 as the geometric distribution coefficient. Assume that the geometric distribution coefficient of point P is 5.

[0089] Repeat the above steps for all points in all point cloud sub-regions to calculate the regional density count value and geometric distribution coefficient of each point.

[0090] Set the area density threshold and the geometric distribution coefficient threshold. For example, the area density threshold is 10 and the geometric distribution coefficient threshold is 8.

[0091] Determine whether the regional density count value of each point is less than the regional density threshold, or whether the geometric distribution coefficient is greater than the geometric distribution coefficient threshold. If any condition is met, the point is marked as a noise point. For example, the regional density count value 20 of point P is greater than 10, and the geometric distribution coefficient 5 is less than 8, so point P is not marked as a noise point.

[0092] All points marked as noise are removed from the original point cloud data to obtain the point cloud data after noise is filtered out.

[0093] For each point in the point cloud data after noise is filtered out, the local point cloud covariance matrix is ​​calculated. Set the first radius to 0.2m, take point P as an example, construct a spherical area with point P as the center and a radius of 0.2m, and calculate the covariance matrix of all points in the spherical area as the local point cloud covariance matrix of point P.

[0094] Perform eigenvalue decomposition on the local point cloud covariance matrix to obtain three eigenvectors and three eigenvalues. The three eigenvectors represent the main feature directions of point P. Assume that the three eigenvectors are v1, v2, and v3.

[0095] The principal curvature values ​​k1 and k2 of point P are calculated by performing least squares quadratic surface fitting on the local point cloud.

[0096] The Gaussian curvature value K=k1*k2 and the shape index S=(k1+k2) / (k1-k2) are calculated based on the principal curvature values. Assume k1=0.1, k2=0.05, K=0.005, and S=3.

[0097] The enhanced feature vector is constructed by combining the three-dimensional coordinates (x, y, z), the main feature direction (v1, v2, v3), the main curvature value (k1, k2), the Gaussian curvature value K and the shape index S of the point P. For example, the enhanced feature vector of point P is (x, y, z, v1, v2, v3, k1, k2, K, S).

[0098] A third spherical area is constructed based on a preset third radius with a point in the point cloud data after noise is filtered out as the center. For example, the third radius is set to 0.15 m. Taking point P as an example, a spherical area with point P as the center and a radius of 0.15 m is constructed.

[0099] Calculate the entropy of the direction vectors of all points in the third spherical area. The direction vector can be obtained by calculating the vector difference between each point and its neighboring points. The entropy value is used to measure the degree of discreteness of the direction vector.

[0100] The boundary feature points of the point cloud are determined based on a preset entropy value threshold. For example, the entropy value threshold is set to 0.8. If the entropy value of point P is greater than 0.8, point P is marked as a boundary feature point.

[0101] In this embodiment, noise points are identified and eliminated through regional density and geometric distribution coefficient, which effectively improves the quality of point cloud data and lays the foundation for subsequent point cloud processing tasks. By calculating the main feature direction, principal curvature value, Gaussian curvature value and shape index, an enhanced feature vector is constructed, which more comprehensively describes the geometric features of the point cloud, which is conducive to improving the accuracy of point cloud recognition and classification. By calculating the local surface direction entropy value, the boundary feature points of the point cloud are effectively identified, which provides reliable information for point cloud segmentation and boundary extraction. In summary, this embodiment realizes noise filtering, feature enhancement and boundary extraction of point cloud data, improves the quality and feature expression ability of point cloud data, and provides a reliable data foundation for subsequent precise segmentation and recognition.

[0102] S2. Construct four spherical search areas with radii that are doubly increasing in sequence for each point in the enhanced feature vector, construct a local geometric descriptor by calculating the eigenvalues ​​of the covariance matrix of the points in the spherical search area, input the local geometric descriptor and the point cloud boundary features into a three-branch spatial attention network, extract the spatial distribution information, neighborhood structure information and boundary shape information of the points respectively, and perform weighted fusion of features through a self-attention mechanism to obtain spatial features, add the spatial features to a multi-level geometric feature extraction network, extract curvature change features, normal vector change features and shape change features in sequence, use a feature pyramid structure to perform multi-scale feature fusion to obtain geometric features, construct a bidirectional feature propagation network to calculate the temporal dependency of the geometric features to obtain propagation features, perform jump connections and residual connections on the propagation features and the enhanced feature vector to obtain fused feature data;

[0103] The spherical search area refers to a three-dimensional area composed of all points within a specified radius from a certain point as the center in the point cloud data. It is commonly used in neighborhood search and feature extraction of point clouds. The local geometric descriptor is a way to describe the shape and structure of the area by analyzing the geometric features of the local area in the point cloud data. It is commonly used in tasks such as point cloud registration, classification and clustering. The three-branch spatial attention network is a network structure that uses three branches and processes the spatial information of the input data through an attention mechanism to improve the network's understanding and extraction capabilities of spatial features. The feature pyramid structure is a multi-scale feature extraction method that enhances the model's learning ability for features of different scales by constructing multi-level feature maps. It is commonly used in tasks such as target detection and image segmentation.

[0104] In an optional embodiment,

[0105] For each point in the enhanced feature vector, four spherical search areas with radii that are doubly increasing in sequence are constructed, and a local geometric descriptor is constructed by calculating the eigenvalues ​​of the covariance matrix of the points in the spherical search area. The local geometric descriptor and the point cloud boundary features are input into a three-branch spatial attention network, and the spatial distribution information, neighborhood structure information and boundary shape information of the points are extracted respectively, and feature weighted fusion is performed through a self-attention mechanism to obtain spatial features. The spatial features are added to a multi-level geometric feature extraction network, and curvature change features, normal vector change features and shape change features are extracted in sequence. Multi-scale feature fusion is performed using a feature pyramid structure to obtain geometric features, and a bidirectional feature propagation network is constructed to calculate the temporal dependency of the geometric features to obtain propagation features. The propagation features are jump-connected and residual-connected with the enhanced feature vector to obtain fused feature data, including:

[0106] Taking each point in the enhanced feature vector as the center, four concentric spherical regions are constructed, wherein the radii of the four concentric spherical regions are in a two-fold increasing relationship, and the radius of the first spherical region is a preset initial search radius;

[0107] Extract all points in the four concentric spherical regions, calculate the covariance matrix corresponding to each point in the four concentric spherical regions, perform eigenvalue decomposition on the covariance matrix to obtain three eigenvalues ​​and an eigenvector corresponding to each eigenvalue, arrange the eigenvalues ​​in order of size, use the eigenvectors corresponding to the first two eigenvalues ​​as the main eigenvector, and construct the three eigenvalues ​​and the main eigenvector as a local geometric descriptor;

[0108] The local geometric descriptor and the point cloud boundary feature are input into a preset three-branch spatial attention network, the first branch of the three-branch spatial attention network performs a convolution operation on the local geometric descriptor to extract spatial distribution features, the second branch performs a graph convolution operation on the local geometric descriptor to extract neighborhood structure features, the third branch performs a convolution operation on the point cloud boundary feature to extract boundary shape features, and the spatial distribution features, the neighborhood structure features and the boundary shape features are weightedly fused through a self-attention mechanism to obtain spatial features;

[0109] The spatial feature is input into a multi-level geometric feature extraction network, the multi-level geometric feature extraction network includes a plurality of feature extraction modules, each feature extraction module includes three parallel branches, the first parallel branch obtains a curvature change feature by calculating the difference of local curvature of the point cloud, the second parallel branch obtains a normal vector change feature by calculating the deviation degree of the local normal vector of the point cloud, and the third parallel branch obtains a shape change feature by calculating the difference of the local geometric descriptor, and the curvature change feature, the normal vector change feature and the shape change feature are multi-scale fused through a feature pyramid structure to obtain a geometric feature;

[0110] The geometric features are input into a bidirectional feature propagation network, which adopts a bidirectional recurrent neural network structure. The geometric features are transmitted forward and backward in a point cloud sequence to model the contextual information and temporal evolution law of the point cloud, thereby obtaining propagation features. The propagation features and the enhanced feature vectors are jump-connected and residual-connected to obtain fused feature data.

[0111] The boundary shape feature refers to the feature used to describe the shape of the boundary of an object or area, including the curvature of the boundary, shape complexity, etc., and is commonly used in shape analysis and object recognition. The local curvature difference of the point cloud describes the degree of difference in curvature between different points in the local area of ​​the point cloud, and is commonly used to analyze the uneven areas or abnormal points on the surface of the point cloud. The local normal vector deviation degree of the point cloud is calculated by calculating the difference between the normal vector and the reference normal vector in the local area of ​​the point cloud, and is used to measure the change or irregularity of the surface shape of the area.

[0112] Obtain the point cloud data to be processed. The point cloud data can be obtained by a 3D scanner, a depth camera or other sensors, and each point is represented in the form of (x, y, z) coordinates. For example, a point cloud data can contain thousands of points, each of which represents a sampling point on the surface of an object. In order to improve the efficiency and accuracy of subsequent processing, the acquired raw point cloud data is preprocessed, including removing noise points, outliers and data missing points, and downsampling the point cloud data to reduce the amount of data. The preprocessed point cloud data is denoted as P.

[0113] Perform feature enhancement on the preprocessed point cloud data P. Extract local features of the point cloud, such as local curvature, normal vector, and local density, and concatenate these local features with the coordinate information of the original point cloud data P to form an enhanced feature vector. Assuming that the point cloud P contains 1000 points and the coordinates of each point are (x, y, z), the dimension of the enhanced feature vector is 1000x6 (assuming that 3 local features are extracted).

[0114] For each point in the enhanced feature vector, four spherical search regions with radii that are doubly increasing are constructed. The initial search radius is set to r = 0.1. With each point in the enhanced feature vector as the center, four concentric spherical regions are constructed with radii of 0.1, 0.2, 0.4, and 0.8, respectively. For the first point in the enhanced feature vector, all points in the spherical region with a radius of 0.1 are searched around it. Assume that 50 points are found.

[0115] Extract all points within the four spherical search areas and calculate the covariance matrix of the points within each spherical area. For the first point, 50 points are found within the spherical area with a radius of 0.1, and the covariance matrix of these 50 points is calculated.

[0116] Perform eigenvalue decomposition on each covariance matrix to obtain three eigenvalues ​​and the eigenvector corresponding to each eigenvalue. Arrange the three eigenvalues ​​in order of size, and use the eigenvectors corresponding to the first two eigenvalues ​​as the main eigenvectors. Construct these three eigenvalues ​​and two main eigenvectors as local geometric descriptors. Assume that the three eigenvalues ​​of the first point in a spherical region with a radius of 0.1 are 10, 5, 1, and the corresponding eigenvectors are v1, v2, v3, then the local geometric descriptor of the point is (10, 5, 1, v1, v2).

[0117] Extract the boundary features of the point cloud. You can use an edge detection algorithm, such as the Canny edge detection algorithm, to extract the boundary points of the point cloud data and use them as the boundary features of the point cloud. Suppose 100 boundary points are extracted.

[0118] The local geometric descriptor and point cloud boundary features are input into a three-branch spatial attention network. The first branch of the network performs a convolution operation on the local geometric descriptor to extract spatial distribution features, the second branch performs a graph convolution operation on the local geometric descriptor to extract neighborhood structure features, and the third branch performs a convolution operation on the point cloud boundary features to extract boundary shape features. Then, the features extracted by these three branches are weighted and fused through the self-attention mechanism to obtain spatial features.

[0119] The spatial features are input into a multi-level geometric feature extraction network. The network contains multiple feature extraction modules, each of which contains three parallel branches. The first branch calculates the difference in local curvature of the point cloud to obtain the curvature change feature, the second branch calculates the deviation degree of the local normal vector of the point cloud to obtain the normal vector change feature, and the third branch calculates the difference in local geometric descriptors to obtain the shape change feature. The features of these three branches are multi-scale fused using a feature pyramid structure to obtain geometric features.

[0120] The geometric features are input into the bidirectional feature propagation network. The network adopts a bidirectional recurrent neural network structure, which models the contextual information and temporal evolution law of the point cloud by forward and backward propagation of geometric features in the point cloud sequence to obtain the propagation features.

[0121] The propagated features and enhanced feature vectors are skipped and residually connected to obtain fused feature data.

[0122] In this embodiment, by constructing a multi-scale spherical search area and extracting local geometric descriptors, the local geometric information of the point cloud can be captured more comprehensively, thereby improving the expression ability of the point cloud features. By combining the spatial attention mechanism and the multi-level geometric feature extraction network, the spatial distribution information, neighborhood structure information and boundary shape information of the point cloud can be effectively integrated, thereby enhancing the robustness of the point cloud features. By modeling the temporal dependency of the point cloud through a bidirectional feature propagation network, the global contextual information of the point cloud can be better captured, thereby improving the performance of tasks such as point cloud classification and segmentation. In summary, this embodiment realizes multi-scale extraction and multi-level fusion of point cloud features, effectively integrates spatial, geometric and temporal features through a deep learning network architecture, significantly improves the feature expression ability, and provides high-quality feature representation for subsequent point cloud segmentation tasks.

[0123] In an optional embodiment,

[0124] The propagation feature and the enhanced feature vector are skip-connected and residual-connected to obtain fused feature data including:

[0125] Taking the current layer as the starting point, perform average pooling downsampling on the propagation features and enhanced feature vectors in the first 50 layers of the current layer, and concatenate the propagation features of the first 50 layers and the enhanced feature vectors of the first 50 layers after downsampling with the propagation features of the current layer and the enhanced feature vectors of the current layer in the feature dimension to obtain multi-scale jump features;

[0126] The multi-scale jump feature is multiplied by the preset query matrix weight, key matrix weight and value matrix weight respectively, and the corresponding bias term is added to obtain the query matrix, key matrix and value matrix, the query matrix is ​​multiplied by the transpose of the key matrix, and divided by the square root of the dimension of the key matrix, and the attention weight matrix is ​​obtained by the softmax function;

[0127] The attention weight matrix is ​​multiplied by the value matrix to obtain an attention feature, the attention feature is concatenated with the multi-scale jump feature, a transformation feature is obtained by combining a linear transformation operation, the transformation feature is element-wise added to the enhanced feature vector of the current layer to obtain a residual feature, the residual feature is multiplied by a preset residual gating weight matrix, the enhanced feature vector of the current layer is multiplied by a preset enhanced gating weight matrix, a residual gating signal and an enhanced gating signal are obtained by combining a gated bias term and passing through a sigmoid function;

[0128] Multiply the residual gating signal by the residual feature, multiply the enhanced gating signal by the enhanced feature vector of the current layer, add the two products to obtain the initial fused feature, multiply the initial fused feature by the preset self-attention weight matrix, determine the self-attention weight vector in combination with the self-attention bias term, multiply the self-attention weight vector by the initial fused feature to obtain the selection feature, repeat the calculation to obtain the selection feature corresponding to each layer, and splice the selection features corresponding to each layer on the feature dimension to obtain the fused feature.

[0129] The residual gating signal is a gating mechanism for a neural network, which enhances the learning ability of the network by controlling the residual signal of the information flow, and is usually used to improve the stability and expressiveness of the network. The enhanced gating signal refers to the introduction of a gating mechanism in a neural network to enhance the influence of certain information flows, thereby improving the network's learning ability and robustness for important features.

[0130] Preprocess the propagation features and enhancement features extracted from each layer of the network. The propagation features can be the output feature maps of the middle layer of the network, and the enhancement features can be the features learned through other auxiliary tasks. Assuming that the network layer currently being processed is the i-th layer, the propagation features and enhancement features of the first 50 layers (if i is less than 50, the first i layers are collected). Perform average pooling downsampling operations on the collected propagation features and enhancement features of each layer. For example, an average pooling kernel of size 2x2 can be used for downsampling. The propagation features of the first 50 layers (or i layers) after downsampling are spliced ​​with the propagation features of the current layer (i-th layer) in the feature dimension to obtain a multi-scale representation of the propagation features. Similarly, the enhancement features of the first 50 layers (or i layers) after downsampling are spliced ​​with the enhancement features of the current layer (i-th layer) in the feature dimension to obtain a multi-scale representation of the enhancement features. Finally, the multi-scale representation of the propagation features and the multi-scale representation of the enhancement features are spliced ​​in the feature dimension to obtain a multi-scale jump feature. For example, assuming that the propagation feature dimension of the current layer is 256, and the propagation feature dimensions after downsampling of the first 50 layers are all 256, then the multi-scale representation dimension of the spliced ​​propagation features is 256*51=13056.

[0131] The multi-scale jump features are processed using the attention mechanism. The multi-scale jump features are multiplied by the preset query matrix weights, key matrix weights, and value matrix weights, and the corresponding bias terms are added to obtain the query matrix, key matrix, and value matrix. The query matrix and the transpose of the key matrix are matrix multiplied, and the result is divided by the square root of the key matrix dimension, and then the softmax function is applied to obtain the attention weight matrix. The attention weight matrix is ​​multiplied by the value matrix to obtain the attention feature. The attention feature is concatenated with the multi-scale jump feature. After that, a linear transformation operation is performed on the concatenated features, for example, a fully connected layer is used to transform the features to obtain the transformed features.

[0132] The transformed features are added element by element to the enhanced feature vector of the current layer to obtain the residual features. The residual features are multiplied by the preset residual gating weight matrix, and the enhanced feature vector of the current layer is multiplied by the preset enhanced gating weight matrix, and the corresponding gated bias terms are added respectively, and then the sigmoid function is applied to obtain the residual gated signal and the enhanced gated signal. The residual gated signal is multiplied by the residual features, and the enhanced gated signal is multiplied by the enhanced feature vector of the current layer, and the two products are added to obtain the initial fusion features.

[0133] The initial fusion feature is processed by the self-attention mechanism. The initial fusion feature is multiplied by the preset self-attention weight matrix, and the self-attention bias term is added to obtain the self-attention weight vector. The self-attention weight vector is multiplied by the initial fusion feature to obtain the selection feature. Repeat the above operation for each layer to obtain the selection feature corresponding to each layer. The selection features corresponding to each layer are spliced ​​in the feature dimension to obtain the final fusion feature.

[0134] For example, assuming that the network has 100 layers in total, and the dimension of the initial fused features obtained in each layer is 512, the dimension of the final fused features is 512*100=51200.

[0135] In this embodiment, by fusing multi-scale features, contextual information of different scales can be captured, thereby enhancing the representation ability of features. The attention mechanism and the gating mechanism can effectively screen and enhance important features, suppress the influence of noise and irrelevant information, thereby improving the robustness of the model. By fusing multi-scale features, attention mechanisms and gating mechanisms, the performance of the model on various tasks can be effectively improved. In summary, this embodiment realizes the deep fusion and optimal selection of point cloud features through multi-scale feature fusion, attention mechanism and gated residual learning, improves the discriminability and robustness of feature expression, and provides high-quality feature representation for subsequent point cloud segmentation tasks.

[0136] S3. Calculate the feature matching loss value, spatial continuity loss value and category distribution loss value, construct a segmentation energy function, calculate the gradient value and energy loss value of the segmentation energy function based on the variational inference criterion, stop the iteration and output the initial segmentation label when the energy loss value is less than a preset energy loss threshold or the energy loss change rate of three consecutive iterations is less than a preset loss change threshold, perform connectivity analysis on the points based on the initial segmentation label, assign points that are spatially connected and have the same initial segmentation label to the same segmentation area, determine the area to be processed, and merge the areas based on feature similarity and boundary continuity, and repeat the merging to obtain the final segmentation result.

[0137] The feature matching loss value is a loss function used to measure the feature difference between two data samples or images. It is usually used in tasks such as image matching and target detection to calculate the distance or similarity of feature vectors. The spatial continuity loss value is used to measure the continuity of adjacent pixels or points in the image or data in space. It is often used in tasks such as semantic segmentation and image denoising to ensure that adjacent regions have consistent features. The category distribution loss value is a loss function used to measure the difference between the category distribution predicted by the model and the actual category distribution. It is usually used for classification problems, especially in multi-category classification tasks. The segmentation energy function is an objective function used in image segmentation. By calculating the difference between the region division after image segmentation and the actual target, the algorithm is guided to optimize the segmentation effect. The variational inference criterion is a criterion based on the variational inference method, which is used to calculate the posterior distribution of complex probability distribution through approximate inference. It is often used in tasks such as Bayesian inference and probability models. The energy loss change rate refers to the rate of change of energy loss with the training process during the optimization process. It is used to evaluate the convergence speed and optimization effect during the model training process. The connectivity analysis is a method for analyzing the connectivity of adjacent regions in images or point cloud data. It is often used in image segmentation, shape analysis and network topology analysis.

[0138] In an optional embodiment,

[0139] Calculate the feature matching loss value, spatial continuity loss value and category distribution loss value, construct a segmentation energy function, calculate the gradient value and energy loss value of the segmentation energy function based on the variational inference criterion, stop the iteration and output the initial segmentation label when the energy loss value is less than the preset energy loss threshold or the energy loss change rate of three consecutive iterations is less than the preset loss change threshold, perform connectivity analysis on the points based on the initial segmentation label, attribute points that are connected in space and have the same initial segmentation label to the same segmentation area, determine the area to be processed and merge the areas based on feature similarity and boundary continuity, and repeat the merging to obtain the final segmentation result including:

[0140] Add the fused feature to a preset classifier to obtain a category prediction result, determine the category label corresponding to the fused feature based on the category prediction result, extract the time series adjacent samples corresponding to the fused feature as adjacent features, count the category prediction results to obtain category distribution, use the cross entropy loss function to calculate the matching degree between the fused feature data and the category label to obtain a feature matching loss value, use the paired potential function to calculate the spatial continuity of the adjacent features to obtain a spatial continuity loss value, and use the information entropy function to calculate the overall consistency of the category distribution to obtain a category distribution loss value;

[0141] The feature matching loss value, the spatial continuity loss value and the category distribution loss value are weighted and summed according to a preset weight coefficient to construct a segmentation energy function, and the gradient value and energy loss value of the segmentation energy function are calculated based on the variational inference criterion. When the energy loss value is less than a preset energy loss threshold or the energy loss change rate of three consecutive iterations is less than a preset energy loss change rate, the iteration is stopped and an initial segmentation label is output;

[0142] Based on the initial segmentation label, connectivity analysis is performed on the points, points that are spatially connected and have the same initial segmentation label are assigned to the same segmentation area, the number of points and the boundary curvature of each segmentation area are calculated, and the area with a number of points less than a preset number threshold or a boundary curvature greater than a preset boundary curvature threshold is marked as a to-be-processed area. For each to-be-processed area, the feature similarity and boundary continuity between the current to-be-processed area and the adjacent area are calculated, and the merge score corresponding to each adjacent area is calculated based on the feature similarity and the boundary continuity. The to-be-processed area is merged into the adjacent area with the highest merge score, and the merging is repeated until there is no area that can be merged, thereby obtaining the final segmentation result.

[0143] The time-series adjacent samples refer to sample points that are temporally adjacent in time series data, and are usually used in time series analysis and prediction models to represent the time dependence of data. The paired potential function is a function that describes the interaction between two samples or objects, and is often used in image segmentation, image matching, graph neural networks and other fields to model the relationship between objects. The information entropy function is a function that quantifies the uncertainty or amount of information in a system or data, and is often used in data compression, feature selection and decision analysis to represent the randomness and complexity of the system. The boundary curvature is a mathematical quantity that describes the degree of curvature of a boundary line or surface, and is often used in shape analysis and image processing to reflect the smoothness or sharpness of the boundary.

[0144] Input the image data to be segmented, and extract the multi-dimensional features of the image, such as color, texture, spatial position, etc. In order to more effectively capture the local and global information of the image, a multi-scale feature extraction method is adopted, such as extracting the texture features of the image at different scales, and fusing the multi-scale features to obtain more representative fusion features. For example, for a 256x256 pixel color image, three scales of texture features can be extracted, namely the local binary pattern (LBP) features calculated with 3x3, 5x5 and 7x7 window sizes, and then the local binary pattern features of these three scales, as well as the RGB color features and pixel coordinate position features of the image are spliced ​​together to form the final fusion feature vector.

[0145] The fused features are input into a pre-trained classifier to obtain the category prediction results for each pixel. This classifier can be a support vector machine (SVM), random forest, or neural network. According to the category prediction results, the category label of each pixel is determined. For example, if the classifier outputs 5 categories, each pixel will be assigned a category label between 0 and 4. At the same time, the category prediction results of all pixels are counted to obtain the category distribution of the image, and the feature matching loss value, spatial continuity loss value, and category distribution loss value are calculated. The feature matching loss value is used to measure the matching degree between the fused features and the category label, which can be calculated using the cross entropy loss function. The spatial continuity loss value is used to measure the spatial continuity between adjacent pixels, which can be calculated using a pairwise potential function, such as the Potts model. The category distribution loss value is used to measure the overall consistency of the category distribution, which can be calculated using the information entropy function. For example, if all pixels in the image belong to the same category, the information entropy is 0, indicating that the category distribution is highly consistent. The three loss values ​​are weighted and summed according to the pre-set weight coefficients, such as 0.5, 0.3, and 0.2, to construct the segmentation energy function. Using the variational inference criterion, the gradient value and energy loss value of the segmentation energy function are iteratively calculated. In each iteration, the category label of the pixel is updated according to the gradient value, and the energy loss value is recalculated. When the energy loss value is less than the preset energy loss threshold (e.g., 0.01), or the energy loss change rate for three consecutive iterations is less than the preset loss change threshold (e.g., 0.001), the iteration is stopped and the initial segmentation label is output.

[0146] Perform region merging and perform connectivity analysis on pixels in the image based on the initial segmentation label. Pixels that are connected in spatial position and have the same initial segmentation label are assigned to the same segmentation region. Calculate the number of pixels and boundary curvature of each segmentation region. Mark regions where the number of pixels is less than a preset number threshold (e.g., 10) or the boundary curvature is greater than a preset boundary curvature threshold (e.g., 0.5) as regions to be processed. For each region to be processed, calculate the feature similarity and boundary continuity between it and the adjacent regions. Feature similarity can be calculated based on fusion features, such as using cosine similarity. Boundary continuity can be calculated based on the length and curvature change of the shared boundary between adjacent regions. Calculate the merge score corresponding to each adjacent region based on feature similarity and boundary continuity. Merge the region to be processed into the adjacent region with the highest merge score. Repeat the merge operation until there is no region that can be merged, and obtain the final segmentation result. For example, a region to be processed has three adjacent regions, and their merge scores are calculated to be 0.8, 0.6, and 0.2, respectively. Then, the region to be processed is merged into the adjacent region with a merge score of 0.8.

[0147] In this embodiment, by fusing multi-dimensional features and adopting variational inference criteria, the local and global information of the image can be captured more accurately, thereby improving the segmentation accuracy. By presetting weight coefficients and thresholds, the parameter setting process is simplified, making the method easier to use. Connectivity analysis ensures the integrity of the segmented area. It combines local features and global constraints to adaptively process targets of different scales, maintains the accuracy of the boundary, and reduces the risk of over-segmentation and under-segmentation. In summary, this embodiment achieves high-quality segmentation of point cloud data, provides accurate, coherent and physically meaningful segmentation results, and provides a reliable foundation for subsequent applications of point cloud data. .

[0148] In an optional embodiment,

[0149] The spatial continuity of the adjacent features is calculated using a pairwise potential function to obtain the spatial continuity loss as shown in the following formula:

[0150]

[0151] Among them, L spatial represents the spatial continuity loss, N represents the total number of points, and f i represents the fusion feature corresponding to the i-th point, f j represents the fusion feature corresponding to the jth point, p i represents the spatial coordinates of the i-th point, p j represents the spatial coordinates of the jth point, σ f The scale parameter representing the feature similarity, σ p The scale parameter representing the spatial distance, ||f i-f j || 2 represents the distance between point i and point j in the feature space, ||p i -p j || 2 Represents the distance between point i and point j in space.

[0152] In this embodiment, by constraining the consistency of features of adjacent points, the influence of noise and outliers can be effectively reduced and the accuracy of point cloud segmentation can be improved. By learning the spatial continuity features of point cloud data, the generalization ability of the model can be improved so that it can better adapt to new point cloud data. A smooth distance measurement is achieved through an exponential function to ensure the continuity of the segmentation boundary in features and space, and reduce the influence of noise and outliers. In summary, this embodiment combines the constraints of feature space and physical space to achieve the spatial continuity of point cloud segmentation results, improve the accuracy and smoothness of the segmentation boundary, and provide an effective optimization target for high-quality point cloud segmentation.

[0153] Figure 2 FIG. 1 is a schematic diagram of a structure of a 3D point cloud data segmentation system based on multi-scale point features according to an embodiment of the present invention. Figure 2 As shown, the system comprises:

[0154] The first unit is used to obtain original point cloud data, construct a three-dimensional hash index table and map the original point cloud data to the three-dimensional hash index table, calculate the neighborhood density value of the point in the original point cloud data and construct a point cloud density histogram, use the least squares method to fit the point cloud density histogram to obtain a density distribution curve, extract the local maximum value in the density distribution curve as a point cloud stratification reference value, adaptively stratify the original point cloud data based on the point cloud stratification reference value to obtain a point cloud sub-region set, calculate the regional density count value and geometric distribution coefficient of each point in the point cloud sub-region set, identify and eliminate noise points in the point cloud sub-region based on a preset regional density threshold and geometric distribution coefficient threshold, and for points that are not eliminated, construct an enhanced feature vector by calculating the main feature direction, the main curvature value, the Gaussian curvature value and the shape index, and determine the point cloud boundary feature by calculating the local surface direction entropy value;

[0155] The second unit is used to construct four spherical search areas with radii that are doubly increasing in sequence for each point in the enhanced feature vector, construct a local geometric descriptor by calculating the eigenvalues ​​of the covariance matrix of the points in the spherical search area, input the local geometric descriptor and the point cloud boundary feature into a three-branch spatial attention network, extract the spatial distribution information, neighborhood structure information and boundary shape information of the point respectively, and perform feature weighted fusion through a self-attention mechanism to obtain spatial features, add the spatial features to a multi-level geometric feature extraction network, extract curvature change features, normal vector change features and shape change features in sequence, use a feature pyramid structure to perform multi-scale feature fusion to obtain geometric features, construct a bidirectional feature propagation network to calculate the temporal dependency of the geometric features to obtain propagation features, and perform jump connections and residual connections between the propagation features and the enhanced feature vector to obtain fused feature data;

[0156] The third unit is used to calculate the feature matching loss value, the spatial continuity loss value and the category distribution loss value, construct a segmentation energy function, calculate the gradient value and the energy loss value of the segmentation energy function based on the variational inference criterion, stop the iteration and output the initial segmentation label when the energy loss value is less than a preset energy loss threshold or the energy loss change rate of three consecutive iterations is less than a preset loss change threshold, perform connectivity analysis on the points based on the initial segmentation label, attribute points that are spatially connected and have the same initial segmentation label to the same segmentation area, determine the area to be processed, merge the areas based on feature similarity and boundary continuity, and repeat the merging to obtain the final segmentation result.

[0157] According to a third aspect of the embodiments of the present invention,

[0158] An electronic device is provided, comprising:

[0159] processor;

[0160] a memory for storing processor-executable instructions;

[0161] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0162] According to a fourth aspect of the embodiments of the present invention,

[0163] A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.

[0164] The present invention may be a method, an apparatus, a system and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing various aspects of the present invention.

[0165] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A three-dimensional point cloud data segmentation method based on multi-scale point cloud features, characterized in that: include: Obtain original point cloud data, construct a three-dimensional hash index table and map the original point cloud data to the three-dimensional hash index table, calculate the neighborhood density value of the point in the original point cloud data and construct a point cloud density histogram, use the least squares method to fit the point cloud density histogram to obtain a density distribution curve, extract the local maximum value in the density distribution curve as a point cloud stratification reference value, adaptively stratify the original point cloud data based on the point cloud stratification reference value to obtain a point cloud sub-region set, calculate the regional density count value and geometric distribution coefficient of each point in the point cloud sub-region set, identify and eliminate noise points in the point cloud sub-region based on a pre-set regional density threshold and geometric distribution coefficient threshold, and for points that are not eliminated, construct an enhanced feature vector by calculating the main feature direction, the main curvature value, the Gaussian curvature value and the shape index, and determine the point cloud boundary feature by calculating the local surface direction entropy value; For each point in the enhanced feature vector, four spherical search areas with radii that are doubly increasing in sequence are constructed, a local geometric descriptor is constructed by calculating the eigenvalues ​​of the covariance matrix of the points in the spherical search area, the local geometric descriptor and the point cloud boundary features are input into a three-branch spatial attention network, the spatial distribution information, neighborhood structure information and boundary shape information of the points are extracted respectively, and feature weighted fusion is performed through a self-attention mechanism to obtain spatial features, the spatial features are added to a multi-level geometric feature extraction network, the curvature change features, normal vector change features and shape change features are extracted in sequence, multi-scale feature fusion is performed using a feature pyramid structure to obtain geometric features, a bidirectional feature propagation network is constructed to calculate the temporal dependency of the geometric features to obtain propagation features, and the propagation features are jump-connected and residual-connected with the enhanced feature vector to obtain fused feature data; Calculate the feature matching loss value, spatial continuity loss value and category distribution loss value, construct a segmentation energy function, calculate the gradient value and energy loss value of the segmentation energy function based on the variational inference criterion, stop the iteration and output the initial segmentation label when the energy loss value is less than a preset energy loss threshold or the energy loss change rate of three consecutive iterations is less than a preset loss change threshold, perform connectivity analysis on the points based on the initial segmentation label, assign points that are spatially connected and have the same initial segmentation label to the same segmentation area, determine the area to be processed, merge the areas based on feature similarity and boundary continuity, and repeat the merging to obtain the final segmentation result.

2. The method according to claim 1, characterized in that: Acquire original point cloud data, construct a three-dimensional hash index table and map the original point cloud data to the three-dimensional hash index table, calculate the neighborhood density value of the midpoint of the original point cloud data and construct a point cloud density histogram, use the least squares method to fit the point cloud density histogram to obtain a density distribution curve, extract the local maximum value in the density distribution curve as a point cloud stratification reference value, and adaptively stratify the original point cloud data based on the point cloud stratification reference value to obtain a point cloud sub-region set including: Acquire original point cloud data, divide the three-dimensional space into cubic grids according to a preset side length value, calculate the corresponding grid index based on the three-dimensional coordinates of the point in the original point cloud data, store the original point cloud data in the position of the grid index, construct a three-dimensional hash index table, take the point in the original point cloud data as the center, construct a first spherical area according to a preset first radius, count the number of points falling into the first spherical area as the neighborhood density value of the point, divide the neighborhood density value into a plurality of equal-width intervals, count the number of points in each of the equal-width intervals to construct a point cloud density histogram; Based on the median value of each density interval in the point cloud density histogram and the corresponding number of points, a density distribution curve is obtained by least square fitting, the density distribution curve is numerically differentiated, and a local maximum point on the density distribution curve is detected based on a sign change of the numerical differentiation, and the density value corresponding to the local maximum point is used as a reference value for point cloud stratification; For each point cloud layer reference value, a density tolerance value is set and points whose density values ​​fall within an interval centered on the point cloud layer reference value and within a range of the density tolerance value are searched in the original point cloud data, and the searched points are added to the same point cloud sub-region to obtain a point cloud sub-region set.

3. The method according to claim 1, characterized in that Calculate the regional density count value and geometric distribution coefficient of each point in the point cloud sub-region set, identify and remove noise points in the point cloud sub-region based on the preset regional density threshold and geometric distribution coefficient threshold, and for the points that are not removed, calculate the main feature direction, main curvature value, Gaussian curvature value and shape index, construct an enhanced feature vector and determine the point cloud boundary features by calculating the local surface direction entropy value, including: Taking the points in the point cloud sub-region set as the center, constructing a second spherical region according to a preset second radius, counting the number of points that fall into the second spherical region and belong to the same point cloud sub-region as the regional density count value, calculating the eigenvalues ​​of the covariance matrix of the points in the second spherical region and calculating the eigenvalue ratio as the geometric distribution coefficient; Determine whether the area density count value of each point in the point cloud sub-area set is less than a preset area density threshold or whether the geometric distribution coefficient is greater than a preset geometric distribution coefficient threshold, mark the points that meet any condition as noise points and remove them from the original point cloud data, and obtain point cloud data after noise is filtered out; Calculating a local point cloud covariance matrix for each point in the point cloud data after noise is filtered out, determining a main feature direction based on an eigenvector of the local point cloud covariance matrix, calculating a main curvature value based on a least squares fit of the local point cloud covariance matrix, calculating a Gaussian curvature value and a shape index based on the main curvature value, and combining the three-dimensional coordinates of the point, the main feature direction, the main curvature value, the Gaussian curvature value and the shape index to construct an enhanced feature vector; Taking the point in the point cloud data after noise filtering as the center, construct a third spherical area according to a preset third radius, calculate the entropy value of the direction vector in the third spherical area, and determine the boundary feature points of the point cloud based on a preset entropy value threshold.

4. The method according to claim 1, characterized in that For each point in the enhanced feature vector, four spherical search areas with radii that are doubly increasing in sequence are constructed, and a local geometric descriptor is constructed by calculating the eigenvalues ​​of the covariance matrix of the points in the spherical search area. The local geometric descriptor and the point cloud boundary features are input into a three-branch spatial attention network, and the spatial distribution information, neighborhood structure information and boundary shape information of the points are extracted respectively, and feature weighted fusion is performed through a self-attention mechanism to obtain spatial features. The spatial features are added to a multi-level geometric feature extraction network, and curvature change features, normal vector change features and shape change features are extracted in sequence. Multi-scale feature fusion is performed using a feature pyramid structure to obtain geometric features, and a bidirectional feature propagation network is constructed to calculate the temporal dependency of the geometric features to obtain propagation features. The propagation features are jump-connected and residual-connected with the enhanced feature vector to obtain fused feature data, including: Taking each point in the enhanced feature vector as the center, four concentric spherical regions are constructed, wherein the radii of the four concentric spherical regions are in a two-fold increasing relationship, and the radius of the first spherical region is a preset initial search radius; Extract all points in the four concentric spherical regions, calculate the covariance matrix corresponding to each point in the four concentric spherical regions, perform eigenvalue decomposition on the covariance matrix to obtain three eigenvalues ​​and an eigenvector corresponding to each eigenvalue, arrange the eigenvalues ​​in order of size, use the eigenvectors corresponding to the first two eigenvalues ​​as the main eigenvector, and construct the three eigenvalues ​​and the main eigenvector as a local geometric descriptor; The local geometric descriptor and the point cloud boundary feature are input into a preset three-branch spatial attention network, the first branch of the three-branch spatial attention network performs a convolution operation on the local geometric descriptor to extract spatial distribution features, the second branch performs a graph convolution operation on the local geometric descriptor to extract neighborhood structure features, the third branch performs a convolution operation on the point cloud boundary feature to extract boundary shape features, and the spatial distribution features, the neighborhood structure features and the boundary shape features are weightedly fused through a self-attention mechanism to obtain spatial features; The spatial feature is input into a multi-level geometric feature extraction network, the multi-level geometric feature extraction network includes a plurality of feature extraction modules, each feature extraction module includes three parallel branches, the first parallel branch obtains a curvature change feature by calculating the difference of local curvature of the point cloud, the second parallel branch obtains a normal vector change feature by calculating the deviation degree of the local normal vector of the point cloud, and the third parallel branch obtains a shape change feature by calculating the difference of the local geometric descriptor, and the curvature change feature, the normal vector change feature and the shape change feature are multi-scale fused through a feature pyramid structure to obtain a geometric feature; The geometric features are input into a bidirectional feature propagation network, which adopts a bidirectional recurrent neural network structure. The geometric features are transmitted forward and backward in a point cloud sequence to model the contextual information and temporal evolution law of the point cloud, thereby obtaining propagation features. The propagation features and the enhanced feature vectors are jump-connected and residual-connected to obtain fused feature data.

5. The method according to claim 4, characterized in that The propagation feature and the enhanced feature vector are skip-connected and residual-connected to obtain fused feature data including: Taking the current layer as the starting point, perform average pooling downsampling on the propagation features and enhanced feature vectors in the first 50 layers of the current layer, and concatenate the propagation features of the first 50 layers and the enhanced feature vectors of the first 50 layers after downsampling with the propagation features of the current layer and the enhanced feature vectors of the current layer in the feature dimension to obtain multi-scale jump features; The multi-scale jump feature is multiplied by the preset query matrix weight, key matrix weight and value matrix weight respectively, and the corresponding bias term is added to obtain the query matrix, key matrix and value matrix, the query matrix is ​​multiplied by the transpose of the key matrix, and divided by the square root of the dimension of the key matrix, and the attention weight matrix is ​​obtained by the softmax function; The attention weight matrix is ​​multiplied by the value matrix to obtain an attention feature, the attention feature is concatenated with the multi-scale jump feature, a transformation feature is obtained by combining a linear transformation operation, the transformation feature is element-wise added to the enhanced feature vector of the current layer to obtain a residual feature, the residual feature is multiplied by a preset residual gating weight matrix, the enhanced feature vector of the current layer is multiplied by a preset enhanced gating weight matrix, a residual gating signal and an enhanced gating signal are obtained by combining a gated bias term and passing through a sigmoid function; Multiply the residual gating signal by the residual feature, multiply the enhanced gating signal by the enhanced feature vector of the current layer, add the two products to obtain the initial fused feature, multiply the initial fused feature by the preset self-attention weight matrix, determine the self-attention weight vector in combination with the self-attention bias term, multiply the self-attention weight vector by the initial fused feature to obtain the selection feature, repeat the calculation to obtain the selection feature corresponding to each layer, and splice the selection features corresponding to each layer on the feature dimension to obtain the fused feature.

6. The method according to claim 1, characterized in that Calculate the feature matching loss value, spatial continuity loss value and category distribution loss value, construct a segmentation energy function, calculate the gradient value and energy loss value of the segmentation energy function based on the variational inference criterion, stop the iteration and output the initial segmentation label when the energy loss value is less than the preset energy loss threshold or the energy loss change rate of three consecutive iterations is less than the preset loss change threshold, perform connectivity analysis on the points based on the initial segmentation label, attribute points that are connected in space and have the same initial segmentation label to the same segmentation area, determine the area to be processed and merge the areas based on feature similarity and boundary continuity, and repeat the merging to obtain the final segmentation result including: Add the fused feature to a preset classifier to obtain a category prediction result, determine the category label corresponding to the fused feature based on the category prediction result, extract the time series adjacent samples corresponding to the fused feature as adjacent features, count the category prediction results to obtain category distribution, use the cross entropy loss function to calculate the matching degree between the fused feature data and the category label to obtain a feature matching loss value, use the paired potential function to calculate the spatial continuity of the adjacent features to obtain a spatial continuity loss value, and use the information entropy function to calculate the overall consistency of the category distribution to obtain a category distribution loss value; The feature matching loss value, the spatial continuity loss value and the category distribution loss value are weighted and summed according to a preset weight coefficient to construct a segmentation energy function, and the gradient value and energy loss value of the segmentation energy function are calculated based on the variational inference criterion. When the energy loss value is less than a preset energy loss threshold or the energy loss change rate of three consecutive iterations is less than a preset energy loss change rate, the iteration is stopped and an initial segmentation label is output; Based on the initial segmentation label, connectivity analysis is performed on the points, points that are spatially connected and have the same initial segmentation label are assigned to the same segmentation area, the number of points and the boundary curvature of each segmentation area are calculated, and the area with a number of points less than a preset number threshold or a boundary curvature greater than a preset boundary curvature threshold is marked as a to-be-processed area. For each to-be-processed area, the feature similarity and boundary continuity between the current to-be-processed area and the adjacent area are calculated, and the merge score corresponding to each adjacent area is calculated based on the feature similarity and the boundary continuity. The to-be-processed area is merged into the adjacent area with the highest merge score, and the merging is repeated until there is no area that can be merged, thereby obtaining the final segmentation result.

7. The method according to claim 6, characterized in that The spatial continuity of the adjacent features is calculated using a paired potential function to obtain a spatial continuity loss value as shown in the following formula: Among them, L spatial represents the spatial continuity loss, N represents the total number of points, and f i represents the fusion feature corresponding to the i-th point, f j represents the fusion feature corresponding to the jth point, p i represents the spatial coordinates of the i-th point, p j represents the spatial coordinates of the jth point, σ f The scale parameter representing the feature similarity, σ p The scale parameter representing the spatial distance, ||f i -f j || 2 represents the distance between point i and point j in the feature space, ||p i -p j || 2 Represents the distance between point i and point j in space.

8. A three-dimensional point cloud data segmentation system based on multi-scale point features, used to implement the method according to any one of claims 1 to 7, characterized in that: include: The first unit is used to obtain original point cloud data, construct a three-dimensional hash index table and map the original point cloud data to the three-dimensional hash index table, calculate the neighborhood density value of the point in the original point cloud data and construct a point cloud density histogram, use the least squares method to fit the point cloud density histogram to obtain a density distribution curve, extract the local maximum value in the density distribution curve as a point cloud stratification reference value, adaptively stratify the original point cloud data based on the point cloud stratification reference value to obtain a point cloud sub-region set, calculate the regional density count value and geometric distribution coefficient of each point in the point cloud sub-region set, identify and eliminate noise points in the point cloud sub-region based on a preset regional density threshold and geometric distribution coefficient threshold, and for points that are not eliminated, construct an enhanced feature vector by calculating the main feature direction, the main curvature value, the Gaussian curvature value and the shape index, and determine the point cloud boundary feature by calculating the local surface direction entropy value; The second unit is used to construct four spherical search areas with radii that are doubly increasing in sequence for each point in the enhanced feature vector, construct a local geometric descriptor by calculating the eigenvalues ​​of the covariance matrix of the points in the spherical search area, input the local geometric descriptor and the point cloud boundary feature into a three-branch spatial attention network, extract the spatial distribution information, neighborhood structure information and boundary shape information of the point respectively, and perform feature weighted fusion through a self-attention mechanism to obtain spatial features, add the spatial features to a multi-level geometric feature extraction network, extract curvature change features, normal vector change features and shape change features in sequence, use a feature pyramid structure to perform multi-scale feature fusion to obtain geometric features, construct a bidirectional feature propagation network to calculate the temporal dependency of the geometric features to obtain propagation features, and perform jump connections and residual connections between the propagation features and the enhanced feature vector to obtain fused feature data; The third unit is used to calculate the feature matching loss value, the spatial continuity loss value and the category distribution loss value, construct a segmentation energy function, calculate the gradient value and the energy loss value of the segmentation energy function based on the variational inference criterion, stop the iteration and output the initial segmentation label when the energy loss value is less than a preset energy loss threshold or the energy loss change rate of three consecutive iterations is less than a preset loss change threshold, perform connectivity analysis on the points based on the initial segmentation label, attribute points that are spatially connected and have the same initial segmentation label to the same segmentation area, determine the area to be processed, merge the areas based on feature similarity and boundary continuity, and repeat the merging to obtain the final segmentation result.

9. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the method described in any one of claims 1 to 7.

10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images