Three-dimensional point cloud data segmentation method and system based on multi-scale point cloud features

Abstract

The present invention provides a three-dimensional point cloud data segmentation method and system based on multi-scale point cloud features, relating to the field of point cloud processing technology. The method comprises: obtaining raw point cloud data and constructing a three-dimensional hash index table; performing adaptive hierarchical segmentation based on the local maximum of the density distribution curve; removing noise points based on regional density counts and geometric distribution coefficients; constructing enhanced feature vectors containing main feature directions, principal curvature values, etc. for unremoved points; constructing local geometric descriptors using four spherical search regions of increasing radius; inputting spatial features into a three-branch spatial attention network; and obtaining fused feature data through a multi-level geometric feature extraction network and a bidirectional feature propagation network. The final segmentation result is obtained through segmentation energy function optimization and region merging strategies.

Description

Technical Field

[0001] The present invention relates to the field of point cloud processing technology, and in particular to a three-dimensional point cloud data segmentation method and system based on multi-scale point cloud features. Background Art

[0002] With the rapid development and widespread application of 3D scanning technology, point cloud data has become an important data form for describing and analyzing 3D objects. Point cloud is a discrete sampling set consisting of a large number of points with 3D coordinates. It is acquired through laser scanning, depth cameras, multi-view stereo matching, etc. Compared with 2D images, point cloud data can truly record the 3D spatial information of objects.

[0003] Since point cloud data lacks regular topological structure and semantic information, directly analyzing and processing raw point clouds faces many challenges. Point cloud segmentation is one of the key tasks in point cloud processing. It aims to divide the point cloud into multiple non-overlapping subsets so that points in the same subset belong to the same physical entity or have the same semantic attributes.

[0004] Traditional point cloud segmentation methods are mainly based on geometric features and clustering algorithms, but they have problems such as being insensitive to point cloud density and distribution, lacking multi-scale feature representation, and being susceptible to noise and outlier interference.

[0005] Therefore, a solution is urgently needed to solve the problems existing in the prior art. Summary of the Invention

[0006] The embodiments of the present invention provide a three-dimensional point cloud data segmentation method and system based on multi-scale point cloud features, which can at least solve some of the problems existing in the prior art.

[0007] A first aspect of an embodiment of the present invention provides a method for segmenting three-dimensional point cloud data based on multi-scale point cloud features, comprising:

[0008] Obtain original point cloud data, construct a three-dimensional hash index table and map the original point cloud data to the three-dimensional hash index table, calculate the neighborhood density value of the point in the original point cloud data and construct a point cloud density histogram, use the least squares method to fit the point cloud density histogram to obtain a density distribution curve, extract the local maximum value in the density distribution curve as a point cloud stratification reference value, adaptively divide the original point cloud data into layers based on the point cloud stratification reference value to obtain a point cloud sub-region set, calculate the regional density count value and geometric distribution coefficient of each point in the point cloud sub-region set, identify and eliminate noise points in the point cloud sub-region based on a preset regional density threshold and geometric distribution coefficient threshold, and for points that are not eliminated, construct an enhanced feature vector by calculating the main feature direction, main curvature value, Gaussian curvature value and shape index, and determine the point cloud boundary feature by calculating the local surface direction entropy value;

[0009] For each point in the enhanced feature vector, four spherical search areas with radii that are in a doubly increasing relationship are constructed. A local geometric descriptor is constructed by calculating the eigenvalues ​​of the covariance matrix of the points in the spherical search area. The local geometric descriptor and the point cloud boundary features are input into a three-branch spatial attention network. The spatial distribution information, neighborhood structure information, and boundary shape information of the points are extracted respectively, and feature weighted fusion is performed through a self-attention mechanism to obtain spatial features. The spatial features are added to a multi-level geometric feature extraction network. Curvature change features, normal vector change features, and shape change features are extracted in sequence. Multi-scale feature fusion is performed using a feature pyramid structure to obtain geometric features. A bidirectional feature propagation network is constructed to calculate the temporal dependency of the geometric features to obtain propagation features. The propagation features are skipped and residually connected with the enhanced feature vector to obtain fused feature data.

[0010] Calculate the feature matching loss value, spatial continuity loss value and category distribution loss value, construct a segmentation energy function, calculate the gradient value and energy loss value of the segmentation energy function based on the variational inference criterion, stop the iteration and output the initial segmentation label when the energy loss value is less than the preset energy loss threshold or the energy loss change rate of three consecutive iterations is less than the preset loss change threshold, perform connectivity analysis on the points based on the initial segmentation label, assign points that are spatially connected and have the same initial segmentation label to the same segmentation area, determine the area to be processed, and merge the areas based on feature similarity and boundary continuity, and repeat the merging to obtain the final segmentation result.

[0011] In an optional embodiment,

[0012] Obtaining original point cloud data, constructing a three-dimensional hash index table and mapping the original point cloud data to the three-dimensional hash index table, calculating neighborhood density values ​​of points in the original point cloud data and constructing a point cloud density histogram, fitting the point cloud density histogram using the least squares method to obtain a density distribution curve, extracting local maximum values ​​in the density distribution curve as point cloud stratification reference values, and adaptively dividing the original point cloud data into layers based on the point cloud stratification reference values ​​to obtain a point cloud sub-region set including:

[0013] Obtaining original point cloud data, dividing the three-dimensional space into a cubic grid according to a preset side length value, calculating a corresponding grid index based on the three-dimensional coordinates of a point in the original point cloud data, storing the original point cloud data in a location of the grid index, constructing a three-dimensional hash index table, constructing a first spherical region centered on the point in the original point cloud data according to a preset first radius, counting the number of points falling within the first spherical region as a neighborhood density value of the point, dividing the neighborhood density value into a plurality of equal-width intervals, and counting the number of points within each of the equal-width intervals to construct a point cloud density histogram;

[0014] Based on the median value of each density interval and the corresponding number of points in the point cloud density histogram, a density distribution curve is obtained by least squares fitting, the density distribution curve is numerically differentiated, a local maximum point on the density distribution curve is detected based on a sign change of the numerical differentiation, and the density value corresponding to the local maximum point is used as a reference value for point cloud stratification;

[0015] For each point cloud layer reference value, a density tolerance value is set and points whose density values ​​fall within an interval centered on the point cloud layer reference value and within a range of the density tolerance value are searched in the original point cloud data. The found points are added to the same point cloud sub-region to obtain a point cloud sub-region set.

[0016] In an optional embodiment,

[0017] Calculate the regional density count value and geometric distribution coefficient of each point in the point cloud sub-region set, identify and remove noise points in the point cloud sub-region based on the preset regional density threshold and geometric distribution coefficient threshold, and for the points that are not removed, calculate the main feature direction, main curvature value, Gaussian curvature value and shape index, construct an enhanced feature vector and determine the point cloud boundary features by calculating the local surface direction entropy value, including:

[0018] Taking the points in the point cloud sub-region set as the center, construct a second spherical region according to a preset second radius, count the number of points that fall within the second spherical region and belong to the same point cloud sub-region as the regional density count value, calculate the eigenvalues ​​of the covariance matrix of the points in the second spherical region, and calculate the eigenvalue ratio as the geometric distribution coefficient;

[0019] Determining whether the area density count value of each point in the point cloud sub-area set is less than a preset area density threshold or whether the geometric distribution coefficient is greater than a preset geometric distribution coefficient threshold, marking the point that meets either condition as a noise point and removing it from the original point cloud data, thereby obtaining point cloud data after noise filtering;

[0020] Calculating a local point cloud covariance matrix for each point in the noise-filtered point cloud data, determining a principal eigendirection based on an eigenvector of the local point cloud covariance matrix, calculating a principal curvature value based on a least squares fit of the local point cloud covariance matrix, calculating a Gaussian curvature value and a shape index based on the principal curvature value, and combining the three-dimensional coordinates of the point, the principal eigendirection, the principal curvature value, the Gaussian curvature value, and the shape index to construct an enhanced eigenvector;

[0021] Taking the point in the point cloud data after noise filtering as the center, a third spherical area is constructed according to a preset third radius, the entropy value of the direction vector in the third spherical area is calculated, and the boundary feature points of the point cloud are determined based on a preset entropy value threshold.

[0022] In an optional embodiment,

[0023] For each point in the enhanced feature vector, four spherical search areas with radii that are in a doubly increasing relationship are constructed. A local geometric descriptor is constructed by calculating the eigenvalues ​​of the covariance matrix of the points in the spherical search area. The local geometric descriptor and the point cloud boundary features are input into a three-branch spatial attention network. The spatial distribution information, neighborhood structure information and boundary shape information of the points are extracted respectively and feature weighted fusion is performed through a self-attention mechanism to obtain spatial features. The spatial features are added to a multi-level geometric feature extraction network, and curvature change features, normal vector change features and shape change features are extracted in sequence. Multi-scale feature fusion is performed using a feature pyramid structure to obtain geometric features. A bidirectional feature propagation network is constructed to calculate the temporal dependency of the geometric features to obtain propagation features. The propagation features are skipped and residually connected with the enhanced feature vector to obtain fused feature data including:

[0024] Construct four concentric spherical regions with each point in the enhanced feature vector as the center. The radii of the four concentric spherical regions are in a doubly increasing relationship. The radius of the first spherical region is the preset initial search radius.

[0025] Extracting all points within the four concentric spherical regions, calculating a covariance matrix corresponding to each point within the four concentric spherical regions, performing eigenvalue decomposition on the covariance matrix to obtain three eigenvalues ​​and an eigenvector corresponding to each eigenvalue, arranging the eigenvalues ​​in order of magnitude, using the eigenvectors corresponding to the first two eigenvalues ​​as the main eigenvector, and constructing the three eigenvalues ​​and the main eigenvector as a local geometric descriptor;

[0026] Inputting the local geometric descriptor and the point cloud boundary feature into a pre-set three-branch spatial attention network, the first branch of the three-branch spatial attention network performs a convolution operation on the local geometric descriptor to extract spatial distribution features, the second branch performs a graph convolution operation on the local geometric descriptor to extract neighborhood structure features, and the third branch performs a convolution operation on the point cloud boundary feature to extract boundary shape features. The spatial distribution features, the neighborhood structure features, and the boundary shape features are weightedly fused through a self-attention mechanism to obtain spatial features;

[0027] Inputting the spatial features into a multi-level geometric feature extraction network, the multi-level geometric feature extraction network includes multiple feature extraction modules, each feature extraction module includes three parallel branches, the first parallel branch obtains a curvature change feature by calculating the local curvature difference of the point cloud, the second parallel branch obtains a normal vector change feature by calculating the degree of deviation of the local normal vector of the point cloud, and the third parallel branch obtains a shape change feature by calculating the difference of local geometric descriptors. The curvature change feature, the normal vector change feature, and the shape change feature are multi-scale fused through a feature pyramid structure to obtain a geometric feature;

[0028] The geometric features are input into a bidirectional feature propagation network, which adopts a bidirectional recurrent neural network structure. By transmitting the geometric features forward and backward in the point cloud sequence, the contextual information and temporal evolution law of the point cloud are modeled to obtain propagation features. The propagation features and the enhanced feature vectors are jump-connected and residual-connected to obtain fused feature data.

[0029] In an optional embodiment,

[0030] Performing skip connection and residual connection on the propagation feature and the enhanced feature vector to obtain fused feature data includes:

[0031] Taking the current layer as the starting point, perform average pooling downsampling on the propagation features and enhanced feature vectors of the first 50 layers of the current layer. Then, concatenate the propagation features and enhanced feature vectors of the first 50 layers after downsampling with the propagation features and enhanced feature vectors of the current layer in the feature dimension to obtain multi-scale jump features.

[0032] Multiply the multi-scale jump features with the preset query matrix weight, key matrix weight, and value matrix weight respectively, and add the corresponding bias terms to obtain the query matrix, key matrix, and value matrix. Multiply the query matrix by the transpose of the key matrix and divide it by the square root of the dimension of the key matrix. Then, pass it through the softmax function to obtain the attention weight matrix.

[0033] Multiplying the attention weight matrix by the value matrix to obtain an attention feature, concatenating the attention feature with the multi-scale jump feature, combining with a linear transformation operation to obtain a transformation feature, adding the transformation feature to the enhanced feature vector of the current layer element-by-element to obtain a residual feature, multiplying the residual feature by a preset residual gating weight matrix, multiplying the enhanced feature vector of the current layer by a preset enhanced gating weight matrix, combining with a gated bias term and passing through a sigmoid function to obtain a residual gated signal and an enhanced gated signal;

[0034] Multiply the residual gating signal by the residual feature, multiply the enhanced gating signal by the enhanced feature vector of the current layer, add the two products to obtain the initial fusion feature, multiply the initial fusion feature by the preset self-attention weight matrix, determine the self-attention weight vector in combination with the self-attention bias term, multiply the self-attention weight vector by the initial fusion feature to obtain the selection feature, repeat the calculation and obtain the selection feature corresponding to each layer, splice the selection features corresponding to each layer in the feature dimension to obtain the fusion feature.

[0035] In an optional embodiment,

[0036] Calculate the feature matching loss value, spatial continuity loss value and category distribution loss value, construct a segmentation energy function, calculate the gradient value and energy loss value of the segmentation energy function based on the variational inference criterion, stop the iteration and output the initial segmentation label when the energy loss value is less than the preset energy loss threshold or the energy loss change rate of three consecutive iterations is less than the preset loss change threshold, perform connectivity analysis on the points based on the initial segmentation label, assign points with spatially connected positions and the same initial segmentation label to the same segmentation area, determine the area to be processed and merge the areas based on feature similarity and boundary continuity, and repeat the merging to obtain the final segmentation result including:

[0037] The fused feature is added to a pre-set classifier to obtain a category prediction result, the category prediction result is used to determine the category label corresponding to the fused feature, the time series adjacent samples corresponding to the fused feature are extracted as adjacent features, the category prediction result is statistically analyzed to obtain a category distribution, the cross entropy loss function is used to calculate the degree of matching between the fused feature data and the category label to obtain a feature matching loss value, the pairwise potential function is used to calculate the spatial continuity of the adjacent features to obtain a spatial continuity loss value, and the information entropy function is used to calculate the overall consistency of the category distribution to obtain a category distribution loss value;

[0038] The feature matching loss value, the spatial continuity loss value, and the category distribution loss value are weighted and summed according to a preset weight coefficient to construct a segmentation energy function, and the gradient value and energy loss value of the segmentation energy function are calculated based on the variational inference criterion. When the energy loss value is less than a preset energy loss threshold or the energy loss change rate for three consecutive iterations is less than a preset energy loss change rate, the iteration is stopped and the initial segmentation label is output;

[0039] Based on the initial segmentation label, the connectivity analysis of the points is performed, and the points with spatially connected positions and the same initial segmentation label are assigned to the same segmentation area. The number of points and the boundary curvature of each segmentation area are calculated, and the areas with the number of points less than a preset number threshold or the boundary curvature greater than a preset boundary curvature threshold are marked as areas to be processed. For each area to be processed, the feature similarity and boundary continuity between the current area to be processed and the adjacent areas are calculated. Based on the feature similarity and the boundary continuity, the merging score corresponding to each adjacent area is calculated, and the area to be processed is merged into the adjacent area with the highest merging score. The merging is repeated until there is no area that can be merged, and the final segmentation result is obtained.

[0040] In an optional embodiment,

[0041] The spatial continuity of the adjacent features is calculated using a pairwise potential function to obtain the spatial continuity loss as shown in the following formula:

[0042]

[0043] Among them, L spatial represents the spatial continuity loss, N represents the total number of points, and f i represents the fusion feature corresponding to the i-th point, f j represents the fusion feature corresponding to the jth point, p i represents the spatial coordinates of the i-th point, p j represents the spatial coordinates of the jth point, σ f The scale parameter representing the feature similarity, σ p The scale parameter representing the spatial distance, ||f i -f j || 2 represents the distance between point i and point j in the feature space, ||p i -p j || 2 Represents the distance between point i and point j in space.

[0044] A second aspect of an embodiment of the present invention provides a three-dimensional point cloud data segmentation system based on multi-scale point cloud features, comprising:

[0045] The first unit is used to obtain original point cloud data, construct a three-dimensional hash index table and map the original point cloud data to the three-dimensional hash index table, calculate the neighborhood density value of the point in the original point cloud data and construct a point cloud density histogram, use the least squares method to fit the point cloud density histogram to obtain a density distribution curve, extract the local maximum value in the density distribution curve as a point cloud stratification reference value, adaptively divide the original point cloud data into layers based on the point cloud stratification reference value to obtain a point cloud sub-region set, calculate the regional density count value and geometric distribution coefficient of each point in the point cloud sub-region set, identify and eliminate noise points in the point cloud sub-region based on a preset regional density threshold and geometric distribution coefficient threshold, and for points that are not eliminated, construct an enhanced feature vector by calculating the main feature direction, main curvature value, Gaussian curvature value and shape index, and determine the point cloud boundary feature by calculating the local surface direction entropy value;

[0046] The second unit is used to construct four spherical search areas with radii that are in a doubly increasing relationship for each point in the enhanced feature vector, construct a local geometric descriptor by calculating the eigenvalues ​​of the covariance matrix of the points in the spherical search area, input the local geometric descriptor and the point cloud boundary feature into a three-branch spatial attention network, extract the spatial distribution information, neighborhood structure information and boundary shape information of the point respectively, and perform feature weighted fusion through a self-attention mechanism to obtain spatial features, add the spatial features to a multi-level geometric feature extraction network, extract curvature change features, normal vector change features and shape change features in turn, use a feature pyramid structure to perform multi-scale feature fusion to obtain geometric features, construct a bidirectional feature propagation network to calculate the temporal dependency of the geometric features to obtain propagation features, and perform jump connections and residual connections on the propagation features and the enhanced feature vector to obtain fused feature data;

[0047] The third unit is used to calculate the feature matching loss value, the spatial continuity loss value and the category distribution loss value, construct a segmentation energy function, calculate the gradient value and the energy loss value of the segmentation energy function based on the variational inference criterion, stop the iteration and output the initial segmentation label when the energy loss value is less than a preset energy loss threshold or the energy loss change rate of three consecutive iterations is less than a preset loss change threshold, perform connectivity analysis on the points based on the initial segmentation label, assign points that are spatially connected and have the same initial segmentation label to the same segmentation area, determine the area to be processed, and merge the areas based on feature similarity and boundary continuity, and repeat the merging to obtain the final segmentation result.

[0048] According to a third aspect of the embodiments of the present invention,

[0049] An electronic device is provided, comprising:

[0050] processor;

[0051] a memory for storing processor-executable instructions;

[0052] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0053] According to a fourth aspect of the embodiments of the present invention,

[0054] A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the method described above is implemented.

[0055] In the present invention, by constructing a three-dimensional hash index and a point cloud density histogram, the original point cloud is adaptively hierarchically divided, which effectively solves the problems of uneven distribution and large scale differences of point cloud data, and provides a good data foundation for subsequent feature extraction and region segmentation. By combining regional density counting and geometric distribution coefficient, the noise points and outliers in the point cloud are accurately identified and eliminated, thereby improving the quality and reliability of the point cloud data. An enhanced feature vector integrating spatial distribution, neighborhood structure and boundary shape is constructed. Multi-scale and multi-modal point cloud features are extracted through a three-branch spatial attention network, which enhances the representation ability and discriminability of the features. A segmentation energy function including feature matching, spatial continuity and category distribution is constructed. Iterative optimization and adaptive convergence of point cloud labels are achieved through variational inference, which effectively balances data-driven and prior constraints and improves the accuracy and consistency of the segmentation results. In summary, the present invention constructs a complete set of adaptive hierarchical point cloud segmentation methods, which can adapt to point cloud segmentation tasks in different scenarios and for different objects, has stronger robustness, generalization and adaptability, and has broad application prospects in smart cities, autonomous driving, cultural relics protection and other fields. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 Schematic diagram of the process of a three-dimensional point cloud data segmentation method based on multi-scale point cloud features according to an embodiment of the present invention;

[0057] Figure 2 Schematic diagram of the structure of a three-dimensional point cloud data segmentation system based on multi-scale point cloud features according to an embodiment of the present invention. DETAILED DESCRIPTION

[0058] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0059] The following specific embodiments are used to describe the technical solution of the present invention in detail. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0060] Figure 1 FIG. 1 is a flow chart of a method for segmenting three-dimensional point cloud data based on multi-scale point cloud features according to an embodiment of the present invention. Figure 1 As shown, the method includes:

[0061] S1. Acquire original point cloud data, construct a three-dimensional hash index table and map the original point cloud data to the three-dimensional hash index table, calculate the neighborhood density value of the point in the original point cloud data and construct a point cloud density histogram, use the least squares method to fit the point cloud density histogram to obtain a density distribution curve, extract the local maximum value in the density distribution curve as a point cloud stratification reference value, adaptively divide the original point cloud data into layers based on the point cloud stratification reference value to obtain a point cloud sub-region set, calculate the regional density count value and geometric distribution coefficient of each point in the point cloud sub-region set, identify and eliminate noise points in the point cloud sub-region based on a preset regional density threshold and geometric distribution coefficient threshold, and for points that are not eliminated, construct an enhanced feature vector by calculating the main feature direction, main curvature value, Gaussian curvature value and shape index, and determine the point cloud boundary feature by calculating the local surface direction entropy value;

[0062] The three-dimensional hash index table is an index structure for efficiently storing and searching three-dimensional point cloud data. It improves query efficiency by dividing the space into discrete grid units and mapping the point cloud data using a hash algorithm. The neighborhood density value refers to the density of points within a certain range around a point in the point cloud data, which is used to measure the density of the point in the local area. The point cloud density histogram is a histogram formed based on the neighborhood density value statistics of each point in the point cloud data, which is used to represent the distribution of the point cloud at different density levels. The point cloud layer reference value refers to the value used to classify the point cloud by density or other characteristics when processing the point cloud data. The reference standard for stratification helps distinguish different areas or structures in the point cloud. The density count value refers to the number of points in the point cloud data within a specified area or voxel, which is used to indicate the density of points in the area. The geometric distribution coefficient is a coefficient that describes the uniformity of the distribution of point cloud data in space, and is usually used to evaluate the spatial distribution characteristics of point clouds. The Gaussian curvature is a mathematical quantity that describes the local curvature of a surface, especially in point cloud surface analysis, it is used to quantify the curvature properties of the surface. The local surface directional entropy is used to measure the uncertainty and complexity of the local surface direction of a point cloud, and is often used to describe and classify surface features.

[0063] In an optional embodiment,

[0064] Obtaining original point cloud data, constructing a three-dimensional hash index table and mapping the original point cloud data to the three-dimensional hash index table, calculating neighborhood density values ​​of points in the original point cloud data and constructing a point cloud density histogram, fitting the point cloud density histogram using the least squares method to obtain a density distribution curve, extracting local maximum values ​​in the density distribution curve as point cloud stratification reference values, and adaptively dividing the original point cloud data into layers based on the point cloud stratification reference values ​​to obtain a point cloud sub-region set including:

[0065] Obtaining original point cloud data, dividing the three-dimensional space into a cubic grid according to a preset side length value, calculating a corresponding grid index based on the three-dimensional coordinates of a point in the original point cloud data, storing the original point cloud data in a location of the grid index, constructing a three-dimensional hash index table, constructing a first spherical region centered on the point in the original point cloud data according to a preset first radius, counting the number of points falling within the first spherical region as a neighborhood density value of the point, dividing the neighborhood density value into a plurality of equal-width intervals, and counting the number of points within each of the equal-width intervals to construct a point cloud density histogram;

[0066] Based on the median value of each density interval and the corresponding number of points in the point cloud density histogram, a density distribution curve is obtained by least squares fitting, the density distribution curve is numerically differentiated, a local maximum point on the density distribution curve is detected based on a sign change of the numerical differentiation, and the density value corresponding to the local maximum point is used as a reference value for point cloud stratification;

[0067] For each point cloud layer reference value, a density tolerance value is set and points whose density values ​​fall within an interval centered on the point cloud layer reference value and within a range of the density tolerance value are searched in the original point cloud data. The found points are added to the same point cloud sub-region to obtain a point cloud sub-region set.

[0068] The cube grid is a grid structure that divides three-dimensional space into uniform cube units, and is commonly used for segmentation, downsampling and feature extraction of point cloud data. The density tolerance value is used to measure the tolerance range of density changes in a local area of ​​the point cloud, and is usually used in clustering or segmentation algorithms to determine whether an area can be considered as a similar density area.

[0069] Get the original point cloud data. Assume that the acquired point cloud data contains 10,000 points, each of which contains 3D coordinates (x, y, z) and other attribute information such as color and reflection intensity. Example data points: (1.2, 2.5, 3.7), (2.1, 3.2, 1.5), ..., (10.5, 9.8, 7.2).

[0070] Set the side length to 1. This divides the 3D space into a grid of cubes with a side length of 1. The index of each grid cell is determined by its position in the x, y, and z directions. For example, a point with coordinates (2.3, 4.5, 1.8) falls into the grid with index (2, 4, 1).

[0071] Build a 3D hash index table. Store point cloud data at the corresponding grid index. For example, point (2.3, 4.5, 1.8) is stored in the grid with index (2, 4, 1). The hash table allows for quick lookup of points in a specific grid.

[0072] Calculate the neighborhood density of a point. Set the first radius to 0.5. For example, using point (2.3, 4.5, 1.8) as an example, construct a first spherical region with a radius of 0.5 centered on that point. Count the number of points that fall within this spherical region. If the number is 5, the neighborhood density of that point is 5. Perform the same operation for each point in the point cloud data to calculate its neighborhood density.

[0073] Construct a point cloud density histogram. Divide the neighborhood density values ​​into multiple equal-width intervals, such as [0, 2), [2, 4), [4, 6), [6, 8), and [8, 10]. Count the number of points in each interval. For example, suppose there are 1000 points in the interval [0, 2), 3000 points in the interval [2, 4), 4000 points in the interval [4, 6), 1500 points in the interval [6, 8), and 500 points in the interval [8, 10]. The point cloud density histogram is composed of these intervals and the corresponding number of points.

[0074] Fit a density distribution curve. Using the median of each density interval (e.g., the median of [0, 2) is 1, the median of [2, 4) is 3, and so on) and the corresponding number of points as input, a density distribution curve is fitted using the least squares method. This curve describes the density distribution of the point cloud data.

[0075] Extract reference values ​​for point cloud stratification. Perform numerical differentiation on the density distribution curve and detect local maxima on the density distribution curve by observing the sign change of the numerical differentiation. For example, suppose local maxima are detected at density values ​​of 3 and 7. 3 and 7 are then used as reference values ​​for point cloud stratification.

[0076] Adaptively divide the point cloud into layers. For a point cloud layer reference value of 3, set the density tolerance to 0.5. Search the original point cloud data for points whose density values ​​fall within the interval [2.5, 3.5]. Add these points to the same point cloud subregion. Similarly, for a point cloud layer reference value of 7, set the density tolerance to 0.5, search for points whose density values ​​fall within the interval [6.5, 7.5], and add these points to another point cloud subregion. Finally, a set of point cloud subregions is obtained.

[0077] In this embodiment, by dividing the point cloud data into multiple sub-regions, each sub-region can be processed independently, thereby improving the parallelism and efficiency of point cloud processing. By dividing the space into a cubic grid and establishing a hash index, the complexity of spatial search is greatly reduced, the efficiency of subsequent density calculation and neighborhood search is improved, and traversal search of the entire point cloud is avoided. By fitting the density distribution curve, the noise caused by histogram discretization is eliminated, and the spherical area statistical density is used to ensure the isotropy of density calculation. In summary, this embodiment realizes adaptive density stratification of point cloud data, provides a reliable density stratification basis for subsequent point cloud segmentation and feature extraction, and helps to improve the accuracy and robustness of the point cloud processing algorithm.

[0078] In an optional embodiment,

[0079] Calculate the regional density count value and geometric distribution coefficient of each point in the point cloud sub-region set, identify and remove noise points in the point cloud sub-region based on the preset regional density threshold and geometric distribution coefficient threshold, and for the points that are not removed, calculate the main feature direction, main curvature value, Gaussian curvature value and shape index, construct an enhanced feature vector and determine the point cloud boundary features by calculating the local surface direction entropy value, including:

[0080] Taking the points in the point cloud sub-region set as the center, construct a second spherical region according to a preset second radius, count the number of points that fall within the second spherical region and belong to the same point cloud sub-region as the regional density count value, calculate the eigenvalues ​​of the covariance matrix of the points in the second spherical region, and calculate the eigenvalue ratio as the geometric distribution coefficient;

[0081] Determining whether the area density count value of each point in the point cloud sub-area set is less than a preset area density threshold or whether the geometric distribution coefficient is greater than a preset geometric distribution coefficient threshold, marking the point that meets either condition as a noise point and removing it from the original point cloud data, thereby obtaining point cloud data after noise filtering;

[0082] Calculating a local point cloud covariance matrix for each point in the noise-filtered point cloud data, determining a principal eigendirection based on an eigenvector of the local point cloud covariance matrix, calculating a principal curvature value based on a least squares fit of the local point cloud covariance matrix, calculating a Gaussian curvature value and a shape index based on the principal curvature value, and combining the three-dimensional coordinates of the point, the principal eigendirection, the principal curvature value, the Gaussian curvature value, and the shape index to construct an enhanced eigenvector;

[0083] Taking the point in the point cloud data after noise filtering as the center, a third spherical area is constructed according to a preset third radius, the entropy value of the direction vector in the third spherical area is calculated, and the boundary feature points of the point cloud are determined based on a preset entropy value threshold.

[0084] The local point cloud covariance matrix is ​​a matrix obtained by calculating the covariance between the coordinates of each point in the local neighborhood of the point cloud, and is used to describe the geometric shape and directionality of the point cloud in the local area. The main characteristic direction refers to the main direction obtained by decomposing the eigenvalues ​​of the local covariance matrix of the point cloud, which represents the main direction of the point cloud distribution in the area.

[0085] Get the original point cloud data, which contains a large number of spatial 3D points, each with a 3D coordinate (x, y, z). For example, a point cloud data contains 10,000 points, each with a coordinate value expressed in meters, distributed in a spatial range of 10mx10mx5m.

[0086] Segment the point cloud data into several sub-regions. This segmentation method can be based on spatial partitioning. For example, divide the point cloud data into a 1mx1mx1m grid, with each point in the grid constituting a point cloud sub-region. In this example, the point cloud can be divided into 10x10x5=500 sub-regions.

[0087] For each point cloud subregion, calculate the regional density count and geometric distribution coefficient for each point. For example, consider point P, setting the second radius to 0.1m. Construct a spherical region with a radius of 0.1m centered on point P. Count the number of points within this spherical region that belong to the same point cloud subregion, and use this as the regional density count for point P. Assume that the regional density count for point P is 20.

[0088] Calculate the covariance matrix for all points within the second spherical region where point P lies. Perform eigenvalue decomposition on the covariance matrix to obtain three eigenvalues, λ1, λ2, and λ3, which are sorted by size, with λ1 ≥ λ2 ≥ λ3. Calculate the eigenvalue ratio, λ1 / λ3, as the geometric distribution coefficient. Assume that the geometric distribution coefficient for point P is 5.

[0089] Repeat the above steps for all points in all point cloud sub-regions to calculate the regional density count value and geometric distribution coefficient of each point.

[0090] Set the area density threshold and the geometric distribution coefficient threshold. For example, the area density threshold is 10 and the geometric distribution coefficient threshold is 8.

[0091] Determine whether the area density count value of each point is less than the area density threshold, or whether the geometric distribution coefficient is greater than the geometric distribution coefficient threshold. If either condition is met, the point is marked as a noise point. For example, point P has an area density count value of 20 greater than 10 and a geometric distribution coefficient of 5 less than 8, so point P is not marked as a noise point.

[0092] All points marked as noise are removed from the original point cloud data to obtain the point cloud data after noise filtering.

[0093] For each point in the noise-filtered point cloud data, calculate the local point cloud covariance matrix. Set the first radius to 0.2m. Taking point P as an example, construct a spherical region with a radius of 0.2m centered on point P. Calculate the covariance matrix of all points within this spherical region as the local point cloud covariance matrix of point P.

[0094] Perform eigenvalue decomposition on the local point cloud covariance matrix to obtain three eigenvectors and three eigenvalues. The three eigenvectors represent the main characteristic directions of point P. Assume that the three eigenvectors are v1, v2, and v3 respectively.

[0095] The principal curvature values ​​k1 and k2 of point P are calculated by performing least squares quadratic surface fitting on the local point cloud.

[0096] The Gaussian curvature value K=k1*k2 and the shape index S=(k1+k2) / (k1-k2) are calculated based on the principal curvature values. Assume that k1=0.1, k2=0.05, K=0.005, and S=3.

[0097] The enhanced feature vector is constructed by combining the three-dimensional coordinates (x, y, z), main feature directions (v1, v2, v3), main curvature values ​​(k1, k2), Gaussian curvature value K, and shape index S of point P. For example, the enhanced feature vector of point P is (x, y, z, v1, v2, v3, k1, k2, K, S).

[0098] A third spherical region is constructed with a pre-set third radius, centered at a point in the noise-filtered point cloud data. For example, the third radius is set to 0.15m. For example, a spherical region with a radius of 0.15m is constructed with point P as the center.

[0099] Calculate the entropy of the direction vectors of all points within the third spherical region. Direction vectors can be obtained by calculating the vector difference between each point and its neighboring points. Entropy is used to measure the degree of dispersion of direction vectors.

[0100] The boundary feature points of the point cloud are determined based on a preset entropy threshold. For example, the entropy threshold is set to 0.8. If the entropy value of point P is greater than 0.8, point P is marked as a boundary feature point.

[0101] In this embodiment, noise points are identified and eliminated through regional density and geometric distribution coefficient, which effectively improves the quality of point cloud data and lays the foundation for subsequent point cloud processing tasks. By calculating the main feature direction, main curvature value, Gaussian curvature value and shape index, an enhanced feature vector is constructed, which more comprehensively describes the geometric features of the point cloud, which is conducive to improving the accuracy of point cloud recognition and classification. By calculating the local surface direction entropy value, the boundary feature points of the point cloud are effectively identified, providing reliable information for point cloud segmentation and boundary extraction. In summary, this embodiment realizes noise filtering, feature enhancement and boundary extraction of point cloud data, improves the quality and feature expression ability of point cloud data, and provides a reliable data foundation for subsequent precise segmentation and recognition.

[0102] S2. For each point in the enhanced feature vector, four spherical search regions with radii that are in a doubly increasing relationship are constructed. A local geometric descriptor is constructed by calculating the eigenvalues ​​of the covariance matrix of the points in the spherical search region. The local geometric descriptor and the point cloud boundary features are input into a three-branch spatial attention network, and the spatial distribution information, neighborhood structure information, and boundary shape information of the points are extracted respectively. The spatial features are weightedly fused through a self-attention mechanism to obtain the spatial features. The spatial features are added to a multi-level geometric feature extraction network, and curvature change features, normal vector change features, and shape change features are extracted in sequence. Multi-scale feature fusion is performed using a feature pyramid structure to obtain geometric features. A bidirectional feature propagation network is constructed to calculate the temporal dependency of the geometric features to obtain propagation features. The propagation features are jump-connected and residual-connected to the enhanced feature vector to obtain fused feature data.

[0103] The spherical search area refers to a three-dimensional area in point cloud data, centered on a certain point and consisting of all points within a specified radius from the point. It is commonly used for neighborhood search and feature extraction of point clouds. The local geometric descriptor is a way to describe the shape and structure of a local area in point cloud data by analyzing the geometric features of the area. It is commonly used in tasks such as point cloud registration, classification, and clustering. The three-branch spatial attention network is a network structure that utilizes three branches and processes the spatial information of input data through an attention mechanism to improve the network's understanding and extraction capabilities of spatial features. The feature pyramid structure is a multi-scale feature extraction method that enhances the model's learning ability for features of different scales by constructing multi-level feature maps. It is commonly used in tasks such as target detection and image segmentation.

[0104] In an optional embodiment,

[0105] For each point in the enhanced feature vector, four spherical search areas with radii that are in a doubly increasing relationship are constructed. A local geometric descriptor is constructed by calculating the eigenvalues ​​of the covariance matrix of the points in the spherical search area. The local geometric descriptor and the point cloud boundary features are input into a three-branch spatial attention network. The spatial distribution information, neighborhood structure information and boundary shape information of the points are extracted respectively and feature weighted fusion is performed through a self-attention mechanism to obtain spatial features. The spatial features are added to a multi-level geometric feature extraction network, and curvature change features, normal vector change features and shape change features are extracted in sequence. Multi-scale feature fusion is performed using a feature pyramid structure to obtain geometric features. A bidirectional feature propagation network is constructed to calculate the temporal dependency of the geometric features to obtain propagation features. The propagation features are skipped and residually connected with the enhanced feature vector to obtain fused feature data including:

[0106] Construct four concentric spherical regions with each point in the enhanced feature vector as the center. The radii of the four concentric spherical regions are in a doubly increasing relationship. The radius of the first spherical region is the preset initial search radius.

[0107] Extracting all points within the four concentric spherical regions, calculating a covariance matrix corresponding to each point within the four concentric spherical regions, performing eigenvalue decomposition on the covariance matrix to obtain three eigenvalues ​​and an eigenvector corresponding to each eigenvalue, arranging the eigenvalues ​​in order of magnitude, using the eigenvectors corresponding to the first two eigenvalues ​​as the main eigenvector, and constructing the three eigenvalues ​​and the main eigenvector as a local geometric descriptor;

[0108] Inputting the local geometric descriptor and the point cloud boundary feature into a pre-set three-branch spatial attention network, the first branch of the three-branch spatial attention network performs a convolution operation on the local geometric descriptor to extract spatial distribution features, the second branch performs a graph convolution operation on the local geometric descriptor to extract neighborhood structure features, and the third branch performs a convolution operation on the point cloud boundary feature to extract boundary shape features. The spatial distribution features, the neighborhood structure features, and the boundary shape features are weightedly fused through a self-attention mechanism to obtain spatial features;

[0109] Inputting the spatial features into a multi-level geometric feature extraction network, the multi-level geometric feature extraction network includes multiple feature extraction modules, each feature extraction module includes three parallel branches, the first parallel branch obtains a curvature change feature by calculating the local curvature difference of the point cloud, the second parallel branch obtains a normal vector change feature by calculating the degree of deviation of the local normal vector of the point cloud, and the third parallel branch obtains a shape change feature by calculating the difference of local geometric descriptors. The curvature change feature, the normal vector change feature, and the shape change feature are multi-scale fused through a feature pyramid structure to obtain a geometric feature;

[0110] The geometric features are input into a bidirectional feature propagation network, which adopts a bidirectional recurrent neural network structure. By transmitting the geometric features forward and backward in the point cloud sequence, the contextual information and temporal evolution law of the point cloud are modeled to obtain propagation features. The propagation features and the enhanced feature vectors are jump-connected and residual-connected to obtain fused feature data.

[0111] The boundary shape feature refers to the feature used to describe the boundary shape of an object or area, including the curvature of the boundary, shape complexity, etc., and is often used in shape analysis and object recognition. The local curvature difference of the point cloud describes the degree of difference in curvature between different points in the local area of ​​the point cloud, and is often used to analyze the uneven areas or abnormal points on the surface of the point cloud. The local normal vector deviation degree of the point cloud is calculated by calculating the degree of difference between the normal vector and the reference normal vector in the local area of ​​the point cloud, and is used to measure the change or irregularity of the surface shape of the area.

[0112] Obtain the point cloud data to be processed. This point cloud data can be acquired by a 3D scanner, depth camera, or other sensor, and each point is represented by (x, y, z) coordinates. For example, a point cloud can contain thousands of points, each representing a sample point on the surface of an object. To improve the efficiency and accuracy of subsequent processing, the acquired raw point cloud data is preprocessed. This includes removing noise points, outliers, and missing data points, and downsampling the point cloud data to reduce the data volume. The preprocessed point cloud data is denoted as P.

[0113] Perform feature enhancement on the preprocessed point cloud data P. Extract local features of the point cloud, such as local curvature, normal vector, and local density, and concatenate these local features with the coordinate information of the original point cloud data P to form an enhanced feature vector. Assuming that point cloud P contains 1000 points, each with coordinates (x, y, z), the dimension of the enhanced feature vector is 1000x6 (assuming three local features were extracted).

[0114] For each point in the enhanced feature vector, construct four spherical search regions with radii that increase by a factor of two. Set the initial search radius to r = 0.1. Construct four concentric spherical regions with radii of 0.1, 0.2, 0.4, and 0.8, centered around each point in the enhanced feature vector. For the first point in the enhanced feature vector, search for all points within a spherical region of radius 0.1 around it. Suppose 50 points are found.

[0115] Extract all points within the four spherical search areas and calculate the covariance matrix of the points within each spherical area. For the first point, 50 points are found within its spherical area with a radius of 0.1, and the covariance matrix of these 50 points is calculated.

[0116] Perform eigenvalue decomposition on each covariance matrix to obtain three eigenvalues ​​and the eigenvectors corresponding to each eigenvalue. Arrange the three eigenvalues ​​in order of magnitude, and use the eigenvectors corresponding to the first two eigenvalues ​​as the principal eigenvectors. These three eigenvalues ​​and the two principal eigenvectors are constructed as a local geometric descriptor. Assume that the three eigenvalues ​​of the first point within a spherical region with a radius of 0.1 are 10, 5, and 1, and the corresponding eigenvectors are v1, v2, and v3. Then the local geometric descriptor of this point is (10, 5, 1, v1, v2).

[0117] Extract point cloud boundary features. You can use an edge detection algorithm, such as the Canny edge detection algorithm, to extract boundary points from the point cloud data and use them as point cloud boundary features. Suppose 100 boundary points are extracted.

[0118] The local geometric descriptors and point cloud boundary features are fed into a three-branch spatial attention network. The first branch of the network performs a convolution operation on the local geometric descriptors to extract spatial distribution features, the second branch performs a graph convolution operation on the local geometric descriptors to extract neighborhood structure features, and the third branch performs a convolution operation on the point cloud boundary features to extract boundary shape features. The features extracted by these three branches are then weighted and fused through a self-attention mechanism to obtain spatial features.

[0119] Spatial features are input into a multi-level geometric feature extraction network. This network consists of multiple feature extraction modules, each of which has three parallel branches. The first branch calculates the local curvature differences of the point cloud to obtain curvature change features. The second branch calculates the deviation of the local normal vector of the point cloud to obtain normal vector change features. The third branch calculates the difference in local geometric descriptors to obtain shape change features. The features of these three branches are fused at multiple scales using a feature pyramid structure to obtain geometric features.

[0120] The geometric features are fed into a bidirectional feature propagation network. This network uses a bidirectional recurrent neural network structure to propagate geometric features forward and backward in the point cloud sequence, modeling the contextual information and temporal evolution of the point cloud to obtain propagated features.

[0121] The propagated features and enhanced feature vectors are skipped and residually connected to obtain fused feature data.

[0122] In this embodiment, by constructing a multi-scale spherical search area and extracting local geometric descriptors, the local geometric information of the point cloud can be captured more comprehensively, thereby improving the expression ability of the point cloud features. By combining the spatial attention mechanism and the multi-level geometric feature extraction network, the spatial distribution information, neighborhood structure information and boundary shape information of the point cloud can be effectively integrated, thereby enhancing the robustness of the point cloud features. By modeling the temporal dependency of the point cloud through the bidirectional feature propagation network, the global contextual information of the point cloud can be better captured, thereby improving the performance of tasks such as point cloud classification and segmentation. In summary, this embodiment realizes multi-scale extraction and multi-level fusion of point cloud features, effectively integrates spatial, geometric and temporal features through the deep learning network architecture, significantly improves the feature expression ability, and provides high-quality feature representation for subsequent point cloud segmentation tasks.

[0123] In an optional embodiment,

[0124] Performing skip connection and residual connection on the propagation feature and the enhanced feature vector to obtain fused feature data includes:

[0125] Taking the current layer as the starting point, perform average pooling downsampling on the propagation features and enhanced feature vectors of the first 50 layers of the current layer. Then, concatenate the propagation features and enhanced feature vectors of the first 50 layers after downsampling with the propagation features and enhanced feature vectors of the current layer in the feature dimension to obtain multi-scale jump features.

[0126] Multiply the multi-scale jump features with the preset query matrix weight, key matrix weight, and value matrix weight respectively, and add the corresponding bias terms to obtain the query matrix, key matrix, and value matrix. Multiply the query matrix by the transpose of the key matrix and divide it by the square root of the dimension of the key matrix. Then, pass it through the softmax function to obtain the attention weight matrix.

[0127] Multiplying the attention weight matrix by the value matrix to obtain an attention feature, concatenating the attention feature with the multi-scale jump feature, combining with a linear transformation operation to obtain a transformation feature, adding the transformation feature to the enhanced feature vector of the current layer element-by-element to obtain a residual feature, multiplying the residual feature by a preset residual gating weight matrix, multiplying the enhanced feature vector of the current layer by a preset enhanced gating weight matrix, combining with a gated bias term and passing through a sigmoid function to obtain a residual gated signal and an enhanced gated signal;

[0128] Multiply the residual gating signal by the residual feature, multiply the enhanced gating signal by the enhanced feature vector of the current layer, add the two products to obtain the initial fusion feature, multiply the initial fusion feature by the preset self-attention weight matrix, determine the self-attention weight vector in combination with the self-attention bias term, multiply the self-attention weight vector by the initial fusion feature to obtain the selection feature, repeat the calculation and obtain the selection feature corresponding to each layer, splice the selection features corresponding to each layer in the feature dimension to obtain the fusion feature.

[0129] The residual gating signal is a gating mechanism for neural networks, which enhances the learning ability of the network by controlling the residual signal of the information flow, and is usually used to improve the stability and expressiveness of the network. The enhanced gating signal refers to the introduction of a gating mechanism in the neural network to enhance the influence of certain information flows, thereby improving the network's learning ability and robustness for important features.

[0130] Preprocess the propagation features and enhancement features extracted from each network layer. Propagation features can be the output feature maps of an intermediate layer in the network, and enhancement features can be features learned through other auxiliary tasks. Assuming the network layer currently being processed is layer i, the propagation features and enhancement features of the first 50 layers (if i is less than 50, then the first i layers are collected). Average pooling and downsampling operations are performed on the collected propagation features and enhancement features of each layer. For example, a 2x2 average pooling kernel can be used for downsampling. The downsampled propagation features of the first 50 layers (or i layers) are concatenated with the propagation features of the current layer (i layers) in the feature dimension to obtain a multi-scale representation of the propagation features. Similarly, the downsampled enhancement features of the first 50 layers (or i layers) are concatenated with the enhancement features of the current layer (i layers) in the feature dimension to obtain a multi-scale representation of the enhancement features. Finally, the multi-scale representation of the propagation features and the multi-scale representation of the enhancement features are concatenated in the feature dimension to obtain a multi-scale jump feature. For example, assuming that the propagation feature dimension of the current layer is 256, and the propagation feature dimensions after downsampling of the first 50 layers are all 256, then the multi-scale representation dimension of the spliced ​​propagation features is 256*51=13056.

[0131] The multi-scale jump features are processed using the attention mechanism. The multi-scale jump features are multiplied by the preset query matrix weights, key matrix weights, and value matrix weights, respectively, and the corresponding bias terms are added to obtain the query matrix, key matrix, and value matrix. The query matrix and the transpose of the key matrix are matrix multiplied, and the result is divided by the square root of the key matrix dimension. The softmax function is then applied to obtain the attention weight matrix. The attention weight matrix is ​​multiplied by the value matrix to obtain the attention features. The attention features are concatenated with the multi-scale jump features. Afterwards, a linear transformation operation is performed on the concatenated features, for example, using a fully connected layer to perform feature transformation to obtain the transformed features.

[0132] The transformed features are element-wise added to the enhanced feature vector of the current layer to obtain the residual features. The residual features are multiplied by the preset residual gating weight matrix, and the enhanced feature vector of the current layer is multiplied by the preset enhanced gating weight matrix. The corresponding gated bias terms are added, and then the sigmoid function is applied to obtain the residual gated signal and the enhanced gated signal. The residual gated signal is multiplied by the residual features, and the enhanced gated signal is multiplied by the enhanced feature vector of the current layer. The two products are added to obtain the initial fused features.

[0133] Apply the self-attention mechanism to the initial fused features. Multiply the initial fused features by the pre-set self-attention weight matrix and add the self-attention bias term to obtain the self-attention weight vector. Multiply the self-attention weight vector by the initial fused features to obtain the selected features. Repeat this process for each layer to obtain the selected features corresponding to each layer. Concatenate the selected features corresponding to each layer along the feature dimension to obtain the final fused features.

[0134] For example, assuming that the network has 100 layers and the dimension of the initial fused features obtained in each layer is 512, the dimension of the final fused features is 512*100=51200.

[0135] In this embodiment, by fusing multi-scale features, contextual information of different scales can be captured, thereby enhancing the representation ability of features. The attention mechanism and gating mechanism can effectively screen and enhance important features, suppress the influence of noise and irrelevant information, thereby improving the robustness of the model. By fusing multi-scale features, attention mechanism and gating mechanism, the performance of the model on various tasks can be effectively improved. In summary, this embodiment realizes the deep fusion and optimal selection of point cloud features through multi-scale feature fusion, attention mechanism and gated residual learning, improves the discriminability and robustness of feature expression, and provides high-quality feature representation for subsequent point cloud segmentation tasks.

[0136] S3. Calculate the feature matching loss value, spatial continuity loss value and category distribution loss value, construct a segmentation energy function, calculate the gradient value and energy loss value of the segmentation energy function based on the variational inference criterion, stop the iteration and output the initial segmentation label when the energy loss value is less than the preset energy loss threshold or the energy loss change rate of three consecutive iterations is less than the preset loss change threshold, perform connectivity analysis on the points based on the initial segmentation label, assign points that are spatially connected and have the same initial segmentation label to the same segmentation area, determine the area to be processed, and merge the areas based on feature similarity and boundary continuity, and repeat the merging to obtain the final segmentation result.

[0137] The feature matching loss is a loss function used to measure the feature difference between two data samples or images. It is commonly used in tasks such as image matching and target detection to calculate the distance or similarity of feature vectors. The spatial continuity loss is used to measure the spatial continuity of adjacent pixels or points in an image or data. It is commonly used in tasks such as semantic segmentation and image denoising to ensure that adjacent regions have consistent features. The category distribution loss is a loss function used to measure the difference between the category distribution predicted by the model and the true category distribution. It is commonly used in classification problems, especially in multi-category classification tasks. The segmentation energy function is an objective function used in image segmentation. It guides the algorithm to optimize the segmentation effect by calculating the difference between the region partitioned after image segmentation and the actual target. The variational inference criterion is a criterion based on the variational inference method. It is used to calculate the posterior distribution of complex probability distributions through approximate inference and is commonly used in tasks such as Bayesian inference and probabilistic models. The energy loss change rate refers to the rate of change of energy loss as the training progresses during the optimization process. It is used to evaluate the convergence speed and optimization effect during model training. The connectivity analysis is a method for analyzing the connectivity of adjacent regions in images or point cloud data. It is commonly used in image segmentation, shape analysis, and network topology analysis.

[0138] In an optional embodiment,

[0139] Calculate the feature matching loss value, spatial continuity loss value and category distribution loss value, construct a segmentation energy function, calculate the gradient value and energy loss value of the segmentation energy function based on the variational inference criterion, stop the iteration and output the initial segmentation label when the energy loss value is less than the preset energy loss threshold or the energy loss change rate of three consecutive iterations is less than the preset loss change threshold, perform connectivity analysis on the points based on the initial segmentation label, assign points with spatially connected positions and the same initial segmentation label to the same segmentation area, determine the area to be processed and merge the areas based on feature similarity and boundary continuity, and repeat the merging to obtain the final segmentation result including:

[0140] The fused feature is added to a pre-set classifier to obtain a category prediction result, the category prediction result is used to determine the category label corresponding to the fused feature, the time series adjacent samples corresponding to the fused feature are extracted as adjacent features, the category prediction result is statistically analyzed to obtain a category distribution, the cross entropy loss function is used to calculate the degree of matching between the fused feature data and the category label to obtain a feature matching loss value, the pairwise potential function is used to calculate the spatial continuity of the adjacent features to obtain a spatial continuity loss value, and the information entropy function is used to calculate the overall consistency of the category distribution to obtain a category distribution loss value;

[0141] The feature matching loss value, the spatial continuity loss value, and the category distribution loss value are weighted and summed according to a preset weight coefficient to construct a segmentation energy function, and the gradient value and energy loss value of the segmentation energy function are calculated based on the variational inference criterion. When the energy loss value is less than a preset energy loss threshold or the energy loss change rate for three consecutive iterations is less than a preset energy loss change rate, the iteration is stopped and the initial segmentation label is output;

[0142] Based on the initial segmentation label, the connectivity analysis of the points is performed, and the points with spatially connected positions and the same initial segmentation label are assigned to the same segmentation area. The number of points and the boundary curvature of each segmentation area are calculated, and the areas with the number of points less than a preset number threshold or the boundary curvature greater than a preset boundary curvature threshold are marked as areas to be processed. For each area to be processed, the feature similarity and boundary continuity between the current area to be processed and the adjacent areas are calculated. Based on the feature similarity and the boundary continuity, the merging score corresponding to each adjacent area is calculated, and the area to be processed is merged into the adjacent area with the highest merging score. The merging is repeated until there is no area that can be merged, and the final segmentation result is obtained.

[0143] The time-series adjacent samples refer to sample points that are temporally adjacent in time series data, and are usually used in time series analysis and prediction models to represent the time dependence of data. The paired potential function is a function that describes the interaction between two samples or objects, and is often used in image segmentation, image matching, graph neural networks and other fields to model the relationship between objects. The information entropy function is a function that quantifies the uncertainty or amount of information of a system or data, and is often used in data compression, feature selection and decision analysis to represent the randomness and complexity of the system. The boundary curvature is a mathematical quantity that describes the degree of curvature of a boundary line or surface, and is often used in shape analysis and image processing to reflect the smoothness or sharpness of the boundary.

[0144] The image data to be segmented is input and multi-dimensional features of the image, such as color, texture, and spatial position, are extracted. To more effectively capture local and global image information, a multi-scale feature extraction method is used. For example, texture features of the image are extracted at different scales and then fused to obtain more representative fused features. For example, for a 256x256 pixel color image, texture features can be extracted at three scales: local binary pattern (LBP) features calculated with window sizes of 3x3, 5x5, and 7x7. These three scales of LBP features are then concatenated with the image's RGB color features and pixel coordinate position features to form the final fused feature vector.

[0145] The fused features are input into a pre-trained classifier to obtain a category prediction for each pixel. This classifier can be a support vector machine (SVM), random forest, or neural network. Based on the category prediction results, a category label is determined for each pixel. For example, if the classifier outputs five categories, each pixel is assigned a category label between 0 and 4. Simultaneously, the category prediction results for all pixels are counted to obtain the category distribution of the image. Feature matching loss, spatial continuity loss, and category distribution loss are calculated. Feature matching loss measures the degree of match between the fused features and the category label and can be calculated using the cross-entropy loss function. Spatial continuity loss measures the spatial continuity between adjacent pixels and can be calculated using a pairwise potential function, such as the Potts model. Category distribution loss measures the overall consistency of the category distribution and can be calculated using the information entropy function. For example, if all pixels in an image belong to the same category, the information entropy is 0, indicating a highly consistent category distribution. These three loss values ​​are weighted and summed using pre-set weight coefficients, such as 0.5, 0.3, and 0.2, to construct the segmentation energy function. Using the variational inference criterion, the gradient and energy loss of the segmentation energy function are iteratively calculated. During each iteration, the pixel's class label is updated based on the gradient, and the energy loss is recalculated. When the energy loss falls below a preset energy loss threshold (e.g., 0.01), or the rate of change of energy loss for three consecutive iterations falls below a preset loss change threshold (e.g., 0.001), the iteration is terminated and the initial segmentation label is output.

[0146] Perform region merging and perform connectivity analysis on the pixels in the image based on the initial segmentation label. Pixels that are spatially connected and have the same initial segmentation label are assigned to the same segmentation region. Calculate the number of pixels and boundary curvature of each segmentation region. Mark regions with a number of pixels less than a preset threshold (e.g., 10) or a boundary curvature greater than a preset boundary curvature threshold (e.g., 0.5) as regions to be processed. For each region to be processed, calculate the feature similarity and boundary continuity between it and the adjacent regions. Feature similarity can be calculated based on fusion features, such as using cosine similarity. Boundary continuity can be calculated based on the length and curvature change of the shared boundary between adjacent regions. Calculate the merge score corresponding to each adjacent region based on feature similarity and boundary continuity. Merge the region to be processed into the adjacent region with the highest merge score. Repeat the merging operation until there is no region that can be merged, and obtain the final segmentation result. For example, a region to be processed has three adjacent regions, and their merge scores are calculated to be 0.8, 0.6, and 0.2, respectively. Then, merge the region to be processed into the adjacent region with a merge score of 0.8.

[0147] This embodiment, by fusing multidimensional features and employing a variational inference criterion, more accurately captures both local and global image information, thereby improving segmentation accuracy. Presetting weight coefficients and thresholds simplifies parameter setup, making the method easier to use. Connectivity analysis ensures the integrity of the segmented regions. Combining local features with global constraints adaptively processes objects of varying scales, maintaining boundary accuracy while reducing the risk of over-segmentation and under-segmentation. In summary, this embodiment achieves high-quality segmentation of point cloud data, providing accurate, coherent, and physically meaningful segmentation results, laying a solid foundation for subsequent applications of point cloud data.

[0148] In an optional embodiment,

[0149] The spatial continuity of the adjacent features is calculated using a pairwise potential function to obtain the spatial continuity loss as shown in the following formula:

[0150]

[0151] Among them, L spatial represents the spatial continuity loss, N represents the total number of points, and f i represents the fusion feature corresponding to the i-th point, f j represents the fusion feature corresponding to the jth point, p i represents the spatial coordinates of the i-th point, p j represents the spatial coordinates of the jth point, σ f The scale parameter representing the feature similarity, σ p The scale parameter representing the spatial distance, ||f i-f j || 2 represents the distance between point i and point j in the feature space, ||p i -p j || 2 Represents the distance between point i and point j in space.

[0152] In this embodiment, by constraining the feature consistency of adjacent points, the influence of noise and outliers can be effectively reduced, and the accuracy of point cloud segmentation can be improved. By learning the spatial continuity characteristics of point cloud data, the generalization ability of the model can be improved, so that it can better adapt to new point cloud data. A smooth distance measurement is achieved through an exponential function, which ensures the continuity of the segmentation boundary in features and space, and reduces the influence of noise and outliers. In summary, this embodiment combines the constraints of feature space and physical space to achieve the spatial continuity of point cloud segmentation results, improves the accuracy and smoothness of the segmentation boundary, and provides an effective optimization target for high-quality point cloud segmentation.

[0153] Figure 2 FIG is a structural diagram of a three-dimensional point cloud data segmentation system based on multi-scale point cloud features according to an embodiment of the present invention. Figure 2 As shown, the system includes:

[0154] The first unit is used to obtain original point cloud data, construct a three-dimensional hash index table and map the original point cloud data to the three-dimensional hash index table, calculate the neighborhood density value of the point in the original point cloud data and construct a point cloud density histogram, use the least squares method to fit the point cloud density histogram to obtain a density distribution curve, extract the local maximum value in the density distribution curve as a point cloud stratification reference value, adaptively divide the original point cloud data into layers based on the point cloud stratification reference value to obtain a point cloud sub-region set, calculate the regional density count value and geometric distribution coefficient of each point in the point cloud sub-region set, identify and eliminate noise points in the point cloud sub-region based on a preset regional density threshold and geometric distribution coefficient threshold, and for points that are not eliminated, construct an enhanced feature vector by calculating the main feature direction, main curvature value, Gaussian curvature value and shape index, and determine the point cloud boundary feature by calculating the local surface direction entropy value;

[0155] The second unit is used to construct four spherical search areas with radii that are in a doubly increasing relationship for each point in the enhanced feature vector, construct a local geometric descriptor by calculating the eigenvalues ​​of the covariance matrix of the points in the spherical search area, input the local geometric descriptor and the point cloud boundary feature into a three-branch spatial attention network, extract the spatial distribution information, neighborhood structure information and boundary shape information of the point respectively, and perform feature weighted fusion through a self-attention mechanism to obtain spatial features, add the spatial features to a multi-level geometric feature extraction network, extract curvature change features, normal vector change features and shape change features in turn, use a feature pyramid structure to perform multi-scale feature fusion to obtain geometric features, construct a bidirectional feature propagation network to calculate the temporal dependency of the geometric features to obtain propagation features, and perform jump connections and residual connections on the propagation features and the enhanced feature vector to obtain fused feature data;

[0156] The third unit is used to calculate the feature matching loss value, the spatial continuity loss value and the category distribution loss value, construct a segmentation energy function, calculate the gradient value and the energy loss value of the segmentation energy function based on the variational inference criterion, stop the iteration and output the initial segmentation label when the energy loss value is less than a preset energy loss threshold or the energy loss change rate of three consecutive iterations is less than a preset loss change threshold, perform connectivity analysis on the points based on the initial segmentation label, assign points that are spatially connected and have the same initial segmentation label to the same segmentation area, determine the area to be processed, and merge the areas based on feature similarity and boundary continuity, and repeat the merging to obtain the final segmentation result.

[0157] According to a third aspect of the embodiments of the present invention,

[0158] An electronic device is provided, comprising:

[0159] processor;

[0160] a memory for storing processor-executable instructions;

[0161] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0162] According to a fourth aspect of the embodiments of the present invention,

[0163] A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the method described above is implemented.

[0164] The present invention may be a method, an apparatus, a system and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing various aspects of the present invention.

[0165] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A three-dimensional point cloud data segmentation method based on multi-scale point cloud features, characterized in that: include: Obtain original point cloud data, construct a three-dimensional hash index table and map the original point cloud data to the three-dimensional hash index table, calculate the neighborhood density value of the point in the original point cloud data and construct a point cloud density histogram, use the least squares method to fit the point cloud density histogram to obtain a density distribution curve, extract the local maximum value in the density distribution curve as a point cloud stratification reference value, adaptively divide the original point cloud data into layers based on the point cloud stratification reference value to obtain a point cloud sub-region set, calculate the regional density count value and geometric distribution coefficient of each point in the point cloud sub-region set, identify and eliminate noise points in the point cloud sub-region based on a preset regional density threshold and geometric distribution coefficient threshold, and for points that are not eliminated, construct an enhanced feature vector by calculating the main feature direction, main curvature value, Gaussian curvature value and shape index, and determine the point cloud boundary feature by calculating the local surface direction entropy value; For each point in the enhanced feature vector, four spherical search areas with radii that are in a doubly increasing relationship are constructed. A local geometric descriptor is constructed by calculating the eigenvalues ​​of the covariance matrix of the points in the spherical search area. The local geometric descriptor and the point cloud boundary features are input into a three-branch spatial attention network. The spatial distribution information, neighborhood structure information, and boundary shape information of the points are extracted respectively, and feature weighted fusion is performed through a self-attention mechanism to obtain spatial features. The spatial features are added to a multi-level geometric feature extraction network. Curvature change features, normal vector change features, and shape change features are extracted in sequence. Multi-scale feature fusion is performed using a feature pyramid structure to obtain geometric features. A bidirectional feature propagation network is constructed to calculate the temporal dependency of the geometric features to obtain propagation features. The propagation features are skipped and residually connected with the enhanced feature vector to obtain fused feature data. Calculate the feature matching loss value, spatial continuity loss value and category distribution loss value, construct a segmentation energy function, calculate the gradient value and energy loss value of the segmentation energy function based on the variational inference criterion, stop the iteration and output the initial segmentation label when the energy loss value is less than the preset energy loss threshold or the energy loss change rate of three consecutive iterations is less than the preset loss change threshold, perform connectivity analysis on the points based on the initial segmentation label, assign points that are spatially connected and have the same initial segmentation label to the same segmentation area, determine the area to be processed, and merge the areas based on feature similarity and boundary continuity, and repeat the merging to obtain the final segmentation result.

2. The method according to claim 1, characterized in that Obtaining original point cloud data, constructing a three-dimensional hash index table and mapping the original point cloud data to the three-dimensional hash index table, calculating neighborhood density values ​​of points in the original point cloud data and constructing a point cloud density histogram, fitting the point cloud density histogram using the least squares method to obtain a density distribution curve, extracting local maximum values ​​in the density distribution curve as point cloud stratification reference values, and adaptively dividing the original point cloud data into layers based on the point cloud stratification reference values ​​to obtain a point cloud sub-region set including: Obtaining original point cloud data, dividing the three-dimensional space into a cubic grid according to a preset side length value, calculating a corresponding grid index based on the three-dimensional coordinates of a point in the original point cloud data, storing the original point cloud data in a location of the grid index, constructing a three-dimensional hash index table, constructing a first spherical region centered on the point in the original point cloud data according to a preset first radius, counting the number of points falling within the first spherical region as a neighborhood density value of the point, dividing the neighborhood density value into a plurality of equal-width intervals, and counting the number of points within each of the equal-width intervals to construct a point cloud density histogram; Based on the median value of each density interval and the corresponding number of points in the point cloud density histogram, a density distribution curve is obtained by least squares fitting, the density distribution curve is numerically differentiated, a local maximum point on the density distribution curve is detected based on a sign change of the numerical differentiation, and the density value corresponding to the local maximum point is used as a reference value for point cloud stratification; For each point cloud layer reference value, a density tolerance value is set and points whose density values ​​fall within an interval centered on the point cloud layer reference value and within a range of the density tolerance value are searched in the original point cloud data. The found points are added to the same point cloud sub-region to obtain a point cloud sub-region set.

3. The method according to claim 1, characterized in that Calculate the regional density count value and geometric distribution coefficient of each point in the point cloud sub-region set, identify and remove noise points in the point cloud sub-region based on the preset regional density threshold and geometric distribution coefficient threshold, and for the points that are not removed, calculate the main feature direction, main curvature value, Gaussian curvature value and shape index, construct an enhanced feature vector and determine the point cloud boundary features by calculating the local surface direction entropy value, including: Taking the points in the point cloud sub-region set as the center, construct a second spherical region according to a preset second radius, count the number of points that fall within the second spherical region and belong to the same point cloud sub-region as the regional density count value, calculate the eigenvalues ​​of the covariance matrix of the points in the second spherical region, and calculate the eigenvalue ratio as the geometric distribution coefficient; Determining whether the area density count value of each point in the point cloud sub-area set is less than a preset area density threshold or whether the geometric distribution coefficient is greater than a preset geometric distribution coefficient threshold, marking the point that meets either condition as a noise point and removing it from the original point cloud data, thereby obtaining point cloud data after noise filtering; Calculating a local point cloud covariance matrix for each point in the noise-filtered point cloud data, determining a principal eigendirection based on an eigenvector of the local point cloud covariance matrix, calculating a principal curvature value based on a least squares fit of the local point cloud covariance matrix, calculating a Gaussian curvature value and a shape index based on the principal curvature value, and combining the three-dimensional coordinates of the point, the principal eigendirection, the principal curvature value, the Gaussian curvature value, and the shape index to construct an enhanced eigenvector; Taking the point in the point cloud data after noise filtering as the center, a third spherical area is constructed according to a preset third radius, the entropy value of the direction vector in the third spherical area is calculated, and the boundary feature points of the point cloud are determined based on a preset entropy value threshold.

4. The method according to claim 1, wherein For each point in the enhanced feature vector, four spherical search areas with radii that are in a doubly increasing relationship are constructed. A local geometric descriptor is constructed by calculating the eigenvalues ​​of the covariance matrix of the points in the spherical search area. The local geometric descriptor and the point cloud boundary features are input into a three-branch spatial attention network. The spatial distribution information, neighborhood structure information and boundary shape information of the points are extracted respectively and feature weighted fusion is performed through a self-attention mechanism to obtain spatial features. The spatial features are added to a multi-level geometric feature extraction network, and curvature change features, normal vector change features and shape change features are extracted in sequence. Multi-scale feature fusion is performed using a feature pyramid structure to obtain geometric features. A bidirectional feature propagation network is constructed to calculate the temporal dependency of the geometric features to obtain propagation features. The propagation features are skipped and residually connected with the enhanced feature vector to obtain fused feature data including: Construct four concentric spherical regions with each point in the enhanced feature vector as the center. The radii of the four concentric spherical regions are in a doubly increasing relationship. The radius of the first spherical region is the preset initial search radius. Extracting all points within the four concentric spherical regions, calculating a covariance matrix corresponding to each point within the four concentric spherical regions, performing eigenvalue decomposition on the covariance matrix to obtain three eigenvalues ​​and an eigenvector corresponding to each eigenvalue, arranging the eigenvalues ​​in order of magnitude, using the eigenvectors corresponding to the first two eigenvalues ​​as the main eigenvector, and constructing the three eigenvalues ​​and the main eigenvector as a local geometric descriptor; Inputting the local geometric descriptor and the point cloud boundary feature into a pre-set three-branch spatial attention network, the first branch of the three-branch spatial attention network performs a convolution operation on the local geometric descriptor to extract spatial distribution features, the second branch performs a graph convolution operation on the local geometric descriptor to extract neighborhood structure features, and the third branch performs a convolution operation on the point cloud boundary feature to extract boundary shape features. The spatial distribution features, the neighborhood structure features, and the boundary shape features are weightedly fused through a self-attention mechanism to obtain spatial features; Inputting the spatial features into a multi-level geometric feature extraction network, the multi-level geometric feature extraction network includes multiple feature extraction modules, each feature extraction module includes three parallel branches, the first parallel branch obtains a curvature change feature by calculating the local curvature difference of the point cloud, the second parallel branch obtains a normal vector change feature by calculating the degree of deviation of the local normal vector of the point cloud, and the third parallel branch obtains a shape change feature by calculating the difference of local geometric descriptors. The curvature change feature, the normal vector change feature, and the shape change feature are multi-scale fused through a feature pyramid structure to obtain a geometric feature; The geometric features are input into a bidirectional feature propagation network, which adopts a bidirectional recurrent neural network structure. By transmitting the geometric features forward and backward in the point cloud sequence, the contextual information and temporal evolution law of the point cloud are modeled to obtain propagation features. The propagation features and the enhanced feature vectors are jump-connected and residual-connected to obtain fused feature data.

5. The method according to claim 4, characterized in that Performing skip connection and residual connection on the propagation feature and the enhanced feature vector to obtain fused feature data includes: Taking the current layer as the starting point, perform average pooling downsampling on the propagation features and enhanced feature vectors of the first 50 layers of the current layer. Then, concatenate the propagation features and enhanced feature vectors of the first 50 layers after downsampling with the propagation features and enhanced feature vectors of the current layer in the feature dimension to obtain multi-scale jump features. Multiply the multi-scale jump features with the preset query matrix weight, key matrix weight, and value matrix weight respectively, and add the corresponding bias terms to obtain the query matrix, key matrix, and value matrix. Multiply the query matrix by the transpose of the key matrix and divide it by the square root of the dimension of the key matrix. Then, pass it through the softmax function to obtain the attention weight matrix. Multiplying the attention weight matrix by the value matrix to obtain an attention feature, concatenating the attention feature with the multi-scale jump feature, combining with a linear transformation operation to obtain a transformation feature, adding the transformation feature to the enhanced feature vector of the current layer element-by-element to obtain a residual feature, multiplying the residual feature by a preset residual gating weight matrix, multiplying the enhanced feature vector of the current layer by a preset enhanced gating weight matrix, combining with a gated bias term and passing through a sigmoid function to obtain a residual gated signal and an enhanced gated signal; Multiply the residual gating signal by the residual feature, multiply the enhanced gating signal by the enhanced feature vector of the current layer, add the two products to obtain the initial fusion feature, multiply the initial fusion feature by the preset self-attention weight matrix, determine the self-attention weight vector in combination with the self-attention bias term, multiply the self-attention weight vector by the initial fusion feature to obtain the selection feature, repeat the calculation and obtain the selection feature corresponding to each layer, splice the selection features corresponding to each layer in the feature dimension to obtain the fusion feature.

6. The method according to claim 1, characterized in that Calculate the feature matching loss value, spatial continuity loss value and category distribution loss value, construct a segmentation energy function, calculate the gradient value and energy loss value of the segmentation energy function based on the variational inference criterion, stop the iteration and output the initial segmentation label when the energy loss value is less than the preset energy loss threshold or the energy loss change rate of three consecutive iterations is less than the preset loss change threshold, perform connectivity analysis on the points based on the initial segmentation label, assign points with spatially connected positions and the same initial segmentation label to the same segmentation area, determine the area to be processed and merge the areas based on feature similarity and boundary continuity, and repeat the merging to obtain the final segmentation result including: The fused feature is added to a pre-set classifier to obtain a category prediction result, the category prediction result is used to determine the category label corresponding to the fused feature, the time series adjacent samples corresponding to the fused feature are extracted as adjacent features, the category prediction result is statistically analyzed to obtain a category distribution, the cross entropy loss function is used to calculate the degree of matching between the fused feature data and the category label to obtain a feature matching loss value, the pairwise potential function is used to calculate the spatial continuity of the adjacent features to obtain a spatial continuity loss value, and the information entropy function is used to calculate the overall consistency of the category distribution to obtain a category distribution loss value; The feature matching loss value, the spatial continuity loss value and the category distribution loss value are weighted and summed according to a preset weight coefficient to construct a segmentation energy function. The gradient value and energy loss value of the segmentation energy function are calculated based on the variational inference criterion. When the energy loss value is less than a preset energy loss threshold or the energy loss change rate for three consecutive iterations is less than a preset threshold, the segmentation energy function is automatically generated. When the energy loss rate changes, stop the iteration and output the initial segmentation label; Based on the initial segmentation label, the connectivity analysis of the points is performed, and the points with spatially connected positions and the same initial segmentation label are assigned to the same segmentation area. The number of points and the boundary curvature of each segmentation area are calculated, and the areas with the number of points less than a preset number threshold or the boundary curvature greater than a preset boundary curvature threshold are marked as areas to be processed. For each area to be processed, the feature similarity and boundary continuity between the current area to be processed and the adjacent areas are calculated. Based on the feature similarity and the boundary continuity, the merging score corresponding to each adjacent area is calculated, and the area to be processed is merged into the adjacent area with the highest merging score. The merging is repeated until there is no area that can be merged, and the final segmentation result is obtained.

7. The method according to claim 6, characterized in that The spatial continuity of the adjacent features is calculated using a pairwise potential function to obtain the spatial continuity loss value as shown in the following formula: Among them, L spatial represents the spatial continuity loss, N represents the total number of points, and f i represents the fusion feature corresponding to the i-th point, f j represents the fusion feature corresponding to the jth point, p i represents the spatial coordinates of the i-th point, p j represents the spatial coordinates of the jth point, σ f The scale parameter representing the feature similarity, σ p The scale parameter representing the spatial distance, ||f i -f j || 2 represents the distance between point i and point j in the feature space, ||p i -p j || 2 Represents the distance between point i and point j in space.

8. A three-dimensional point cloud data segmentation system based on multi-scale point cloud features, used to implement the method according to any one of claims 1 to 7, characterized in that: include: The first unit is used to obtain original point cloud data, construct a three-dimensional hash index table and map the original point cloud data to the three-dimensional hash index table, calculate the neighborhood density value of the point in the original point cloud data and construct a point cloud density histogram, use the least squares method to fit the point cloud density histogram to obtain a density distribution curve, extract the local maximum value in the density distribution curve as a point cloud stratification reference value, adaptively divide the original point cloud data into layers based on the point cloud stratification reference value to obtain a point cloud sub-region set, calculate the regional density count value and geometric distribution coefficient of each point in the point cloud sub-region set, identify and eliminate noise points in the point cloud sub-region based on a preset regional density threshold and geometric distribution coefficient threshold, and for points that are not eliminated, construct an enhanced feature vector by calculating the main feature direction, main curvature value, Gaussian curvature value and shape index, and determine the point cloud boundary feature by calculating the local surface direction entropy value; The second unit is used to construct four spherical search areas with radii that are in a doubly increasing relationship for each point in the enhanced feature vector, construct a local geometric descriptor by calculating the eigenvalues ​​of the covariance matrix of the points in the spherical search area, input the local geometric descriptor and the point cloud boundary feature into a three-branch spatial attention network, extract the spatial distribution information, neighborhood structure information and boundary shape information of the point respectively, and perform feature weighted fusion through a self-attention mechanism to obtain spatial features, add the spatial features to a multi-level geometric feature extraction network, extract curvature change features, normal vector change features and shape change features in turn, use a feature pyramid structure to perform multi-scale feature fusion to obtain geometric features, construct a bidirectional feature propagation network to calculate the temporal dependency of the geometric features to obtain propagation features, and perform jump connections and residual connections on the propagation features and the enhanced feature vector to obtain fused feature data; The third unit is used to calculate the feature matching loss value, the spatial continuity loss value and the category distribution loss value, construct a segmentation energy function, calculate the gradient value and the energy loss value of the segmentation energy function based on the variational inference criterion, stop the iteration and output the initial segmentation label when the energy loss value is less than a preset energy loss threshold or the energy loss change rate of three consecutive iterations is less than a preset loss change threshold, perform connectivity analysis on the points based on the initial segmentation label, assign points that are spatially connected and have the same initial segmentation label to the same segmentation area, determine the area to be processed, and merge the areas based on feature similarity and boundary continuity, and repeat the merging to obtain the final segmentation result.

9. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images