Point cloud tree segmentation method and system based on cooperative attention and density adaptive voxelization, terminal and storage medium

Abstract

The application relates to the technical field of point cloud data processing, and discloses a point cloud tree segmentation method and system based on collaborative attention and density adaptive voxelization, a terminal and a storage medium.The method comprises the following steps: performing density adaptive voxelization on original forest point cloud to construct a non-uniform voxel grid; then inputting the non-uniform voxel grid into a sparse convolutional neural network to perform feature extraction by using a collaborative spatial channel attention module; and finally performing model training by using a boundary perception composite loss function, so that the segmentation result of each tree in the forest point cloud can be extracted.Under the premise of ensuring accuracy, the application optimizes the allocation of computing resources through adaptive voxelization, improves the efficiency of processing large-scale forest point cloud, and finally generates a segmentation result with more accurate geometric morphology and clearer contour, thereby providing a higher-quality data basis for subsequent forestry parameter extraction.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of point cloud data processing, and particularly relates to a point cloud individual tree segmentation method and system based on collaborative attention and density adaptive voxelization, a terminal and a computer readable storage medium. BACKGROUND

[0002] With the rapid development of three-dimensional sensing technology, in the fields of intelligent forestry, ecological monitoring and carbon storage estimation, automatically and accurately segmenting individual trees (ITS) from large-scale point clouds is a core pre-task for realizing fine management and evaluation of forestry resources.

[0003] However, the current mainstream method generally uses fixed-size voxels to discretize point clouds, which fundamentally contradicts the inherent non-uniform density characteristics of tree point clouds (i.e., dense trunks and sparse crowns). Standard three-dimensional convolutional networks homogenize all spatial positions and feature channels when processing feature maps, failing to distinguish their contribution to the segmentation task. Moreover, mainstream segmentation loss functions, such as cross-entropy or Dice loss, are essentially based on the measurement of region overlap. Such loss functions tend to optimize the main body region with a large proportion, and are insensitive to errors on the boundary points which only account for a small proportion.

[0004] Therefore, the prior art still needs to be improved and developed. SUMMARY

[0005] The main purpose of the present application is to provide a point cloud individual tree segmentation method and system based on collaborative attention and density adaptive voxelization, a terminal and a computer readable storage medium, which aims to solve the problem of insufficient boundary segmentation accuracy caused by fixed voxelization and lack of advanced feature refinement mechanism when processing tree point clouds with non-uniform density in the prior art.

[0006] To achieve the above purpose, the present application provides a point cloud individual tree segmentation method based on collaborative attention and density adaptive voxelization, which comprises the following steps:

[0007] Obtain the original forest point cloud, calculate the density value of each sample point in the original forest point cloud, and perform mapping processing on all the density values to obtain a non-uniform voxel grid;

[0008] Input the voxel grid into a sparse convolutional neural network to output a voxel feature map in the original forest point cloud, input the voxel feature map into a spatial channel attention model for multiple pooling processing to output a refined feature map, and input the refined feature map into a segmentation head to output the semantic prediction probability of each sample point.

[0009] constructing a boundary loss function by using the feature representation of all boundary points in the refined feature map, constructing a region loss function according to the real label and the semantic prediction probability of each sample point, constructing an offset loss function according to the real instance center offset vector and the predicted instance center offset vector of each sample point, and optimizing and training the sparse convolutional neural network, the spatial channel attention model and the segmentation head by using the boundary loss function, the region loss function and the offset loss function;

[0010] inputting a target forest point cloud into the trained sparse convolutional neural network, the spatial channel attention model and the segmentation head, outputting a semantic class prediction result of each voxel in the target forest point cloud, and clustering all the semantic class prediction results by using a clustering algorithm to obtain a plurality of single tree segmentation results.

[0011] In addition, to achieve the above-mentioned purpose, the application further provides a point cloud single tree segmentation system based on collaborative attention and density adaptive voxelization, wherein the point cloud single tree segmentation system based on collaborative attention and density adaptive voxelization comprises:

[0012] a voxel grid construction module configured to acquire an original forest point cloud, calculate a density value of each sample point in the original forest point cloud, and perform mapping processing on all the density values to obtain a non-uniform voxel grid;

[0013] a collaborative attention module configured to input the voxel grid into a sparse convolutional neural network, output a voxel feature map in the original forest point cloud, input the voxel feature map into a spatial channel attention model for multiple times of pooling processing, output a refined feature map, input the refined feature map into a segmentation head, and output a semantic prediction probability of each sample point;

[0014] The model training module is configured to construct a boundary loss function by using the feature representation of all boundary points in the refined feature map, construct a region loss function according to the true label and semantic prediction probability of each sample point, construct an offset loss function according to the true instance center offset vector and the predicted instance center offset vector of each sample point, and optimize and train the sparse convolutional neural network, the spatial channel attention model and the segmentation head by using the boundary loss function, the region loss function and the offset loss function.

[0015] The result detection module is configured to input a target forest point cloud into the trained sparse convolutional neural network, the spatial channel attention model and the segmentation head, output a semantic class prediction result of each voxel in the target forest point cloud, and cluster all the semantic class prediction results by using a clustering algorithm to obtain a plurality of single tree segmentation results.

[0016] In addition, to achieve the above object, the present application further provides a terminal, wherein the terminal comprises a memory, a processor and a point cloud single tree segmentation program based on collaborative attention and density adaptive voxelization stored in the memory and executable on the processor, and the point cloud single tree segmentation program based on collaborative attention and density adaptive voxelization implements the steps of the point cloud single tree segmentation method based on collaborative attention and density adaptive voxelization when executed by the processor.

[0017] In addition, to achieve the above object, the present application further provides a computer readable storage medium, wherein the computer readable storage medium stores a point cloud single tree segmentation program based on collaborative attention and density adaptive voxelization, and the point cloud single tree segmentation program based on collaborative attention and density adaptive voxelization implements the steps of the point cloud single tree segmentation method based on collaborative attention and density adaptive voxelization when executed by a processor.

[0018] In the present application, the original forest point cloud is obtained, the density value of each sample point in the original forest point cloud is calculated, and the non-uniform voxel grid is obtained by mapping all the density values; the voxel grid is input into a sparse convolutional neural network, and the voxel feature map in the original forest point cloud is output; the voxel feature map is input into a spatial channel attention model for multiple pooling processing, and the refined feature map is output; the refined feature map is input into a segmentation head, and the semantic prediction probability of each sample point is output; the boundary loss function is constructed by using the feature representation of all boundary points in the refined feature map, the region loss function is constructed according to the real label and the semantic prediction probability of each sample point, and the offset loss function is constructed according to the real instance center offset vector and the predicted instance center offset vector of each sample point; and the sparse convolutional neural network, the spatial channel attention model and the segmentation head are optimized and trained by using the boundary loss function, the region loss function and the offset loss function; according to the real instance center and the predicted instance center offset vector of each sample point, the offset loss function is constructed, and the sparse convolutional neural network, the spatial channel attention model and the segmentation head are optimized and trained by using the boundary loss function, the region loss function and the offset loss function; the target forest point cloud is input into the trained sparse convolutional neural network, spatial channel attention model and segmentation head, and the semantic class prediction result of each voxel in the target forest point cloud is output; and all the semantic class prediction results are clustered by using a clustering algorithm to obtain multiple single tree segmentation results. The present application optimizes the allocation of computing resources by adaptive voxelization under the premise of ensuring accuracy, improves the efficiency of processing large-scale forest point clouds, and finally generates segmentation results with more accurate geometric shapes and clearer contours, providing a higher-quality data basis for subsequent forestry parameter extraction. BRIEF DESCRIPTION OF DRAWINGS

[0019] Figure 1 is a flowchart of a preferred embodiment of the point cloud single tree segmentation method based on collaborative attention and density adaptive voxelization of the present application;

[0020] Figure 2 is a general architecture block diagram of a preferred embodiment of the point cloud single tree segmentation method based on collaborative attention and density adaptive voxelization of the present application;

[0021] Figure 3 is a flowchart of a spatial channel attention model of a preferred embodiment of the point cloud single tree segmentation method based on collaborative attention and density adaptive voxelization of the present application;

[0022] Figure 4 is a structure diagram of a preferred embodiment of the point cloud single tree segmentation system based on collaborative attention and density adaptive voxelization of the present application;

[0023] Figure 5 Structure diagram of a preferred embodiment of the terminal of the present application. DETAILED DESCRIPTION

[0024] To make the objectives, technical solutions and advantages of the present application clearer and more explicit, the present application is further described in detail below with reference to the drawings and examples. It should be understood that the specific examples described herein are only used to explain the present application and do not limit the present application.

[0025] The point cloud tree segmentation method based on collaborative attention and density adaptive voxelization described in the preferred embodiment of the present application, as shown in Figure 1 The point cloud tree segmentation method based on collaborative attention and density adaptive voxelization includes the following steps:

[0026] Step S10, obtain the original forest point cloud, calculate the density value of each sample point in the original forest point cloud, and perform mapping processing on all the density values to obtain a non-uniform voxel grid.

[0027] For the current tree segmentation method, a fixed size voxel is generally used to discretize the point cloud. There is a fundamental contradiction between this strategy and the inherent density non-uniformity of the tree point cloud (i.e. dense trunk, sparse crown). Larger voxel size will blur the fine geometric structure of the crown tip, resulting in loss of key information; while smaller voxel size will generate a large amount of redundant data in the dense area such as the trunk, resulting in huge computing and storage overhead.

[0028] Therefore, as shown in Figure 2 For the obtained original forest point cloud, the present application first constructs a non-uniform voxel grid to realize density adaptive voxelization. This part receives the original forest scene point cloud as input, replaces the fixed size downsampling step in the traditional method, and dynamically generates a multi-resolution voxel grid representation according to the local density of the point cloud.

[0029] Specifically, the original forest point cloud is obtained, a plurality of non-ground sample points are extracted from all sample points in the original forest point cloud, and all the non-ground sample points are paired to calculate the pair distance of each pair of non-ground sample points (wherein the pair distance refers to pairing the sample points obtained by sampling two by two, and calculating the Euclidean distance between them); the median of all the pair distances is defined as a global bandwidth parameter, and the density value of all the sample points is calculated according to the global bandwidth parameter:

[0030] ;

[0031] wherein, represents any one sample point, represents the density value of , Indicates the number of sample points. Indicates the index of the sample point. This represents the global bandwidth parameter. Indicates according to Scaled Gaussian kernel, Indicates the first Sample points; map all the density values ​​to obtain the high-density voxel grid, the medium-density voxel grid, and the low-density voxel grid.

[0032] Here, the density value corresponding to each point in the original forest point cloud is calculated (kernel density estimation is a non-parametric density estimation method that can provide a smooth density field); the bandwidth parameter can control the smoothness of the density estimation. In the process of adaptive adjustment of the point cloud, this application uses the global bandwidth parameter for control. This process is completely data-driven. Using the median can effectively resist the interference of outliers, and its physical meaning is that the influence range of the kernel function matches the typical point spacing of the point cloud, thereby avoiding the error caused by manual parameter tuning, while ensuring the robustness of the density estimation and effectively capturing subtle changes in the point cloud density.

[0033] The non-uniform voxel grid includes high-density regions (tree trunk), medium-density regions (main branches), and low-density regions (outer canopy). These regions refer to the same point cloud (i.e., a tree or forest scene, in this embodiment, the original forest point cloud), and are divided according to density values.

[0034] Step S20: Input the voxel grid into a sparse convolutional neural network to output a voxel feature map in the original forest point cloud. Input the voxel feature map into a spatial channel attention model for multiple pooling processes to output a refined feature map. Input the refined feature map into a segmentation head to output the semantic prediction probability of each sample point.

[0035] Specifically, the voxel grid is input into a sparse convolutional neural network (i.e., a sparse 3D convolutional neural network for biomedical image segmentation) for deep feature extraction, outputting a voxel feature map of the original forest point cloud; the voxel feature map is input into the channel attention model to output an intermediate feature map; the intermediate feature map is input into the spatial attention model to output a refined feature map; the refined feature map is input into a segmentation head, and the segmentation head outputs the category prediction, corresponding semantic prediction probability, and instance center offset vector for each sample point.

[0036] The spatial channel attention model (SSCA, Spatial and Channel Synergistic Attention) includes a channel attention model and a spatial attention model. A standard three-dimensional convolutional network homogenizes all spatial positions and feature channels when processing feature maps, and fails to distinguish their contribution to the segmentation task. For trees, which have extremely complex morphology, their branch points, branch connections, and crown boundaries intertwined with other trees obviously contain more key information for distinguishing different instances than smooth tree trunks. Therefore, in the embodiments disclosed in the present application, multi-scale feature extraction is performed on the voxel grid layer by layer, and the synergistic spatial channel attention module (as shown in Figure 3 ) is embedded in the skip connection path between the corresponding levels of the encoder and the decoder to refine the feature map extracted by the backbone network, enhance the significant features related to the segmentation task, and suppress irrelevant information.

[0037] Specifically, the voxel feature map is input into the channel attention model, which performs global pooling on the voxel feature map and outputs average channel descriptors and maximum channel descriptors; the average channel descriptors and the maximum channel descriptors are input into a network of double-layer fully connected layers, and a channel attention weight map is output; the channel attention weight map and the voxel feature map are multiplied element by element in the channel to realize channel attention weighting, and an intermediate feature map is obtained; the intermediate feature map is input into the spatial attention model, which performs average pooling and maximum pooling on the intermediate feature map to obtain average spatial descriptors and maximum spatial descriptors, respectively; the average spatial descriptors and the maximum spatial descriptors are spliced in the channel dimension of the spatial attention model to obtain a spliced feature; the spliced feature is processed by a three-dimensional sparse convolution layer to output a single-channel spatial attention weight map; and the spatial attention weight map and the intermediate feature map are multiplied element by element by broadcast to obtain a refined feature map.

[0038] In the embodiments disclosed in the present application, a synergistic spatial channel attention model is used to process three-dimensional sparse voxel data (i.e., voxel feature map F), which is sequentially passed through the channel attention model and the spatial attention model, and finally obtains a refined feature map .

[0039] Since the backbone network deals with sparse tensors, to maintain computational efficiency, the global pooling operation needs to be performed without "densifying" the tensor. Therefore, the present application implements global pooling by directly operating on the voxel feature map F (dimension: non-zero voxel number x feature channel number) of the sparse tensor, specifically including sparse global average pooling (Sparse Global Average Pooling) and sparse global maximum pooling (Sparse Global Max Pooling); wherein the sparse global average pooling obtains the mean value of the feature matrix in the dimension representing all non-zero voxels, and for each channel, the mean value of the channel in all non-zero voxels is taken; the sparse global maximum pooling calculates the maximum value in the dimension. The computational complexity of this method is only proportional to the number of non-zero voxels, perfectly maintaining the advantages of sparse computing, and finally generating two average channel descriptors and maximum channel descriptors , which capture the global distribution information and the most significant feature response of each channel, respectively. Then and are sent into a shared network consisting of two fully connected layers, which first compress the channel dimension to reduce parameters, then recover, and finally generate a channel attention weight map through a Sigmoid activation function. The generated attention map is multiplied (element-wise multiplication) with the original input feature map channel by channel to obtain an intermediate feature map weighted by channel attention, so that the network can enhance those feature channels that are essential to distinguish different parts of the tree (such as wood and leaves), while suppressing those channels containing noise or redundant information.

[0040] Further, after the channel attention model, a spatial attention model is followed, which emphasizes information-rich spatial regions by learning a three-dimensional spatial weight map. The intermediate feature map is averaged and maximized in the channel dimension of the channel attention model to generate two spatial descriptors with dimensions of 1xHxWxD (H represents height, W represents width, and D represents dimension): average spatial descriptor and maximum spatial descriptor , both of which capture the average and most salient feature information at each spatial location. The two spatial descriptors are concatenated in the channel dimension and then processed by a standard 3D sparse convolutional layer to generate a single-channel spatial attention weight map. Finally, the generated spatial attention map is element-wise broadcasted with the intermediate feature map to obtain the final refined output feature map (refined feature map). Through the above method, in the single tree segmentation task disclosed in the present application, the channel attention can learn to amplify those feature channels that encode "wood" or "leaves", while the spatial attention can learn to concentrate computing resources on the structural key points of the tree, such as the junction of the tree root and the ground, the branching points of the trunk, and the crown boundary with other trees. The synergistic effect of this dual attention mechanism enables the network to extract more robust and discriminative feature representations from noisy raw data.

[0041] Step S30, constructing a boundary loss function using the feature representation of all boundary points in the refined feature map, constructing a region loss function according to the true label and the semantic prediction probability of each sample point, constructing an offset loss function according to the true instance center offset vector and the predicted instance center offset vector of each sample point, and optimizing and training the sparse convolutional neural network, the spatial channel attention model and the segmentation head using the boundary loss function, the region loss function and the offset loss function. According to the true instance center and the predicted instance center offset vector of each sample point, an offset loss function is constructed, and the sparse convolutional neural network, the spatial channel attention model and the segmentation head are optimized and trained using the boundary loss function, the region loss function and the offset loss function.

[0042] Wherein, for the existing segmentation loss function, such as cross entropy or Dice loss, its essence is to measure the region overlap. This kind of loss function tends to optimize the main region with large proportion, and is insensitive to errors on the boundary points which only account for a small proportion. In forest scenes, the number of boundary points defining the tree outline is much less than that of internal points, resulting in a lack of strong and explicit supervision signal for boundary accuracy during model training. Therefore, although the segmented main region may be correct in a macroscopic sense, its boundary often presents fuzzy, smooth or jagged, which seriously affects the accuracy of subsequent estimation of key forestry parameters such as diameter at breast height and crown width. In order to fundamentally solve the problem of low segmentation accuracy in the boundary region of existing methods, the present application proposes a composite loss function. The loss function provides a more comprehensive and balanced supervision signal for network optimization by combining two loss terms with different properties.

[0043] Specifically, three-dimensional coordinates of all the sample points are acquired, and a k-d tree is constructed, for each sample point, the k-d tree is used to determine the k nearest neighboring points of the sample point to construct a field set; the field set of the current sample point is traversed, if there is at least one neighboring point whose semantic label is different from that of the sample point, the current sample point is defined as a boundary point, until all boundary points in all sample points are extracted; from the feature representation of all boundary points in the refined feature map, each boundary point is traversed, and all boundary points satisfying the semi-hard condition are defined as negative samples:

[0044] ;

[0045] wherein, represents the current traversed boundary point, represents the distance from the farthest boundary point, represents the negative sample, represents the feature representation of , represents the feature representation of , represents the feature representation of , represents a preset edge distance, represents the Euclidean distance between and , represents the Euclidean distance between and ; a boundary loss function is constructed according to all the negative samples:

[0046] ;

[0047] wherein, represents the boundary loss function.

[0048] wherein, the purpose of the boundary loss function is to explicitly and strongly constrain the prediction of the model in the boundary region, so that the network learns to generate clear boundaries. In each batch of training, first, boundary points are dynamically identified from the ground truth (i.e. real labels), and then an accurate algorithm based on k-NN (k-NN, K-Nearest Neighbors) search is used to realize this dynamic identification process. For the three-dimensional coordinates of all sample points in the current batch, a k-d tree is constructed to speed up the neighbor search. For each sample point in the batch, the k-d tree is used to efficiently query its k nearest neighbors in space to form a neighborhood set. The neighborhood set of point is traversed to check whether there is at least one sample point whose semantic label is different from that of The labels are different. If they exist, the currently traversed sample points will be... The point is identified as a "boundary point". The parameter k is a key hyperparameter, typically ranging from 8 to 32. Smaller k values ​​are more sensitive to local details but more susceptible to noise; larger k values ​​are more robust but may smooth out subtle boundaries. In practice, the k value can be set based on the average point density or voxel resolution of the data to ensure a reasonable neighborhood range.

[0049] In the embodiments disclosed in this invention, in the feature space, the feature representations of boundary points of the same type are brought closer together, while the feature representations of boundary points of different types are pushed further apart; specifically, the pairwise Euclidean distance matrix between the feature vectors of all identified boundary points in a batch is calculated, and all boundary points are traversed, with each boundary point... First, we define anchor points. For each anchor point... Among other boundary points of the same category, the point furthest from it is selected as the "hardest positive sample". Then at all boundary points Among the boundary points of different categories, select the boundary points that satisfy the semi-hard condition. As negative samples, this process ensures that the triples used for training are challenging enough to pose a positive loss to the model, but not so difficult that training becomes unstable, thus significantly improving learning efficiency and model performance.

[0050] Furthermore, based on the category prediction, the corresponding semantic prediction probability, and the true label of each sample point, a region loss function is constructed:

[0051] ;

[0052] in, Represents the region loss function. Indicates the number of categories. An index representing the number of categories, Indicate category The weight, Indicates the number of sample points. Indicates the prediction point Category semantic prediction probability, Point The true label; obtain the true instance center offset vector and the predicted instance center offset vector for each sample point, and construct an offset loss function based on each true instance center offset vector and the predicted instance center offset vector:

[0053] ;

[0054] in, denotes an offset loss function, denotes a set of instance sample points, denotes a first sample point, denotes a predicted instance center offset vector, denotes a real instance center offset vector, denotes a norm of ; a total loss function is constructed by using the boundary loss function, the region loss function and the offset loss function:

[0055] ;

[0056] wherein, denotes a total loss function, denotes a boundary loss function, denotes a hyperparameter balancing denotes a hyperparameter balancing denotes a hyperparameter balancing denotes a hyperparameter balancing denotes a hyperparameter balancing ; the sparse convolutional neural network, the spatial channel attention model and the segmentation head are optimized and trained by using the total loss function.

[0057] In order to ensure the accuracy of the segmentation result in the main body area, so that most of the sample points are correctly classified, the generalized dice loss function is disclosed as the region loss function, which is used to alleviate the serious imbalance of the number of sample points of different categories in the forest scene. In an embodiment disclosed in the present application, the weight of the category is set as the reciprocal of the number of pixels of the category, so as to give greater weight to small target categories, thereby improving the identification accuracy of small targets and avoiding confusion and misclassification in complex scenes.

[0058] The region loss function, the boundary loss function and the offset loss function disclosed in the present application provide complementary supervision information, ensure the classification correctness of the tree main body, make the model accurately locate the boundaries between different parts, effectively overcome the defect that a single region loss function is not sensitive to the boundary, and significantly improve the accuracy of the final segmentation result on the macro region and the micro boundary.

[0059] Step S40, input the target forest point cloud into the trained sparse convolutional neural network, spatial channel attention model and segmentation head, output the semantic category prediction result of each voxel in the target forest point cloud, and use a clustering algorithm to cluster all the semantic category prediction results to obtain a plurality of single tree segmentation results.

[0060] Specifically, the target forest point cloud is input into the optimized training sparse convolutional neural network, the spatial channel attention model and the segmentation head, and the class of each voxel in the target forest point cloud and the corresponding instance center offset vector are output; according to each class and the corresponding instance center offset vector, all voxels belonging to the same tree are clustered together by using a density-based clustering algorithm (HDBSCAN, Hierarchical Density-Based Spatial Clustering of Applications with Noise), to obtain each single tree segmentation result in the target forest point cloud.

[0061] In the training phase of the model, the network's prediction output is compared with the artificially labeled ground truth, the error is calculated through a novel boundary-aware composite loss function, and the network parameters are optimized through the back propagation algorithm. The loss function ensures that the model not only learns the overall consistency of the region, but also pays special attention to the accuracy of the boundary. Finally, instance clustering is performed on the target forest point cloud, and the output features are refined through the attention module, and the semantic class prediction (e.g., trunk, crown, ground) and instance center offset vector of each voxel are output by the segmentation head of the network. These outputs are then used in a density-based clustering algorithm to aggregate points belonging to the same tree, forming the final single tree instance segmentation result.

[0062] The present application optimizes the allocation of computing resources by adaptive voxelization under the premise of ensuring accuracy, improves the efficiency of processing large-scale forest point clouds, and finally generates a segmentation result with more accurate geometry and clearer contours, providing a higher quality data basis for subsequent forestry parameter extraction.

[0063] Further, as shown in Figure 4 Based on the above point cloud single tree segmentation method based on collaborative attention and density adaptive voxelization, the present application also correspondingly provides a point cloud single tree segmentation system based on collaborative attention and density adaptive voxelization, wherein the point cloud single tree segmentation system based on collaborative attention and density adaptive voxelization comprises:

[0064] The voxel grid construction module 51 is configured to obtain an original forest point cloud, calculate the density value of each sample point in the original forest point cloud, and perform mapping processing on all the density values to obtain a non-uniform voxel grid.

[0065] The cooperative attention module 52 is configured to input the voxel grid into the sparse convolutional neural network, output a voxel feature map in the original forest point cloud, input the voxel feature map into a spatial channel attention model for multiple times of pooling processing, output a refined feature map, and input the refined feature map into a segmentation head to output semantic prediction probabilities of each sample point.

[0066] The model training module 53 is configured to construct a boundary loss function by using feature representations of all boundary points in the refined feature map, construct a region loss function according to a real label and a semantic prediction probability of each sample point, construct an offset loss function according to a real instance center offset vector and a predicted instance center offset vector of each sample point, and optimize and train the sparse convolutional neural network, the spatial channel attention model and the segmentation head by using the boundary loss function, the region loss function and the offset loss function.

[0067] The result detection module 54 is configured to input a target forest point cloud into the trained sparse convolutional neural network, the spatial channel attention model and the segmentation head, output semantic class prediction results of each voxel in the target forest point cloud, and obtain multiple single tree segmentation results by using a clustering algorithm to cluster all the semantic class prediction results.

[0068] Further, as shown in Figure 5 Based on the above-mentioned point cloud single tree segmentation method and system based on cooperative attention and density adaptive voxelization, the application further provides a terminal, which comprises a processor 10, a memory 20 and a display 30. Figure 5 Only part of the components of the terminal are shown, but it should be understood that all the shown components are not required, and more or less components can be alternatively implemented.

[0069] The memory 20 can be an internal storage unit of the terminal in some embodiments, such as a hard disk or memory of the terminal. The memory 20 can also be an external storage device of the terminal in other embodiments, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the terminal. Further, the memory 20 can include both the internal storage unit and the external storage device of the terminal. The memory 20 is used to store application software and various data installed on the terminal, such as program codes of the installed terminal, etc. The memory 20 can also be used to temporarily store data that has been output or will be output. In an embodiment, the memory 20 stores a point cloud tree segmentation program based on collaborative attention and density adaptive voxelization 40, which can be executed by the processor 10 to implement the point cloud tree segmentation method based on collaborative attention and density adaptive voxelization in the present application.

[0070] The processor 10 can be a central processing unit (CPU), a microprocessor or other data processing chip in some embodiments, which is used to run program codes or process data stored in the memory 20, such as to execute the point cloud tree segmentation method based on collaborative attention and density adaptive voxelization, etc.

[0071] The display 30 can be an LED display, a liquid crystal display, a touch liquid crystal display, an OLED (Organic Light-Emitting Diode) touch, etc. in some embodiments. The display 30 is used to display information of the terminal and to display a visualized user interface. The components of the terminal communicate with each other through a system bus.

[0072] In an embodiment, the following steps are implemented when the processor 10 executes the point cloud tree segmentation program based on collaborative attention and density adaptive voxelization 40 in the memory 20:

[0073] Obtain an original forest point cloud, calculate a density value of each sample point in the original forest point cloud, and perform mapping processing on all the density values to obtain a non-uniform voxel grid;

[0074] Input the voxel grid into a sparse convolutional neural network, output a voxel feature map in the original forest point cloud, input the voxel feature map into a spatial channel attention model for multiple times of pooling processing, output a refined feature map, and input the refined feature map into a segmentation head to output a semantic prediction probability of each sample point.

[0075] A boundary loss function is constructed using the feature representations of all boundary points in the refined feature map. A region loss function is constructed based on the true label and semantic prediction probability of each sample point. An offset loss function is constructed based on the true instance center offset vector and the predicted instance center offset vector of each sample point. The sparse convolutional neural network, the spatial channel attention model, and the segmentation head are then optimized and trained using the boundary loss function, the region loss function, and the offset loss function.

[0076] The target forest point cloud is input into a trained sparse convolutional neural network, a spatial channel attention model, and a segmentation head. The semantic category prediction result of each voxel in the target forest point cloud is output, and a clustering algorithm is used to cluster all the semantic category prediction results to obtain multiple individual tree segmentation results.

[0077] The voxel grid includes: high-density voxel grid, medium-density voxel grid and low-density voxel grid;

[0078] The process of acquiring the original forest point cloud, calculating the density value of each sample point in the original forest point cloud, and mapping all the density values ​​to obtain a non-uniform voxel mesh specifically includes:

[0079] Obtain the original forest point cloud, extract multiple non-ground sample points from all sample points in the original forest point cloud, and pair all the non-ground sample points in pairs to calculate the pair distance of each pair of non-ground sample points;

[0080] The median of all the pairwise distances is defined as the global bandwidth parameter, and the density value of all the sample points is calculated based on the global bandwidth parameter:

[0081] ;

[0082] in, Represents any sample point, express density value, Indicates the number of sample points. Indicates the index of the sample point. This represents the global bandwidth parameter. Indicates according to Scaled Gaussian kernel, Indicates the first One sample point;

[0083] mapping all the density values to obtain the high-density voxel grid, the medium-density voxel grid and the low-density voxel grid.

[0084] The spatial channel attention model comprises a channel attention model and a spatial attention model.

[0085] The voxel grid is input into a sparse convolutional neural network to output a voxel feature map in the original forest point cloud, the voxel feature map is input into a spatial channel attention model for multiple pooling processing to output a refined feature map, and the refined feature map is input into a segmentation head to output semantic prediction probabilities of each sample point.

[0086] The voxel grid is input into a sparse convolutional neural network to output a voxel feature map in the original forest point cloud;

[0087] The voxel feature map is input into the channel attention model to output an intermediate feature map, and the intermediate feature map is input into the spatial attention model to output a refined feature map.

[0088] The refined feature map is input into a segmentation head, and the segmentation head outputs class prediction, corresponding semantic prediction probability and instance center offset vector of each sample point.

[0089] The voxel feature map is input into the channel attention model to output an intermediate feature map, and the intermediate feature map is input into the spatial attention model to output a refined feature map.

[0090] The voxel feature map is input into the channel attention model, the channel attention model performs global pooling on the voxel feature map to output an average channel descriptor and a maximum channel descriptor;

[0091] The average channel descriptor and the maximum channel descriptor are input into a network of a double-layer fully connected layer to output a channel attention weight map.

[0092] The channel attention weight map and the voxel feature map are multiplied element by element in the channel to realize channel attention weighting, and an intermediate feature map is obtained.

[0093] The intermediate feature map is input into the spatial attention model, and the spatial attention model performs average pooling and maximum pooling on the intermediate feature map to obtain an average spatial descriptor and a maximum spatial descriptor, respectively.

[0094] The average spatial descriptor and the maximum spatial descriptor are spliced in the channel dimension of the spatial attention model to obtain a spliced feature.

[0095] The splicing feature is processed through a three-dimensional sparse convolution layer to output a single-channel spatial attention weight map;

[0096] The spatial attention weight map is multiplied with the intermediate feature map through element-wise broadcasting to obtain a refined feature map.

[0097] The boundary loss function is constructed using the feature representation of all boundary points in the refined feature map, specifically including:

[0098] The three-dimensional coordinates of all sample points are obtained, and a k-d tree is constructed. For each sample point, the k-d tree is used to determine the k nearest neighboring points to the sample point to construct a neighborhood set;

[0099] The neighborhood set of the current sample point is traversed. If the semantic label of at least one neighboring point is different from the semantic label of the sample point, the current sample point is defined as a boundary point until all boundary points in all sample points are extracted;

[0100] From the feature representation of all boundary points in the refined feature map, each boundary point is traversed, and all boundary points that satisfy the semi-hard condition are defined as negative samples:

[0101] ;

[0102] wherein, represents the currently traversed boundary point, represents the boundary point farthest from , represents the negative sample, represents the feature representation of , represents the feature representation of , represents the feature representation of , represents a preset edge distance, represents the Euclidean distance between and , represents the Euclidean distance between and ;

[0103] The boundary loss function is constructed according to all the negative samples:

[0104] ;

[0105] wherein, represents the boundary loss function.

[0106] Specifically, the process of constructing a region loss function based on the true label and semantic prediction probability of each sample point, constructing an offset loss function based on the true instance center offset vector and the predicted instance center offset vector of each sample point, and optimizing the sparse convolutional neural network, the spatial channel attention model, and the segmentation head using the boundary loss function, the region loss function, and the offset loss function, includes:

[0107] Based on the category prediction, corresponding semantic prediction probability, and true label of each sample point, a region loss function is constructed:

[0108] ;

[0109] in, Represents the region loss function. Indicates the number of categories. An index representing the number of categories, Indicate category The weight, Indicates the number of sample points. Indicates the prediction point Category semantic prediction probability, Point The true label;

[0110] Obtain the true instance center offset vector and the predicted instance center offset vector for each sample point, and construct an offset loss function based on each true instance center offset vector and the predicted instance center offset vector:

[0111] ;

[0112] in, Represents the offset loss function. Represents the set of instance sample points. Indicates the first One sample point, Represents the predicted instance center offset vector. Represents the offset vector of the center of the real instance. express The norm;

[0113] The total loss function is constructed using the boundary loss function, the region loss function, and the offset loss function:

[0114] ;

[0115] in, Represents the total loss function. Represents the boundary loss function. Indicates balance hyperparameters, Indicates balance hyperparameters, Indicates balance hyperparameters;

[0116] The sparse convolutional neural network, the spatial channel attention model, and the segmentation head are optimized and trained using the total loss function.

[0117] Specifically, the process involves inputting the target forest point cloud into a trained sparse convolutional neural network, a spatial channel attention model, and a segmentation head, outputting the semantic category prediction result for each voxel in the target forest point cloud, and using a clustering algorithm to cluster all the semantic category prediction results to obtain multiple individual tree segmentation results, including:

[0118] The target forest point cloud is input into an optimized and trained sparse convolutional neural network, a spatial channel attention model, and a segmentation head, and the output is the category and corresponding instance center offset vector of each voxel in the target forest point cloud.

[0119] Based on each category and the corresponding instance center offset vector, a density-based clustering algorithm is used to cluster all voxels belonging to the same tree together to obtain the segmentation result of each tree in the target forest point cloud.

[0120] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a point cloud single-tree segmentation program based on collaborative attention and density adaptive voxelization, and the point cloud single-tree segmentation program based on collaborative attention and density adaptive voxelization, when executed by a processor, implements the steps of the point cloud single-tree segmentation method based on collaborative attention and density adaptive voxelization as described above.

[0121] In summary, the present application provides a point cloud tree segmentation method based on cooperative attention and density adaptive voxelization and related equipment, the method comprising: obtaining an original forest point cloud, calculating the density value of each sample point in the original forest point cloud, performing mapping processing on all the density values to obtain a non-uniform voxel grid; inputting the voxel grid into a sparse convolutional neural network to output a voxel feature map in the original forest point cloud, inputting the voxel feature map into a spatial channel attention model for multiple pooling processing to output a refined feature map, inputting the refined feature map into a segmentation head to output the semantic prediction probability of each sample point; constructing a boundary loss function using the feature representation of all boundary points in the refined feature map, constructing a region loss function according to the true label and semantic prediction probability of each sample point, constructing an offset loss function according to the true instance center offset vector and predicted instance center offset vector of each sample point, and optimizing and training the sparse convolutional neural network, the spatial channel attention model and the segmentation head using the boundary loss function, the region loss function and the offset loss function; constructing an offset loss function according to the true instance center and the predicted instance center offset vector of each sample point, and optimizing and training the sparse convolutional neural network, the spatial channel attention model and the segmentation head using the boundary loss function, the region loss function and the offset loss function; inputting a target forest point cloud into the trained sparse convolutional neural network, spatial channel attention model and segmentation head to output the semantic class prediction result of each voxel in the target forest point cloud, and clustering all the semantic class prediction results using a clustering algorithm to obtain multiple tree segmentation results. The present application optimizes the allocation of computing resources through adaptive voxelization under the premise of ensuring accuracy, improves the efficiency of processing large-scale forest point clouds, and finally generates segmentation results with more accurate geometric shapes and clearer contours, providing a higher-quality data foundation for subsequent forestry parameter extraction.

[0122] It should be noted that in this document, the terms "comprising", "including", or any other variant thereof are intended to cover non-exclusive inclusions, so that processes, methods, articles or terminals including a series of elements not only include those elements, but also include other elements not explicitly listed, or include elements inherent to such processes, methods, articles or terminals. Without more limitations, the element defined by the statement "including a" does not exclude the presence of another identical element in the process, method, article or terminal including the element.

[0123] Of course, those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing relevant hardware (such as a processor, a controller, etc.) through a computer program. The program can be stored in a computer-readable computer-readable storage medium, and the program can include the processes of the above-mentioned method embodiments when executed. The computer-readable storage medium can be a memory, a disk, an optical disk, etc.

[0124] It should be understood that the application is not limited to the above examples, and those skilled in the art can make improvements or changes according to the above description, and all these improvements and changes shall belong to the protection scope of the appended claims of the application.

Claims

1. A point cloud tree segmentation method based on cooperative attention and density adaptive voxelization, characterized in that, The point cloud tree segmentation method based on collaborative attention and density adaptive voxelization comprises: An original forest point cloud is acquired, density values of each sample point in the original forest point cloud are calculated, and all the density values are mapped to obtain a non-uniform voxel grid; The voxel grid comprises a high-density voxel grid, a medium-density voxel grid, and a low-density voxel grid; The acquisition of the original forest point cloud, the calculation of the density values of each sample point in the original forest point cloud, and the mapping of all the density values to obtain a non-uniform voxel grid specifically comprises: An original forest point cloud is acquired, a plurality of non-ground sample points are extracted from all sample points in the original forest point cloud, and all the non-ground sample points are pairwise matched to calculate the pairwise distance of each pair of non-ground sample points; The median of all the pairwise distances is defined as a global bandwidth parameter, and the density values of all the sample points are calculated according to the global bandwidth parameter: ； wherein, denotes an arbitrary sample point, denotes a density value of , denotes the number of sample points, denotes the index of a sample point, denotes a global bandwidth parameter, denotes a scaled Gaussian kernel according to , denotes a scaled Gaussian kernel according to , denotes the th sample point; All the density values are mapped to obtain the high-density voxel grid, the medium-density voxel grid, and the low-density voxel grid; The voxel grid is input into a sparse convolutional neural network to output a voxel feature map of the original forest point cloud, the voxel feature map is input into a spatial channel attention model for multiple times of pooling processing to output a refined feature map, and the refined feature map is input into a segmentation head to output semantic prediction probabilities of each sample point; A boundary loss function is constructed by using the feature representation of all boundary points in the refined feature map, a region loss function is constructed according to the true label and the semantic prediction probability of each sample point, an offset loss function is constructed according to the true instance center offset vector and the predicted instance center offset vector of each sample point, and the sparse convolutional neural network, the spatial channel attention model, and the segmentation head are optimized and trained by using the boundary loss function, the region loss function, and the offset loss function. A target forest point cloud is input into the trained sparse convolutional neural network, spatial channel attention model, and segmentation head to output semantic class prediction results of each voxel in the target forest point cloud, and all the semantic class prediction results are clustered by using a clustering algorithm to obtain a plurality of tree segmentation results.

2. The point cloud tree segmentation method based on synergic attention and density adaptive voxelization according to claim 1, wherein, The spatial channel attention model comprises a channel attention model and a spatial attention model; The input of the voxel grid into the sparse convolutional neural network to output the voxel feature map of the original forest point cloud, the input of the voxel feature map into the spatial channel attention model for multiple times of pooling processing to output the refined feature map, and the input of the refined feature map into the segmentation head to output the semantic prediction probabilities of each sample point specifically comprise: The voxel grid is input into the sparse convolutional neural network for deep feature extraction to output the voxel feature map of the original forest point cloud; The voxel feature map is input into the channel attention model to output an intermediate feature map, and the intermediate feature map is input into the spatial attention model to output the refined feature map; The refined feature map is input into a segmentation head, and the segmentation head outputs a class prediction and a corresponding semantic prediction probability and an instance center offset vector of each sample point.

3. The point cloud tree segmentation method based on synergic attention and density adaptive voxelization according to claim 2, characterized in that, The voxel feature map is input into the channel attention model, the channel attention model performs global pooling on the voxel feature map, and outputs an average channel descriptor and a maximum channel descriptor. The average channel descriptor and the maximum channel descriptor are input into a network of a double-layer fully connected layer, and a channel attention weight map is output. The channel attention weight map and the voxel feature map are multiplied element by element in the channel to realize channel attention weighting, and an intermediate feature map is obtained. The average spatial descriptor and the maximum spatial descriptor are concatenated in the channel dimension of the spatial attention model, and a concatenated feature is obtained. The concatenated feature is processed through a three-dimensional sparse convolution layer, and a single-channel spatial attention weight map is output. The spatial attention weight map and the intermediate feature map are multiplied element by element, and a refined feature map is obtained. The feature representation of all boundary points in the refined feature map is used to construct a boundary loss function, which specifically includes: The three-dimensional coordinates of all sample points are obtained, and a k-d tree is constructed. For each sample point, the k-d tree is used to determine the nearest k neighboring points of the sample point to construct a neighborhood set.

4. The method of claim 1, wherein the method is based on a synergic attention and density adaptive voxelization for point cloud tree segmentation. The neighborhood set of the current sample point is traversed. If the semantic label of at least one neighboring point is different from the semantic label of the sample point, the current sample point is defined as a boundary point, until all boundary points in all sample points are extracted. From the feature representation of all boundary points in the refined feature map, each boundary point is traversed, and all boundary points that meet the semi-hard condition are defined as negative samples: A boundary loss function is constructed according to all the negative samples: According to the real label and the semantic prediction probability of each sample point, a region loss function is constructed, and an offset loss function is constructed according to the real instance center offset vector and the predicted instance center offset vector of each sample point. The boundary loss function, the region loss function and the offset loss function are used to optimize and train the sparse convolutional neural network, the spatial channel attention model and the segmentation head, which specifically includes: ； wherein, represents a currently traversed boundary point, represents a distance to the farthest boundary point, represents a negative sample, represents a feature representation of, represents a feature representation of, represents a feature representation of, represents a preset margin, represents a Euclidean distance between, and a Euclidean distance between, and ; According to the class prediction, the corresponding semantic prediction probability of each sample point and the real label of each sample point, a region loss function is constructed: ； wherein, denotes the boundary loss function.

5. The method of claim 2, wherein the method is based on a synergic attention and density adaptive voxelization for point cloud tree segmentation. The real instance center offset vector and the predicted instance center offset vector of each sample point are obtained, and an offset loss function is constructed according to each real instance center offset vector and the predicted instance center offset vector: ​ ； wherein, represents a region loss function, represents a number of classes, represents an index of a number of classes, represents a class a weight of, represents a number of sample points, represents a predicted point belongs to a class a semantic prediction probability of, represents a true label of a point . ​ ； in, This represents the offset loss function. Represents the set of instance sample points. Indicates the first One sample point, Represents the predicted instance center offset vector. Represents the offset vector of the center of the real instance. express The norm; constructing a total loss function by using the boundary loss function, the region loss function and the offset loss function: ； wherein, represents the total loss function, represents the boundary loss function, represents the hyperparameter balancing represents the hyperparameter balancing represents the hyperparameter balancing represents the hyperparameter balancing represents the hyperparameter balancing represents the hyperparameter balancing optimizing and training the sparse convolutional neural network, the spatial channel attention model and the segmentation head by using the total loss function.

6. The method of claim 1, wherein, The target forest point cloud is input into the trained sparse convolutional neural network, spatial channel attention model and segmentation head, the semantic class prediction result of each voxel in the target forest point cloud is output, and a clustering algorithm is used to cluster all the semantic class prediction results to obtain multiple single tree segmentation results, specifically including: The target forest point cloud is input into the trained sparse convolutional neural network, spatial channel attention model and segmentation head, the semantic class prediction result of each voxel in the target forest point cloud is output, and a clustering algorithm is used to cluster all the semantic class prediction results to obtain multiple single tree segmentation results, specifically including: According to each of the class and the corresponding instance center offset vector, all voxels belonging to the same tree are clustered together by using a density-based clustering algorithm to obtain each single tree segmentation result in the target forest point cloud.

7. A point cloud tree segmentation system based on co-attention and density adaptive voxelization, characterized in that, The point cloud single tree segmentation system based on collaborative attention and density adaptive voxelization is used to realize the point cloud single tree segmentation method based on collaborative attention and density adaptive voxelization as claimed in any one of claims 1-6, comprising: A voxel grid construction module is configured to obtain an original forest point cloud, calculate a density value of each sample point in the original forest point cloud, and map all the density values to obtain a non-uniform voxel grid. A collaborative attention module is configured to input the voxel grid into a sparse convolutional neural network to output a voxel feature map in the original forest point cloud, input the voxel feature map into a spatial channel attention model for multiple pooling processing to output a refined feature map, and input the refined feature map into a segmentation head to output a semantic prediction probability of each sample point. A model training module is configured to construct a boundary loss function by using feature representations of all boundary points in the refined feature map, construct a region loss function according to a real label and a semantic prediction probability of each sample point, construct an offset loss function according to a real instance center offset vector and a predicted instance center offset vector of each sample point, and optimize and train the sparse convolutional neural network, the spatial channel attention model and the segmentation head by using the boundary loss function, the region loss function and the offset loss function. A result detection module is configured to input a target forest point cloud into a trained sparse convolutional neural network, spatial channel attention model and segmentation head, output a semantic class prediction result of each voxel in the target forest point cloud, and cluster all the semantic class prediction results by using a clustering algorithm to obtain multiple single tree segmentation results.

8. A terminal, characterized by comprising: The terminal comprises a memory, a processor, and a point cloud tree segmentation program based on collaborative attention and density adaptive voxelization stored on the memory and executable on the processor, which, when executed by the processor, implements the steps of the point cloud tree segmentation method based on collaborative attention and density adaptive voxelization according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a point cloud tree segmentation program based on collaborative attention and density adaptive voxelization, which, when executed by the processor, implements the steps of the point cloud tree segmentation method based on collaborative attention and density adaptive voxelization according to any one of claims 1-6.

Images