Point cloud individual tree segmentation method and system based on collaborative attention and density adaptive voxelization, terminal and storage medium

Abstract

The invention relates to the technical field of point cloud data processing, and discloses a point cloud individual tree segmentation method and system based on collaborative attention and density adaptive voxelization, a terminal and a storage medium, and the method comprises the steps: carrying out the density adaptive voxelization of an original forest point cloud to construct a non-uniform voxel grid, and then inputting a sparse convolutional neural network, performing feature extraction by using a collaborative space channel attention module, performing model training through a boundary perception composite loss function, and finally extracting a segmentation result of each individual tree in the forest point cloud. According to the method, on the premise that the precision is guaranteed, distribution of computing resources is optimized through self-adaptive voxelization, the efficiency of processing large-scale forest point cloud is improved, the finally generated segmentation result has a more accurate geometric shape and a clearer contour, and a higher-quality data basis is provided for subsequent forestry parameter extraction.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of point cloud data processing, and particularly relates to a point cloud individual tree segmentation method and system based on collaborative attention and density adaptive voxelization, a terminal and a computer readable storage medium. BACKGROUND

[0002] With the rapid development of three-dimensional sensing technology, in the fields of intelligent forestry, ecological monitoring and carbon storage estimation, automatically and accurately segmenting individual trees (ITS) from large-scale point clouds is a core pre-task for realizing fine management and evaluation of forestry resources.

[0003] However, the current mainstream method generally uses fixed-size voxels to discretize point clouds, which is fundamentally contradictory to the inherent non-uniform density characteristics of tree point clouds (i.e., dense trunks and sparse crowns); standard three-dimensional convolutional networks homogenize all spatial positions and feature channels when processing feature maps, failing to distinguish their contribution to the segmentation task; and mainstream segmentation loss functions, such as cross-entropy or Dice loss, are essentially based on the measurement of region overlap. Such loss functions tend to optimize the main body region with a large proportion, and are insensitive to errors on the boundary points which only account for a small proportion.

[0004] Therefore, the prior art still needs to be improved and developed. SUMMARY

[0005] The main purpose of the present application is to provide a point cloud individual tree segmentation method and system based on collaborative attention and density adaptive voxelization, a terminal and a computer readable storage medium, which aims to solve the problem of insufficient boundary segmentation accuracy caused by fixed voxelization and lack of advanced feature refinement mechanism when processing tree point clouds with non-uniform density in the prior art.

[0006] To achieve the above purpose, the present application provides a point cloud individual tree segmentation method based on collaborative attention and density adaptive voxelization, which comprises the following steps: Obtain the original forest point cloud, calculate the density value of each sample point in the original forest point cloud, and perform mapping processing on all the density values to obtain a non-uniform voxel grid; Input the voxel grid into a sparse convolutional neural network to output a voxel feature map in the original forest point cloud, input the voxel feature map into a spatial channel attention model for multiple pooling processing to output a refined feature map, and input the refined feature map into a segmentation head to output the semantic prediction probability of each sample point; The boundary loss function is constructed by using the feature representation of all boundary points in the refined feature map, the region loss function is constructed according to the true label and the semantic prediction probability of each sample point, the offset loss function is constructed according to the true instance center offset vector and the predicted instance center offset vector of each sample point, and the sparse convolutional neural network, the spatial channel attention model and the segmentation head are optimized and trained by using the boundary loss function, the region loss function and the offset loss function; the offset loss function is constructed according to the true instance center and the predicted instance center offset vector of each sample point, and the sparse convolutional neural network, the spatial channel attention model and the segmentation head are optimized and trained by using the boundary loss function, the region loss function and the offset loss function. The target forest point cloud is input into the trained sparse convolutional neural network, spatial channel attention model and segmentation head, the semantic class prediction result of each voxel in the target forest point cloud is output, and all the semantic class prediction results are clustered by using a clustering algorithm to obtain a plurality of single tree segmentation results.

[0007] In addition, to achieve the above-mentioned purpose, the application further provides a point cloud single tree segmentation system based on collaborative attention and density adaptive voxelization, wherein the point cloud single tree segmentation system based on collaborative attention and density adaptive voxelization comprises: A voxel grid construction module is configured to acquire an original forest point cloud, calculate the density value of each sample point in the original forest point cloud, and perform mapping processing on all the density values to obtain a non-uniform voxel grid. A collaborative attention module is configured to input the voxel grid into a sparse convolutional neural network to output a voxel feature map in the original forest point cloud, input the voxel feature map into a spatial channel attention model for multiple times of pooling processing to output a refined feature map, and input the refined feature map into a segmentation head to output the semantic prediction probability of each sample point. A model training module is configured to construct a boundary loss function by using the feature representation of all boundary points in the refined feature map, construct a region loss function according to the true label and the semantic prediction probability of each sample point, construct an offset loss function according to the true instance center offset vector and the predicted instance center offset vector of each sample point, and optimize and train the sparse convolutional neural network, the spatial channel attention model and the segmentation head by using the boundary loss function, the region loss function and the offset loss function; construct an offset loss function according to the true instance center and the predicted instance center offset vector of each sample point, and optimize and train the sparse convolutional neural network, the spatial channel attention model and the segmentation head by using the boundary loss function, the region loss function and the offset loss function. A result detection module is configured to input the target forest point cloud into a trained sparse convolutional neural network, a spatial channel attention model and a segmentation head, output semantic class prediction results of each voxel in the target forest point cloud, and use a clustering algorithm to cluster all the semantic class prediction results to obtain a plurality of single tree segmentation results.

[0008] In addition, to achieve the above object, the present application also provides a terminal, wherein the terminal comprises a memory, a processor and a point cloud single tree segmentation program based on collaborative attention and density adaptive voxelization stored in the memory and executable on the processor, and the point cloud single tree segmentation program based on collaborative attention and density adaptive voxelization implements the steps of the point cloud single tree segmentation method based on collaborative attention and density adaptive voxelization when executed by the processor.

[0009] In addition, to achieve the above object, the present application also provides a computer readable storage medium, wherein the computer readable storage medium stores a point cloud single tree segmentation program based on collaborative attention and density adaptive voxelization, and the point cloud single tree segmentation program based on collaborative attention and density adaptive voxelization implements the steps of the point cloud single tree segmentation method based on collaborative attention and density adaptive voxelization when executed by a processor.

[0010] In the present application, the original forest point cloud is obtained, the density value of each sample point in the original forest point cloud is calculated, and the non-uniform voxel grid is obtained by mapping all the density values; the voxel grid is input into a sparse convolutional neural network, and the voxel feature map in the original forest point cloud is output; the voxel feature map is input into a spatial channel attention model for multiple pooling processing, and the refined feature map is output; the refined feature map is input into a segmentation head, and the semantic prediction probability of each sample point is output; the boundary loss function is constructed by using the feature representation of all boundary points in the refined feature map, the region loss function is constructed according to the real label and the semantic prediction probability of each sample point, and the offset loss function is constructed according to the real instance center offset vector and the predicted instance center offset vector of each sample point; and the sparse convolutional neural network, the spatial channel attention model and the segmentation head are optimized and trained by using the boundary loss function, the region loss function and the offset loss function; according to the real instance center and the predicted instance center offset vector of each sample point, the offset loss function is constructed, and the sparse convolutional neural network, the spatial channel attention model and the segmentation head are optimized and trained by using the boundary loss function, the region loss function and the offset loss function; the target forest point cloud is input into the trained sparse convolutional neural network, spatial channel attention model and segmentation head, and the semantic class prediction result of each voxel in the target forest point cloud is output; and all the semantic class prediction results are clustered by using a clustering algorithm to obtain multiple single tree segmentation results. The present application optimizes the allocation of computing resources by adaptive voxelization under the premise of ensuring accuracy, improves the efficiency of processing large-scale forest point clouds, and finally generates segmentation results with more accurate geometric shapes and clearer contours, providing a higher-quality data basis for subsequent forestry parameter extraction. BRIEF DESCRIPTION OF DRAWINGS

[0011] Figure 1 is a flowchart of a preferred embodiment of the point cloud single tree segmentation method based on collaborative attention and density adaptive voxelization of the present application; Figure 2 is a general architecture block diagram of a preferred embodiment of the point cloud single tree segmentation method based on collaborative attention and density adaptive voxelization of the present application; Figure 3 is a flowchart of a spatial channel attention model of a preferred embodiment of the point cloud single tree segmentation method based on collaborative attention and density adaptive voxelization of the present application; Figure 4 is a structure diagram of a preferred embodiment of the point cloud single tree segmentation system based on collaborative attention and density adaptive voxelization of the present application; Figure 5 is a structure diagram of a preferred embodiment of the terminal of the present application. DETAILED DESCRIPTION

[0012] To make the objectives, technical solutions, and advantages of this invention clearer and more explicit, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0013] The preferred embodiment of the point cloud tree segmentation method based on collaborative attention and density adaptive voxelization described in this invention, such as... Figure 1 As shown, the point cloud tree segmentation method based on collaborative attention and density adaptive voxelization includes the following steps: Step S10: Obtain the original forest point cloud, calculate the density value of each sample point in the original forest point cloud, and perform mapping processing on all the density values ​​to obtain a non-uniform voxel grid.

[0014] Current tree segmentation methods generally use fixed-size voxels to discretize point clouds. This strategy is fundamentally contradictory to the inherent non-uniform density of tree point clouds (i.e., dense trunks and sparse crowns). Larger voxel sizes blur the fine geometric structure of the crown tips, leading to the loss of key information; while smaller voxel sizes generate massive amounts of redundant data in dense areas such as trunks, resulting in huge computational and storage overhead.

[0015] Therefore, as Figure 2 As shown, for the acquired original forest point cloud, this application first constructs a non-uniform voxel grid to achieve density adaptive voxelization. This part receives the original forest scene point cloud as input, replacing the fixed-size downsampling step in the traditional method, and dynamically generates a multi-resolution voxel grid representation based on the local density of the point cloud.

[0016] Specifically, a primary forest point cloud is acquired, and multiple non-ground sample points are extracted from all sample points in the primary forest point cloud. All non-ground sample points are then paired up to calculate the pairwise distance for each pair (where pairwise distance refers to calculating the Euclidean distance between each pair of sampled points). The median of all pairwise distances is defined as the global bandwidth parameter, and the density value of all sample points is calculated based on the global bandwidth parameter. ; in, Represents any sample point, express density value, Indicates the number of sample points. Indicates the index of the sample point. This represents the global bandwidth parameter. Indicates according to scaled Gaussian kernel, representing the first sample point; mapping all the density values to obtain the high-density voxel grid, the medium-density voxel grid and the low-density voxel grid.

[0017] wherein a corresponding density value is calculated for each point in the original forest point cloud (kernel density estimation is a non-parametric density estimation method, which can provide a smooth density field); the bandwidth parameter can control the smoothness of the density estimation, and in the process of adaptive adjustment of the point cloud, the global bandwidth parameter is used for control, which is completely data-driven, and the use of the median can effectively resist the interference of outliers, and its physical meaning is that the influence range of the kernel function matches the typical point spacing of the point cloud, thereby avoiding the error caused by manual parameter adjustment, and ensuring the robustness of the density estimation, which can effectively capture the subtle changes in the point cloud density.

[0018] wherein the non-uniform voxel grid includes a high-density area (trunk), a medium-density area (main branch) and a low-density area (outer periphery of the crown), and these areas refer to the same point cloud (i.e. a tree or a forest scene, which refers to the original forest point cloud in this embodiment), which is divided according to the density value.

[0019] Step S20, inputting the voxel grid into a sparse convolutional neural network to output a voxel feature map in the original forest point cloud, inputting the voxel feature map into a spatial channel attention model for multiple pooling processing to output a refined feature map, and inputting the refined feature map into a segmentation head to output semantic prediction probabilities of each sample point.

[0020] Specifically, the voxel grid is input into a sparse convolutional neural network (i.e. sparse 3D convolutional neural network (Convolutional Networks for Biomedical Image Segmentation‌)) for deep feature extraction to output a voxel feature map of the original forest point cloud; the voxel feature map is input into the channel attention model to output an intermediate feature map, the intermediate feature map is input into the spatial attention model to output a refined feature map; and the refined feature map is input into the segmentation head, and the segmentation head outputs the class prediction, the corresponding semantic prediction probability and the instance center offset vector of each sample point.

[0021] The spatial channel attention model (SSCA, Spatial and Channel Synergistic Attention) includes a channel attention model and a spatial attention model. A standard three-dimensional convolutional network homogenizes all spatial positions and feature channels when processing a feature map, and fails to distinguish their contribution to the segmentation task. For trees, which have extremely complex morphology, their branch points, branch connections, and crown boundaries intersected with other trees obviously contain more key information for distinguishing different instances than smooth tree trunks. Therefore, in the embodiments disclosed in the present application, multi-scale feature extraction is performed on the voxel grid layer by layer, and a synergistic spatial channel attention module (as shown in Figure 3 ) is embedded in the skip connection path between the corresponding levels of the encoder and the decoder to refine the feature map extracted by the backbone network, enhance the significant features related to the segmentation task, and suppress irrelevant information.

[0022] Specifically, the voxel feature map is input into the channel attention model, which performs global pooling on the voxel feature map and outputs average channel descriptors and maximum channel descriptors; the average channel descriptors and the maximum channel descriptors are input into a network of double-layer fully connected layers, and a channel attention weight map is output; the channel attention weight map and the voxel feature map are multiplied element by element in the channel to realize channel attention weighting, and an intermediate feature map is obtained; the intermediate feature map is input into the spatial attention model, which performs average pooling and maximum pooling on the intermediate feature map to obtain average spatial descriptors and maximum spatial descriptors, respectively; the average spatial descriptors and the maximum spatial descriptors are spliced in the channel dimension of the spatial attention model to obtain a spliced feature; the spliced feature is processed by a three-dimensional sparse convolution layer to output a single-channel spatial attention weight map; and the spatial attention weight map and the intermediate feature map are multiplied element by element by broadcast to obtain a refined feature map.

[0023] In the embodiments disclosed in the present application, a synergistic spatial channel attention model is used to process three-dimensional sparse voxel data (i.e., voxel feature map F), which is sequentially passed through the channel attention model and the spatial attention model, and finally obtains a refined feature map .

[0024] Since the backbone network deals with sparse tensors, to maintain computational efficiency, the global pooling operation needs to be performed without "densifying" the tensor. Therefore, the present application implements global pooling by directly operating on the voxel feature map F (dimension: non-zero voxel number x feature channel number) of the sparse tensor, specifically including sparse global average pooling (Sparse Global Average Pooling) and sparse global maximum pooling (Sparse Global Max Pooling); wherein the sparse global average pooling obtains the mean value of the feature matrix in the dimension representing all non-zero voxels, and for each channel, the mean value of the channel in all non-zero voxels is taken; the sparse global maximum pooling calculates the maximum value in the dimension. The computational complexity of this method is only proportional to the number of non-zero voxels, perfectly maintaining the advantages of sparse computing, and finally generating two average channel descriptors and maximum channel descriptors with dimensions C x 1 x 1 x 1 (C is the number of feature channels), which capture the global distribution information and the most significant feature response of each channel, respectively. and are sent into a shared network consisting of two fully connected layers, which first compress the channel dimension to reduce parameters, then restore, and finally generate a channel attention weight map through a Sigmoid activation function. The generated attention map is multiplied with the original input feature map element-wise (element-wise multiplication) to obtain an intermediate feature map weighted by channel attention, so that the network can enhance those feature channels that are essential to distinguish different parts of the tree (such as wood and leaves), while suppressing those channels containing noise or redundant information.

[0025] Further, after the channel attention model, a spatial attention model is followed, which emphasizes information-rich spatial regions by learning a three-dimensional spatial weight map. The intermediate feature map is averaged and maximized in the channel dimension of the channel attention model to generate two spatial descriptors with dimensions 1 x H x W x D (H represents height, W represents width, and D represents dimension): average spatial descriptor and maximum spatial descriptor , both of which capture the average and most salient feature information at each spatial location. The two spatial descriptors are concatenated in the channel dimension and then processed by a standard 3D sparse convolutional layer to generate a single-channel spatial attention weight map. Finally, the generated spatial attention map is element-wise broadcasted with the intermediate feature map to obtain the final refined output feature map (refined feature map). Through the above method, in the single tree segmentation task disclosed in the present application, the channel attention can learn to amplify those feature channels that encode "wood" or "leaves", while the spatial attention can learn to concentrate computing resources on the structural key points of the tree, such as the junction of the tree root and the ground, the branching points of the trunk, and the crown boundary with other trees. The synergistic effect of this dual attention mechanism enables the network to extract more robust and discriminative feature representations from noisy raw data.

[0026] Step S30, constructing a boundary loss function using the feature representation of all boundary points in the refined feature map, constructing a region loss function according to the true label and the semantic prediction probability of each sample point, constructing an offset loss function according to the true instance center offset vector and the predicted instance center offset vector of each sample point, and optimizing and training the sparse convolutional neural network, the spatial channel attention model and the segmentation head using the boundary loss function, the region loss function and the offset loss function. According to the true instance center and the predicted instance center offset vector of each sample point, an offset loss function is constructed, and the sparse convolutional neural network, the spatial channel attention model and the segmentation head are optimized and trained using the boundary loss function, the region loss function and the offset loss function.

[0027] Where, for existing segmentation loss functions such as cross-entropy or Dice loss, their essence is to measure the region overlap. Such loss functions tend to optimize the main region with a large proportion, and are insensitive to errors on the boundary points which only account for a small proportion. In forest scenes, the number of boundary points defining the tree outline is much smaller than that of internal points, resulting in a lack of strong and explicit supervision signals for boundary accuracy during model training. Therefore, although the segmented main region may be correct in a macroscopic sense, its boundary often presents as fuzzy, smooth or jagged, which seriously affects the accuracy of subsequent estimation of key forestry parameters such as diameter at breast height and crown width. In order to fundamentally solve the problem of low boundary segmentation accuracy of existing methods, the present application proposes a composite loss function. This loss function provides a more comprehensive and balanced supervision signal for network optimization by combining two loss terms with different properties.

[0028] Specifically, three-dimensional coordinates of all the sample points are acquired, and a k-d tree is constructed, for each sample point, the k-d tree is used to determine the k nearest neighboring points of the sample point to construct a field set; the field set of the current sample point is traversed, if there is at least one neighboring point whose semantic label is different from the semantic label of the sample point, the current sample point is defined as a boundary point, until all boundary points in all sample points are extracted; from the feature representation of all boundary points in the refined feature map, each boundary point is traversed, all boundary points satisfying the semi-hard condition are defined as negative samples: ; wherein, represents the current traversed boundary point, represents the distance from the farthest boundary point, represents the negative sample, represents the feature representation of , the feature representation of , the feature representation of represents a preset edge distance, represents the Euclidean distance between and , represents the Euclidean distance between and ; a boundary loss function is constructed according to all the negative samples: ; wherein, represents the boundary loss function.

[0029] wherein, the purpose of the boundary loss function is to explicitly and strongly constrain the prediction of the model in the boundary region, so that the network learns to generate clear boundaries. In each batch of training, first, the boundary points need to be dynamically identified from the ground truth (i.e. the real label), and then an accurate algorithm based on k-NN (k-NN, K-Nearest Neighbors) search is used to realize this dynamic identification process. For the three-dimensional coordinates of all sample points in the current batch, a k-d tree is constructed to speed up the neighbor search. For each sample point in the batch, the k-d tree is used to efficiently query its k nearest neighbors in space to form a neighborhood set. The neighborhood set of the point is traversed to check whether there is at least one sample point whose semantic label is different from the label of . If there is, the currently traversed sample point The point is identified as a "boundary point". The parameter k is a key hyperparameter, typically ranging from 8 to 32. Smaller k values ​​are more sensitive to local details but more susceptible to noise; larger k values ​​are more robust but may smooth out subtle boundaries. In practice, the k value can be set based on the average point density or voxel resolution of the data to ensure a reasonable neighborhood range.

[0030] In the embodiments disclosed in this invention, in the feature space, the feature representations of boundary points of the same type are brought closer together, while the feature representations of boundary points of different types are pushed further apart; specifically, the pairwise Euclidean distance matrix between the feature vectors of all identified boundary points in a batch is calculated, and all boundary points are traversed, with each boundary point... First, we define anchor points. For each anchor point... Among other boundary points of the same category, the point furthest from it is selected as the "hardest positive sample". Then at all boundary points Among the boundary points of different categories, select the boundary points that satisfy the semi-hard condition. As negative samples, this process ensures that the triples used for training are challenging enough to pose a positive loss to the model, but not so difficult that training becomes unstable, thus significantly improving learning efficiency and model performance.

[0031] Furthermore, based on the category prediction, the corresponding semantic prediction probability, and the true label of each sample point, a region loss function is constructed: ; in, Represents the region loss function. Indicates the number of categories. An index representing the number of categories, Indicate category The weight, Indicates the number of sample points. Indicates the prediction point Category semantic prediction probability, Point The true label; obtain the true instance center offset vector and the predicted instance center offset vector for each sample point, and construct an offset loss function based on each true instance center offset vector and the predicted instance center offset vector: ; in, Represents the offset loss function. Represents the set of instance sample points. Indicates the first One sample point, Represents the predicted instance center offset vector. Represents the offset vector of the center of the real instance. express The norm of the loss function; constructing the total loss function using the boundary loss function, the region loss function, and the offset loss function: ; in, Represents the total loss function. Represents the boundary loss function. Indicates balance hyperparameters, Indicates balance hyperparameters, Indicates balance The hyperparameters are determined; the sparse convolutional neural network, the spatial channel attention model, and the segmentation head are optimized and trained using the total loss function.

[0032] To ensure the accuracy of segmentation results within the main region and to correctly classify most sample points, this invention discloses a generalized Dessian loss function as the region loss function. This function is used to alleviate the severe imbalance in the number of sample points of different categories in forest scenes. In one embodiment of this invention, the weight of each category is set to the reciprocal of the number of pixels in that category, giving greater weight to smaller target categories. This improves the accuracy of identifying small targets and avoids confusion and misclassification in complex scenes.

[0033] The region loss function, boundary loss function, and offset loss function disclosed in this invention provide complementary supervision information, ensuring the correct classification of the tree body and enabling the model to accurately locate the boundaries between different parts. This effectively overcomes the defect of the single region loss function being insensitive to boundaries and significantly improves the accuracy of the final segmentation results on both macroscopic regions and microscopic boundaries.

[0034] Step S40: Input the target forest point cloud into the trained sparse convolutional neural network, spatial channel attention model and segmentation head, output the semantic category prediction result of each voxel in the target forest point cloud, and use a clustering algorithm to cluster all the semantic category prediction results to obtain multiple single tree segmentation results.

[0035] Specifically, the target forest point cloud is input into an optimized training sparse convolutional neural network, a spatial channel attention model and a segmentation head, and the class of each voxel in the target forest point cloud and the corresponding instance center offset vector are output; according to each class and the corresponding instance center offset vector, all voxels belonging to the same tree are clustered together by using a density-based clustering algorithm (HDBSCAN, Hierarchical Density-Based Spatial Clustering of Applications with Noise), to obtain each single tree segmentation result in the target forest point cloud.

[0036] In the training phase of the model, the network's predicted output is compared with the artificially annotated ground truth, the error is calculated through a novel boundary-aware composite loss function, and the network parameters are optimized through the back propagation algorithm. The loss function ensures that the model not only learns the overall consistency of the region, but also pays special attention to the accuracy of the boundary. Finally, instance clustering is performed on the target forest point cloud, and the output features are refined through the attention module, and the semantic class prediction (e.g., trunk, crown, ground) and instance center offset vector of each voxel are output by the segmentation head of the network. These outputs are then used in a density-based clustering algorithm to aggregate points belonging to the same tree together to form the final single tree instance segmentation result.

[0037] The present application optimizes the allocation of computing resources by adaptive voxelization under the premise of ensuring accuracy, improves the efficiency of processing large-scale forest point clouds, and finally generates a segmentation result with more accurate geometry and clearer contours, providing a higher quality data basis for subsequent forestry parameter extraction.

[0038] Further, as shown in Figure 4 Based on the above point cloud single tree segmentation method based on collaborative attention and density adaptive voxelization, the present application also correspondingly provides a point cloud single tree segmentation system based on collaborative attention and density adaptive voxelization, wherein the point cloud single tree segmentation system based on collaborative attention and density adaptive voxelization comprises: A voxel grid construction module 51 is configured to obtain an original forest point cloud, calculate the density value of each sample point in the original forest point cloud, and perform mapping processing on all the density values to obtain a non-uniform voxel grid; A collaborative attention module 52 is configured to input the voxel grid into a sparse convolutional neural network, output a voxel feature map in the original forest point cloud, input the voxel feature map into a spatial channel attention model for multiple times of pooling processing, output a refined feature map, and input the refined feature map into a segmentation head to output the semantic prediction probability of each sample point; The model training module 53 is used to construct a boundary loss function using the feature representations of all boundary points in the refined feature map, construct a region loss function based on the true label and semantic prediction probability of each sample point, construct an offset loss function based on the true instance center offset vector and the predicted instance center offset vector of each sample point, and optimize the training of the sparse convolutional neural network, the spatial channel attention model, and the segmentation head using the boundary loss function, the region loss function, and the offset loss function. The result detection module 54 is used to input the target forest point cloud into a trained sparse convolutional neural network, a spatial channel attention model, and a segmentation head, output the semantic category prediction result of each voxel in the target forest point cloud, and use a clustering algorithm to cluster all the semantic category prediction results to obtain multiple single tree segmentation results.

[0039] Furthermore, such as Figure 5 As shown, based on the above-mentioned point cloud single-tree segmentation method and system based on collaborative attention and density adaptive voxelization, the present invention also provides a terminal, which includes a processor 10, a memory 20 and a display 30. Figure 5 Only some of the terminal components are shown; however, it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0040] In some embodiments, the memory 20 may be an internal storage unit of the terminal, such as a hard disk or memory. In other embodiments, the memory 20 may be an external storage device of the terminal, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc. Further, the memory 20 may include both internal and external storage devices. The memory 20 is used to store application software and various types of data installed on the terminal, such as the program code installed on the terminal. The memory 20 can also be used to temporarily store data that has been output or will be output. In one embodiment, the memory 20 stores a point cloud single-tree segmentation program 40 based on collaborative attention and density adaptive voxelization. This point cloud single-tree segmentation program 40 can be executed by the processor 10 to implement the point cloud single-tree segmentation method based on collaborative attention and density adaptive voxelization in this application.

[0041] The processor 10 can be a Central Processing Unit (CPU), microprocessor or other data processing chip in some embodiments, for running program codes stored in the memory 20 or processing data, such as executing the point cloud tree segmentation method based on collaborative attention and density adaptive voxelization.

[0042] The display 30 can be an LED display, a liquid crystal display, a touch liquid crystal display, an OLED (Organic Light-Emitting Diode) touch, etc. in some embodiments. The display 30 is used to display information of the terminal and to display a visualized user interface. The components of the terminal communicate with each other through a system bus.

[0043] In an embodiment, the following steps are implemented when the processor 10 executes the point cloud tree segmentation program 40 based on collaborative attention and density adaptive voxelization in the memory 20: Obtain an original forest point cloud, calculate the density value of each sample point in the original forest point cloud, and perform mapping processing on all the density values to obtain a non-uniform voxel grid; Input the voxel grid into a sparse convolutional neural network, output a voxel feature map in the original forest point cloud, input the voxel feature map into a spatial channel attention model for multiple pooling processing, output a refined feature map, and input the refined feature map into a segmentation head to output semantic prediction probability of each sample point; Construct a boundary loss function using the feature representation of all boundary points in the refined feature map, construct a region loss function according to the true label and semantic prediction probability of each sample point, construct an offset loss function according to the true instance center offset vector and predicted instance center offset vector of each sample point, and optimize and train the sparse convolutional neural network, the spatial channel attention model and the segmentation head using the boundary loss function, the region loss function and the offset loss function; Input a target forest point cloud into the trained sparse convolutional neural network, spatial channel attention model and segmentation head, output semantic class prediction results of each voxel in the target forest point cloud, and use a clustering algorithm to cluster all the semantic class prediction results to obtain multiple tree segmentation results.

[0044] The voxel grid comprises a high-density voxel grid, a medium-density voxel grid and a low-density voxel grid. The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: ; Wherein, represents any one sample point, represents the density value of , the number of sample points, represents the index of the sample point, represents the global bandwidth parameter, represents the global bandwidth parameter, represents the scaled Gaussian kernel according to , the number of sample points, represents the first sample point; The method comprises the following steps:

[0045] The spatial channel attention model comprises a channel attention model and a spatial attention model. The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps:

[0046] The voxel feature map is input into the channel attention model, the channel attention model performs global pooling on the voxel feature map, and average channel descriptors and maximum channel descriptors are output; The voxel feature map is input into the channel attention model, the channel attention model performs global pooling on the voxel feature map, and average channel descriptors and maximum channel descriptors are output; The average channel descriptors and the maximum channel descriptors are input into a network of a double-layer fully connected layer, and a channel attention weight map is output; The channel attention weight map and the voxel feature map are multiplied element by element in the channel to realize channel attention weighting, and an intermediate feature map is obtained; The intermediate feature map is input into the spatial attention model, the spatial attention model performs average pooling and maximum pooling on the intermediate feature map, and average spatial descriptors and maximum spatial descriptors are obtained respectively; The average spatial descriptors and the maximum spatial descriptors are spliced in the channel dimension of the spatial attention model, and a spliced feature is obtained; The spliced feature is processed by a three-dimensional sparse convolution layer, and a single-channel spatial attention weight map is output; The spatial attention weight map and the intermediate feature map are multiplied element by element by broadcast multiplication, and a refined feature map is obtained.

[0047] The feature representation of all boundary points in the refined feature map is used to construct a boundary loss function, specifically including: The three-dimensional coordinates of all sample points are obtained, and a k-d tree is constructed. For each sample point, the k-d tree is used to determine the k nearest neighboring points of the sample point to construct a field set; The field set of the current sample point is traversed. If the semantic label of at least one neighboring point is different from the semantic label of the sample point, the current sample point is defined as a boundary point, until all boundary points in all sample points are extracted; From the feature representation of all boundary points in the refined feature map, each boundary point is traversed, and all boundary points satisfying the semi-hard condition are defined as negative samples: ; Wherein, represents the current traversed boundary point, represents the boundary point farthest from the current traversed boundary point, represents the negative sample, represents the feature representation of the current traversed boundary point, represents the feature representation of characteristic representation of , representing a preset margin, characteristic representation of , Euclidean distance between , Euclidean distance between ; constructing a boundary loss function according to all the negative samples: ; wherein, the boundary loss function.

[0048] wherein, constructing a region loss function according to the true label and the semantic prediction probability of each sample point, constructing an offset loss function according to the true instance center offset vector and the predicted instance center offset vector of each sample point, and optimizing and training the sparse convolutional neural network, the spatial channel attention model and the segmentation head by using the boundary loss function, the region loss function and the offset loss function, constructing an offset loss function according to the true instance center and the predicted instance center offset vector of each sample point, and optimizing and training the sparse convolutional neural network, the spatial channel attention model and the segmentation head by using the boundary loss function, the region loss function and the offset loss function, specifically comprising: constructing a region loss function according to the class prediction, the corresponding semantic prediction probability of each sample point and the true label of each sample point: ; wherein, the region loss function, the number of categories, the index of the number of categories, the weight of the category , the number of sample points, the semantic prediction probability of the predicted point belongs to the category , the true label of the point ; obtaining the true instance center offset vector and the predicted instance center offset vector of each sample point, and constructing an offset loss function according to each of the true instance center offset vector and the predicted instance center offset vector: ; wherein, the offset loss function, the instance sample point set, denotes the i-th sample point, denotes the predicted instance center offset vector, denotes the real instance center offset vector, denotes the i-th sample point, denotes the i-th sample point, denotes the norm of constructing a total loss function by using the boundary loss function, the region loss function and the offset loss function: wherein, denotes the total loss function, denotes the boundary loss function, denotes the hyperparameter balancing denotes the hyperparameter balancing denotes the hyperparameter balancing denotes the hyperparameter balancing denotes the hyperparameter balancing ; optimizing and training the sparse convolutional neural network, the spatial channel attention model and the segmentation head by using the total loss function.

[0049] wherein, the target forest point cloud is input into the trained sparse convolutional neural network, spatial channel attention model and segmentation head, and the semantic class prediction result of each voxel in the target forest point cloud is output, and a clustering algorithm is used to cluster all the semantic class prediction results to obtain a plurality of single tree segmentation results, specifically including: the target forest point cloud is input into the optimized and trained sparse convolutional neural network, spatial channel attention model and segmentation head, and the class of each voxel in the target forest point cloud and the corresponding instance center offset vector are output; according to each of the class and the corresponding instance center offset vector, a density-based clustering algorithm is used to cluster all the voxels belonging to the same tree together to obtain each single tree segmentation result in the target forest point cloud.

[0050] The application also provides a computer readable storage medium, wherein the computer readable storage medium stores a point cloud single tree segmentation program based on collaborative attention and density adaptive voxelization, and the point cloud single tree segmentation program based on collaborative attention and density adaptive voxelization realizes the steps of the point cloud single tree segmentation method based on collaborative attention and density adaptive voxelization when executed by a processor.

[0051] ​In summary, the present application provides a point cloud tree segmentation method based on cooperative attention and density adaptive voxelization and related equipment, the method comprising: obtaining an original forest point cloud, calculating the density value of each sample point in the original forest point cloud, performing mapping processing on all the density values to obtain a non-uniform voxel grid; inputting the voxel grid into a sparse convolutional neural network to output a voxel feature map in the original forest point cloud, inputting the voxel feature map into a spatial channel attention model for multiple pooling processing to output a refined feature map, inputting the refined feature map into a segmentation head to output the semantic prediction probability of each sample point; constructing a boundary loss function using the feature representation of all boundary points in the refined feature map, constructing a region loss function according to the true label and semantic prediction probability of each sample point, constructing an offset loss function according to the true instance center offset vector and predicted instance center offset vector of each sample point, and optimizing and training the sparse convolutional neural network, the spatial channel attention model and the segmentation head using the boundary loss function, the region loss function and the offset loss function; constructing an offset loss function according to the true instance center and the predicted instance center offset vector of each sample point, and optimizing and training the sparse convolutional neural network, the spatial channel attention model and the segmentation head using the boundary loss function, the region loss function and the offset loss function; inputting a target forest point cloud into the trained sparse convolutional neural network, spatial channel attention model and segmentation head to output the semantic class prediction result of each voxel in the target forest point cloud, and clustering all the semantic class prediction results using a clustering algorithm to obtain multiple tree segmentation results. The present application optimizes the allocation of computing resources through adaptive voxelization under the premise of ensuring accuracy, improves the efficiency of processing large-scale forest point clouds, and finally generates segmentation results with more accurate geometric shapes and clearer contours, providing a higher-quality data foundation for subsequent forestry parameter extraction.

[0052] It should be noted that in this document, the terms "comprising", "including", or any other variant thereof are intended to cover non-exclusive inclusions, so that processes, methods, articles or terminals including a series of elements not only include those elements, but also include other elements not explicitly listed, or include elements inherent to such processes, methods, articles or terminals. Without more limitations, the element defined by the statement "including a" does not exclude the presence of another identical element in the process, method, article or terminal including the element.

[0053] Of course, those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing relevant hardware (such as a processor, a controller, etc.) through a computer program. The program can be stored in a computer-readable computer-readable storage medium, and the program can include the processes of the above-mentioned method embodiments when executed. The computer-readable storage medium can be a memory, a disk, an optical disk, etc.

[0054] It should be understood that the application is not limited to the above examples, and those skilled in the art can make improvements or changes according to the above description, and all these improvements and changes shall belong to the protection scope of the appended claims of the application.

Claims

1. A point cloud single-tree segmentation method based on collaborative attention and density adaptive voxelization, characterized in that, The point cloud single-tree segmentation method based on collaborative attention and density adaptive voxelization includes: Obtain the original forest point cloud, calculate the density value of each sample point in the original forest point cloud, and perform mapping processing on all the density values ​​to obtain a non-uniform voxel grid. The voxel grid is input into a sparse convolutional neural network, which outputs a voxel feature map in the original forest point cloud. The voxel feature map is then input into a spatial channel attention model for multiple pooling processes, and a refined feature map is output. The refined feature map is then input into a segmentation head, which outputs the semantic prediction probability of each sample point. A boundary loss function is constructed using the feature representations of all boundary points in the refined feature map. A region loss function is constructed based on the true label and semantic prediction probability of each sample point. An offset loss function is constructed based on the true instance center offset vector and the predicted instance center offset vector of each sample point. The boundary loss function, the region loss function, and the offset loss function are then used to optimize and train the sparse convolutional neural network, the spatial channel attention model, and the segmentation head. The target forest point cloud is input into a trained sparse convolutional neural network, a spatial channel attention model, and a segmentation head. The semantic category prediction result of each voxel in the target forest point cloud is output, and a clustering algorithm is used to cluster all the semantic category prediction results to obtain multiple individual tree segmentation results.

2. The point cloud tree segmentation method based on collaborative attention and density adaptive voxelization according to claim 1, characterized in that, The voxel grid includes: high-density voxel grid, medium-density voxel grid and low-density voxel grid; The process of acquiring the original forest point cloud, calculating the density value of each sample point in the original forest point cloud, and mapping all the density values ​​to obtain a non-uniform voxel mesh specifically includes: Obtain the original forest point cloud, extract multiple non-ground sample points from all sample points in the original forest point cloud, and pair all the non-ground sample points in pairs to calculate the pair distance of each pair of non-ground sample points; The median of all the pairwise distances is defined as the global bandwidth parameter, and the density value of all the sample points is calculated based on the global bandwidth parameter: ； in, Represents any sample point, express density value, Indicates the number of sample points. Indicates the index of the sample point. This represents the global bandwidth parameter. Indicates according to Scaled Gaussian kernel, Indicates the first One sample point; Mapping all the density values ​​yields the high-density voxel grid, the medium-density voxel grid, and the low-density voxel grid.

3. The point cloud tree segmentation method based on collaborative attention and density adaptive voxelization according to claim 1, characterized in that, The spatial channel attention model includes: a channel attention model and a spatial attention model; The process of inputting the voxel grid into a sparse convolutional neural network to output a voxel feature map in the original forest point cloud, inputting the voxel feature map into a spatial channel attention model for multiple pooling processes to output a refined feature map, and inputting the refined feature map into a segmentation head to output the semantic prediction probability of each sample point specifically includes: The voxel grid is input into a sparse convolutional neural network for deep feature extraction, and the voxel feature map of the original forest point cloud is output. The voxel feature map is input into the channel attention model, and an intermediate feature map is output. The intermediate feature map is then input into the spatial attention model, and a refined feature map is output. The refined feature map is input into the segmentation head, and the segmentation head outputs the category prediction, corresponding semantic prediction probability, and instance center offset vector for each sample point.

4. The point cloud tree segmentation method based on collaborative attention and density adaptive voxelization according to claim 3, characterized in that, The step of inputting the voxel feature map into the channel attention model and outputting an intermediate feature map, and then inputting the intermediate feature map into the spatial attention model and outputting a refined feature map, specifically includes: The voxel feature map is input into the channel attention model, which performs global pooling on the voxel feature map and outputs the average channel descriptor and the maximum channel descriptor. The average channel descriptor and the maximum channel descriptor are input into a two-layer fully connected network to output a channel attention weight map. The channel attention weight map and the voxel feature map are multiplied element-wise channel by channel to achieve channel attention weighting and obtain an intermediate feature map. The intermediate feature map is input into the spatial attention model, and the spatial attention model performs average pooling and max pooling on the intermediate feature map to obtain the average spatial descriptor and the max spatial descriptor, respectively. The average spatial descriptor and the maximum spatial descriptor are concatenated along the channel dimension of the spatial attention model to obtain the concatenated feature. The spliced ​​features are processed through a three-dimensional sparse convolutional layer to output a single-channel spatial attention weight map. The spatial attention weight map is multiplied element-wise by broadcasting the intermediate feature map to obtain the refined feature map.

5. The point cloud tree segmentation method based on collaborative attention and density adaptive voxelization according to claim 1, characterized in that, The construction of the boundary loss function using the feature representations of all boundary points in the refined feature map specifically includes: Obtain the three-dimensional coordinates of all the sample points and construct a kd-tree. For each sample point, use the kd-tree to determine the k nearest neighbor points to the sample point in order to construct a neighborhood set. Traverse the neighborhood set of the current sample point. If there is at least one neighboring point whose semantic label is different from the semantic label of the sample point, define the current sample point as a boundary point until all boundary points of all sample points are extracted. From the feature representations of all boundary points determined in the refined feature map, traverse each boundary point and define all boundary points that satisfy the semi-hard condition as negative samples: ； in, This indicates the boundary point being traversed. Indicates distance The farthest boundary point, Indicates a negative sample. express Feature representation, express Feature representation, express Feature representation, Indicates the preset margin. express and The Euclidean distance between them express and The Euclidean distance between them; Construct a boundary loss function based on all the negative samples: ； in, This represents the boundary loss function.

6. The point cloud tree segmentation method based on collaborative attention and density adaptive voxelization according to claim 3, characterized in that, The process of constructing a region loss function based on the true label and semantic prediction probability of each sample point, constructing an offset loss function based on the true instance center offset vector and the predicted instance center offset vector of each sample point, and optimizing the training of the sparse convolutional neural network, the spatial channel attention model, and the segmentation head using the boundary loss function, the region loss function, and the offset loss function specifically includes: Based on the category prediction, corresponding semantic prediction probability, and true label of each sample point, a region loss function is constructed: ； in, Represents the region loss function. Indicates the number of categories. An index representing the number of categories, Indicate category The weight, Indicates the number of sample points. Indicates the prediction point Category semantic prediction probability, Point The true label; Obtain the true instance center offset vector and the predicted instance center offset vector for each sample point, and construct an offset loss function based on each true instance center offset vector and the predicted instance center offset vector: ； in, This represents the offset loss function. Represents the set of instance sample points. Indicates the first One sample point, Represents the predicted instance center offset vector. Represents the offset vector of the center of the real instance. express The norm; The total loss function is constructed using the boundary loss function, the region loss function, and the offset loss function: ； in, Represents the total loss function. Represents the boundary loss function. Indicates balance hyperparameters, Indicates balance hyperparameters, Indicates balance hyperparameters; The sparse convolutional neural network, the spatial channel attention model, and the segmentation head are optimized and trained using the total loss function.

7. The point cloud tree segmentation method based on collaborative attention and density adaptive voxelization according to claim 1, characterized in that, The process involves inputting the target forest point cloud into a trained sparse convolutional neural network, a spatial channel attention model, and a segmentation head. The output is the semantic category prediction result for each voxel in the target forest point cloud. A clustering algorithm is then used to cluster all the semantic category prediction results to obtain multiple individual tree segmentation results. Specifically, this includes: The target forest point cloud is input into an optimized and trained sparse convolutional neural network, a spatial channel attention model, and a segmentation head, and the output is the category and corresponding instance center offset vector of each voxel in the target forest point cloud. Based on each category and the corresponding instance center offset vector, a density-based clustering algorithm is used to cluster all voxels belonging to the same tree together to obtain the segmentation result of each tree in the target forest point cloud.

8. A point cloud single-tree segmentation system based on collaborative attention and density adaptive voxelization, characterized in that, The point cloud single-tree segmentation system based on collaborative attention and density adaptive voxelization includes: A voxel mesh construction module is used to acquire the original forest point cloud, calculate the density value of each sample point in the original forest point cloud, and perform mapping processing on all the density values ​​to obtain a non-uniform voxel mesh. The collaborative attention module is used to input the voxel grid into a sparse convolutional neural network, output voxel feature maps in the original forest point cloud, input the voxel feature maps into a spatial channel attention model for multiple pooling processes, output a refined feature map, input the refined feature map into a segmentation head, and output the semantic prediction probability of each sample point. The model training module is used to construct a boundary loss function using the feature representations of all boundary points in the refined feature map, construct a region loss function based on the true label and semantic prediction probability of each sample point, construct an offset loss function based on the true instance center offset vector and the predicted instance center offset vector of each sample point, and optimize the training of the sparse convolutional neural network, the spatial channel attention model, and the segmentation head using the boundary loss function, the region loss function, and the offset loss function. The result detection module is used to input the target forest point cloud into a trained sparse convolutional neural network, a spatial channel attention model, and a segmentation head, output the semantic category prediction result of each voxel in the target forest point cloud, and use a clustering algorithm to cluster all the semantic category prediction results to obtain multiple individual tree segmentation results.

9. A terminal, characterized in that, The terminal includes: a memory, a processor, and a point cloud single-tree segmentation program based on collaborative attention and density adaptive voxelization stored in the memory and executable on the processor. When the point cloud single-tree segmentation program based on collaborative attention and density adaptive voxelization is executed by the processor, it implements the steps of the point cloud single-tree segmentation method based on collaborative attention and density adaptive voxelization as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a point cloud single-tree segmentation program based on collaborative attention and density adaptive voxelization. When the point cloud single-tree segmentation program based on collaborative attention and density adaptive voxelization is executed by a processor, it implements the steps of the point cloud single-tree segmentation method based on any one of claims 1-7.

Images