Large-scale forest individual tree instance segmentation method and device based on laser radar point cloud

Abstract

The invention provides a large-scale forest individual tree instance segmentation method and device based on laser radar point clouds, and the method comprises the steps: carrying out the voxelization processing of a forest point cloud to be subjected to individual tree instance segmentation, and obtaining a plurality of target voxels; obtaining voxel features, offsets and semantic tags of the target voxels according to a pre-trained sparse convolutional neural network backbone model; constructing an offset space according to the offset; determining a rough instance mask corresponding to each individual tree instance in the forest point cloud to be subjected to individual tree instance segmentation according to each target voxel and a predetermined average voxel number corresponding to individual tree instances of different tree species; determining coordinates and features of initial query vectors corresponding to the individual tree instances according to the rough instance masks; determining super-points, super-point features and super-point space coordinates corresponding to the forest point cloud to be subjected to individual tree instance segmentation; and based on a pre-trained model, the super-point features, the super-point space coordinates and the initial query vectors, obtaining a segmentation result, and realizing high-quality segmentation.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to a method and device for segmenting large-scale forest single tree instances based on laser radar point cloud. Background Art

[0002] Globally, the monitoring and management of forest resources plays a vital role in maintaining ecological balance, responding to climate change, and protecting biodiversity. Large-scale forests, with their dense distribution of trees and vast coverage, pose a huge challenge to traditional monitoring methods. Traditional forest survey methods rely on manual field measurements, which are not only time-consuming and labor-intensive, but also costly and difficult to achieve rapid monitoring over a large area. With the development of remote sensing technology, it has become possible to monitor forests using image data obtained from satellites or aerial platforms.

[0003] When dealing with forest scenes, existing large-scale single tree segmentation technologies usually split large-scale plots into small-scale plots, and then perform reasoning and splicing separately. Currently popular deep learning methods are mainly divided into two categories: clustering models and super-point models. Clustering models, such as clustering methods based on point cloud offset, cluster spatially close points by predicting the offset of each point in the point cloud, and then use ball query and breadth-first search algorithms to cluster and segment these dense point clouds. This method can effectively segment single trees into independent individuals when dealing with large-scale data sets. Super-point models, such as deep learning models based on super-points, first segment the point cloud into super-points according to the attributes of the point cloud itself, and then calculate the similarity between the initialized query vector and the super-point to obtain the instance segmentation result. This method reduces the amount of calculation and improves the segmentation efficiency by converting point cloud data into super-point representation.

[0004] However, both methods have certain limitations. The clustering model relies on the traditional point cloud clustering method when performing single tree segmentation. A major problem with this method is that it is highly sensitive to parameter settings and consumes a lot of time and memory resources when processing large-scale data sets. In order to solve the resource consumption problem, it is usually necessary to divide the plot into smaller parts for processing and then splice the results together. However, this method of segmentation and splicing will result in low quality of generated single tree instances and cannot meet the requirements of high-precision segmentation. On the other hand, the super-point model reduces computational consumption by over-segmenting the forest point cloud into super-points. Although this method improves computational efficiency to a certain extent, it also requires segmentation of the plot during the model training phase because a large number of gradients and optimizer parameters need to be stored. This segmentation strategy limits the generalization ability of the model, resulting in a significant decrease in inference accuracy on large-scale plots. Summary of the invention

[0005] In order to solve the above problems existing in the prior art, the present invention provides a large-scale forest single tree instance segmentation method and device based on laser radar point cloud, which specifically includes:

[0006] In a first aspect, the present invention provides a large-scale forest single tree instance segmentation method based on laser radar point cloud, comprising:

[0007] voxelize the forest point cloud for single tree instance segmentation to obtain multiple target voxels;

[0008] Input multiple target voxels into a pre-trained sparse convolutional neural network backbone model to obtain the voxel features, offsets and semantic labels of each target voxel corresponding to the forest point cloud to be segmented into single trees. The semantic labels are used to identify tree species.

[0009] Constructing an offset space according to the offset of each target voxel;

[0010] In the offset space, the target voxels are clustered by ball query and breadth-first search algorithm, and the rough instance mask corresponding to each single tree instance in the forest point cloud to be segmented is determined according to the predetermined average number of voxels corresponding to single tree instances of different tree species;

[0011] Determine the coordinates and features of an initial query vector corresponding to each single tree instance according to a rough instance mask corresponding to each single tree instance in the forest point cloud for single tree instance segmentation;

[0012] Performing over-segmentation processing on the forest point cloud to be segmented into single tree instances, obtaining super points corresponding to the forest point cloud to be segmented into single tree instances, and determining super point features and super point spatial coordinates;

[0013] Based on the pre-trained Transformer Decoder model, the segmentation result corresponding to the forest point cloud to be segmented into single trees is obtained according to the super-point features, super-point spatial coordinates and each initial query vector.

[0014] In a second aspect, the present invention further provides an electronic device, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

[0015] Memory, used to store computer programs;

[0016] The processor is used to implement any method provided in the first aspect when executing a program stored in the memory.

[0017] In a third aspect, the present invention provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, any one of the methods provided in the first aspect is implemented.

[0018] Beneficial effects of the present invention:

[0019] The large-scale forest single tree instance segmentation method and device based on laser radar point cloud provided by the present invention include voxelizing the forest point cloud to be segmented into single tree instances to obtain multiple target voxels; inputting the multiple target voxels into a pre-trained sparse convolutional neural network backbone model to obtain voxel features, offsets and semantic labels of each target voxel corresponding to the forest point cloud to be segmented into single tree instances, wherein the semantic labels are used to identify tree species; constructing an offset space according to the offsets of each target voxel; and clustering the target voxels in the offset space by ball query and breadth-first search algorithm. And according to the predetermined average number of voxels corresponding to single tree instances of different tree species, the coarse instance mask corresponding to each single tree instance in the forest point cloud to be segmented is determined; according to the coarse instance mask corresponding to each single tree instance in the forest point cloud to be segmented, the coordinates and features of the initial query vector corresponding to each single tree instance are determined; the forest point cloud to be segmented is over-segmented to obtain the super points corresponding to the forest point cloud to be segmented, and the super point features and super point spatial coordinates are determined; based on the pre-trained Transformer The Decoder model obtains the segmentation result corresponding to the forest point cloud for single tree instance segmentation based on the super-point features, super-point spatial coordinates and each initial query vector. Through the application of voxelization processing and feature extraction technology, the generated single tree instance mask is more accurate in spatial position, which significantly improves the segmentation quality. By adopting the Transformer Decoder model to further optimize the features of the query vector, the generalization ability and reasoning accuracy of the model on different plots are enhanced. The automated processing flow reduces manual intervention, simplifies the operation steps, reduces costs, and achieves significant improvement in single tree segmentation in large-scale forests.

[0020] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 The present invention provides a process for a large-scale forest single tree instance segmentation method based on laser radar point cloud;

[0022] Figure 2 A voxel schematic diagram provided by the present invention. DETAILED DESCRIPTION

[0023] The present invention is further described in detail below with reference to specific embodiments, but the embodiments of the present invention are not limited thereto.

[0024] In order to solve the problems existing in the prior art, the present invention provides a large-scale forest single tree instance segmentation method based on lidar point cloud, such as Figure 1 As shown, including:

[0025] S101, voxelizing the forest point cloud to be segmented into single tree instances to obtain a plurality of target voxels.

[0026] like Figure 2 As shown in the figure, a voxel is a cubic unit in three-dimensional space, similar to a pixel in a two-dimensional image. Voxelization is to divide the point cloud data into a series of small cubes, and the points in each voxel can be represented by its centroid or average value. Voxelization helps to reduce the amount of data and maintain the spatial structure.

[0027] S102, inputting a plurality of target voxels into a pre-trained sparse convolutional neural network backbone model to obtain voxel features, offsets and semantic labels of each target voxel corresponding to the forest point cloud to be segmented into single tree instances.

[0028] The sparse convolutional neural network backbone model MinkUNet34C is a U-Net architecture variant based on the Minkowski sparse convolutional network, which is used to process high-dimensional sparse data. After the high-dimensional sparse data passes through the MinkUNet34C backbone network, the features of each voxel are obtained, and the offset distance from the instance center is predicted based on the voxel features. After all voxels are offset, each category is clustered in the spatial coordinate system according to their corresponding semantic categories.

[0029] The method for constructing a sparse convolutional neural network backbone model can be found in the method disclosed in the document "Choy C, Gwak JY, Savarese S. 4d spatio-temporal convnets: Minkowski convolutional neural networks [C] / / Proceedings of the IEEE / CVF conference on computer vision and pattern recognition. 2019: 3075-3084."

[0030] In order to achieve the prediction of offset and semantic label, the present invention adds two multilayer perceptron (MLP) layers on the basis of the original sparse convolutional neural network backbone model to predict the offset value and semantic label.

[0031] Semantic tags are used to identify tree species.

[0032] By constructing a sparse convolutional neural network, it is possible to efficiently process large-scale point cloud data and achieve fast single tree segmentation without sacrificing memory and time resources.

[0033] S103: construct an offset space according to the offset of each target voxel.

[0034] Specifically, constructing the offset space according to the offset of each target voxel includes: constructing the offset space according to the coordinates of each target voxel obtained through voxelization and the offset of each target voxel.

[0035] Exemplarily, the offset space is constructed after the coordinates of each target voxel and the offset of each target voxel are added.

[0036] S104. In the offset space, the target voxels are clustered by ball query and breadth-first search algorithm, and the coarse instance mask corresponding to each single tree instance in the forest point cloud to be segmented is determined according to the predetermined average number of voxels corresponding to single tree instances of different tree species.

[0037] In a possible implementation, in the offset space, the target voxels are clustered by ball query and breadth-first search algorithm, and the rough instance mask corresponding to each single tree in the forest point cloud to be segmented is determined according to the predetermined average number of voxels corresponding to single tree instances of different tree species, including the following steps A1 and A2:

[0038] A1. In the offset space, the target voxels are clustered by ball query and breadth-first search algorithm to obtain the instance mask corresponding to each clustering result.

[0039] For the specific clustering method, please refer to the method disclosed in the document “Vu T, Kim K, Luu TM, et al. Softgroup for 3dinstance segmentation on point clouds [C] / / Proceedings of the IEEE / CVFConference on Computer Vision and Pattern Recognition. 2022: 2708-2717.”

[0040] A2. According to the instance masks corresponding to each clustering result and the predetermined average number of voxels corresponding to single tree instances of different tree species, determine the single tree instances included in the forest point cloud for single tree instance segmentation and the coarse instance masks corresponding to each single tree instance.

[0041] In one possible implementation, the average number of voxels corresponding to individual tree instances of different tree species is determined, including B1-B3:

[0042] B1. Perform voxel processing on multiple labeled forest point cloud samples to obtain voxels corresponding to trees in each forest point cloud sample.

[0043] B2. Count the total number of single tree instances of each tree species and the number of voxels corresponding to the single tree instances of each tree species in the voxels corresponding to the trees.

[0044] B3. Determine the average number of voxels corresponding to the single tree instances of different tree species based on the total number of single tree instances of each tree species and the number of voxels corresponding to the single tree instances of each tree species.

[0045] S105 . Determine the coordinates and features of the initial query vector corresponding to each single tree instance according to the rough instance mask corresponding to each single tree instance in the forest point cloud for single tree instance segmentation.

[0046] In a possible implementation, according to the rough instance mask corresponding to each single tree instance in the forest point cloud for single tree instance segmentation, the coordinates and features of the initial query vector corresponding to each single tree instance are determined, including the following steps C1 and C2:

[0047] C1. According to the rough instance mask corresponding to each single tree instance in the forest point cloud to be segmented, the coordinates and features of the voxels corresponding to each single tree instance in the forest point cloud to be segmented are determined in the offset space.

[0048] C2. Average pooling is performed on the coordinates and features of the voxels corresponding to each single tree instance in the forest point cloud to be segmented in the offset space to obtain the coordinates and features of the initial query vector corresponding to each single tree instance.

[0049] Exemplarily, the offset P of the corresponding voxel of each instance is found according to the correspondence between each instance mask and the voxel l , average pooling is performed on the voxels corresponding to each instance to obtain the initial query vector coordinates and query vector features in and Represents the coordinates and features of the nth query vector.

[0050] S106 , performing over-segmentation processing on the forest point cloud to be segmented into single tree instances, obtaining super points corresponding to the forest point cloud to be segmented into single tree instances, and determining super point features and super point space coordinates.

[0051] Superpoints can be viewed as constant components in the piecewise constant approximation of the surface of geometric and radiant point features, and superpoint indices can be obtained through over-segmentation.

[0052] The over-segmentation method can refer to the point cloud over-segmentation method disclosed in the literature “Robert D, Raguet H, Landrieu L. Efficient 3D semantic segmentation with superpoint transformer [C] / / Proceedings of the IEEE / CVF International Conference on Computer Vision. 2023: 17195-17204.”

[0053] For example, the process of calculating super points by over-segmentation of point clouds usually involves the following key steps:

[0054] 1 Preprocessing: Perform noise reduction on the original point cloud to reduce the interference of outliers in subsequent algorithms.

[0055] 2 Voxelization: The entire point cloud space is divided into voxel sizes to generate a voxel cloud. Each voxel may contain multiple original point cloud data. The resolution of voxelization will affect the shape and size of subsequent super points.

[0056] 3. Establish an adjacency graph: In the voxel space, establish an adjacency graph based on the spatial relationship between voxels, usually considering 26 neighborhoods.

[0057] 4. Superpoint growth: Starting from the seed voxel, the flow-constrained clustering algorithm is used to traverse the adjacent voxels layer by layer, and the similarity distance between the adjacent voxels and the current superpoint is calculated. According to the preset threshold, the voxels are clustered into uniform and smooth superpoints that adhere to the target boundary.

[0058] 5. Superpoint feature extraction: Extract geometric features for each superpoint, such as normal vector, centroid, etc. The similarity between adjacent superpoints is measured by normalizing the distance value.

[0059] 6. Superpoint graph construction: Construct a superpoint graph, where nodes represent superpoints and edges represent adjacency relationships between superpoints.

[0060] 7 Post-processing: For over-segmented point clouds, calculate the concave-convex relationship between different super-points to merge adjacent super-points. Use CC (Convexity Criterion) and SC (Sanity Criterion) criteria to evaluate the geometric relationship between super-points.

[0061] This process can effectively segment large-scale point cloud data into meaningful super-points, providing preprocessing for subsequent semantic segmentation, object recognition and other tasks. In this way, the processing efficiency can be improved while maintaining the details of the point cloud.

[0062] In a possible implementation, determining superpoint features and superpoint spatial coordinates includes: extracting features and spatial coordinates of voxels corresponding to each superpoint; and performing average pooling processing on the features and spatial coordinates of the voxels corresponding to each superpoint to obtain superpoint features and superpoint spatial coordinates.

[0063] S107. Based on the pre-trained Transformer Decoder model, a segmentation result corresponding to the forest point cloud to be segmented into single trees is obtained according to the super-point features, the super-point spatial coordinates and each initial query vector.

[0064] In a possible implementation, based on a pre-trained Transformer Decoder model, the segmentation results corresponding to the forest point cloud to be segmented into single trees are obtained according to the super-point features, super-point spatial coordinates and each initial query vector, including D1-D4:

[0065] D1. Through the pre-trained Transformer Decoder model, according to the superpoint space coordinates, optimize the features of each initial query vector, obtain the target query vector corresponding to each initial query vector, and predict the semantic label and offset of each target query vector. The semantic label is used to identify the tree species.

[0066] D2. Determine the dot product similarity between each target query vector and super-point feature.

[0067] D3. Determine the instance binary mask corresponding to each target query vector based on the dot product similarity.

[0068] D4. According to the instance binary mask and semantic label corresponding to each target query vector, the segmentation result corresponding to the forest point cloud to be segmented into single trees is obtained.

[0069] Optionally, based on a pre-trained Transformer Decoder model, the segmentation result corresponding to the forest point cloud to be segmented into single trees is obtained according to the super-point features, the super-point spatial coordinates and each initial query vector, which is expressed as:

[0070] Q,K,V=MLP(Q f ,Q f ,Q f ),

[0071] SP'=SP f +embedding(SP l ),

[0072]

[0073] Q out =FFN(Q'),

[0074] Q offset ,Q label =MLP(Q out ),

[0075] Q' c =Q c +Q offset ,

[0076] Predscore = (Q out ×Q offset )Sigmoid,

[0077] Predmask=(Predscore>0.5),

[0078] Among them, Q represents the Transformer self-attention Query vector, K represents the Transformer self-attention Key vector, V represents the Transformer self-attention Value vector, MLP() represents the multi-layer perceptron, Q f represents the query vector feature, SP' represents the super-point fusion feature, SP f represents super-point features, embedding() represents Fourier encoding, SP l represents the superpoint coordinates, Q' represents the Transformer cross-attention Query vector, Self-Attention() represents self-attention, Cross-Attention() represents cross-attention, softmax() represents the normalized exponential function, d k represents the number of attention channels, the superscript T represents transposition, Q' represents the query vector after cross-attention optimization, Q out represents the query vector output after optimization by the feedforward neural network FFN(), FFN() represents the feedforward neural network, Q offset represents the offset of the query vector, Q label Represents the semantic label of the query vector, Q' c represents the coordinates of the query vector in the offset space, Q c Represents the initial original coordinates of the query vector, Predscore represents the dot product similarity, Predmask represents the instance binary mask, and Sigmoid represents the S-type function.

[0079] By calculating the dot product similarity between the super-point features and the query vector, single tree instances can be accurately identified and segmented, improving the accuracy of instance segmentation.

[0080] Optionally, before S101, the method further includes: dividing the forest into multiple plots, and obtaining a lidar point cloud corresponding to each plot.

[0081] Accordingly, steps S101 to S107 are executed for each plot, and then all plots are traversed to complete the single tree instance segmentation of the entire forest.

[0082] Optionally, the method further includes: using the task average precision (mAP) as a general evaluation metric for instance segmentation, setting the IoU threshold to an average of 50% to 95%, and setting the step size to 5%.

[0083] Using task average precision as the evaluation metric, we can comprehensively and accurately evaluate the model performance under various IoU thresholds.

[0084] The large-scale forest single tree instance segmentation method based on laser radar point cloud provided by the present invention comprises voxelizing the forest point cloud to be segmented into single trees to obtain a plurality of target voxels; inputting the plurality of target voxels into a pre-trained sparse convolutional neural network backbone model to obtain voxel features, offsets and semantic labels of each target voxel corresponding to the forest point cloud to be segmented into single trees, wherein the semantic labels are used to identify tree species; constructing an offset space according to the offsets of each target voxel; clustering the target voxels in the offset space by ball query and breadth-first search algorithm, and According to the predetermined average number of voxels corresponding to single tree instances of different tree species, the coarse instance mask corresponding to each single tree instance in the forest point cloud to be segmented is determined; according to the coarse instance mask corresponding to each single tree instance in the forest point cloud to be segmented, the coordinates and features of the initial query vector corresponding to each single tree instance are determined; the forest point cloud to be segmented is over-segmented to obtain the super points corresponding to the forest point cloud to be segmented, and the super point features and super point spatial coordinates are determined; based on the pre-trained Transformer The Decoder model obtains the segmentation result corresponding to the forest point cloud for single tree instance segmentation based on the super-point features, super-point spatial coordinates and each initial query vector. Through the application of voxelization processing and feature extraction technology, the generated single tree instance mask is more accurate in spatial position, which significantly improves the segmentation quality. By adopting the Transformer Decoder model to further optimize the features of the query vector, the generalization ability and reasoning accuracy of the model on different plots are enhanced. The automated processing flow reduces manual intervention, simplifies the operation steps, reduces costs, and achieves significant improvement in single tree segmentation in large-scale forests.

[0085] In order to further prove the beneficial effects of the present invention, the present invention also provides a set of experimental data, as follows:

[0086] 1. Simulation conditions

[0087] The present invention is based on a central processing unit of Inter(R)Core(TM)i7-13700KF 5.4GHz CPU, NVIDIAA6000 GPU, Ubuntu 20.04 operating system, the deep learning framework pytorch version is 1.13, the CUDA version is 11.8, and the python version is 3.8.

[0088] The methods compared in the experiment are as follows:

[0089] One is a clustering model, denoted as Softgroup in the experiment, and the reference is Vu T, Kim K, Luu TM, et al. Softgroup for 3d instance segmentation on point clouds[C] / / Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition. 2022: 2708-2717.

[0090] The other is the superpoint model, denoted as SPFormer in the experiment, and the reference is Sun J, Qing C, Tan J, et al. Superpoint transformer for 3d scene instance segmentation [C] / / Proceedings of the AAAI Conference on Artificial Intelligence. 2023, 37 (2): 2393-2401.

[0091] 2. Simulation content

[0092] According to a specific embodiment of the present invention, the instance segmentation accuracy is calculated and compared with the accuracy, inference time and video memory consumption of the Softgroup method and the SPFormer method. The results are shown in Table 1.

[0093] Table 1. Segmentation accuracy, inference time and video memory consumption of single tree instances in the forest

[0094] method Softgroup SPFormer The present invention <![CDATA[mAP 50 (%)]]> 21.61 24.39 45.73 Reasoning time (it / s) 89.2 20.4 38.1 Video memory usage (GB) 24.7 46.3 37.8

[0095] As can be seen from Table 1, since the query vector is initialized based on clustering and then refined by the Transformer decoder, a higher recognition effect can be achieved, which verifies the advancement of the present invention.

[0096] The present invention also provides a structure of an electronic device, including a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus.

[0097] Memory, used to store computer programs;

[0098] The processor is used to implement the steps provided in the above method embodiment when executing the program stored in the memory.

[0099] The communication interface is used for communication between the above electronic device and other devices.

[0100] The method provided in the embodiment of the present invention can be applied to electronic devices. Specifically, the electronic device can be: a desktop computer, a portable computer, an intelligent mobile terminal, a server, etc. This is not limited here, and any electronic device that can implement the present invention belongs to the protection scope of the present invention.

[0101] The present invention also provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps provided in the above method embodiment are implemented.

[0102] As for the electronic device / storage medium embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the specific content and beneficial effects and other related matters can be referred to the partial description of the method embodiment.

[0103] The terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0104] The above contents are further detailed descriptions of the present invention in combination with specific preferred embodiments, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For ordinary technicians in the technical field to which the present invention belongs, several simple deductions or substitutions can be made without departing from the concept of the present invention, which should be regarded as falling within the protection scope of the present invention.

Claims

1. A large-scale forest single tree instance segmentation method based on LiDAR point cloud, characterized in that: include: voxelize the forest point cloud for single tree instance segmentation to obtain multiple target voxels; Inputting the plurality of target voxels into a pre-trained sparse convolutional neural network backbone model, obtaining voxel features, offsets and semantic labels of each of the target voxels corresponding to the forest point cloud to be segmented into single trees, wherein the semantic labels are used to identify tree species; constructing an offset space according to the offset of each of the target voxels; In the offset space, the target voxels are clustered by ball query and breadth-first search algorithm, and a rough instance mask corresponding to each single tree instance in the forest point cloud to be segmented is determined according to a predetermined average number of voxels corresponding to single tree instances of different tree species; Determining coordinates and features of an initial query vector corresponding to each single tree instance according to a rough instance mask corresponding to each single tree instance in the forest point cloud to be segmented into single tree instances; Performing over-segmentation processing on the forest point cloud to be segmented into single tree instances, obtaining super points corresponding to the forest point cloud to be segmented into single tree instances, and determining super point features and super point spatial coordinates; Based on the pre-trained Transformer Decoder model, a segmentation result corresponding to the forest point cloud to be segmented into single trees is obtained according to the super-point features, the super-point spatial coordinates and each of the initial query vectors.

2. The method according to claim 1, characterized in that Determine the average number of voxels corresponding to individual tree instances of different tree species, including: voxelize the labeled forest point cloud samples to obtain voxels corresponding to trees in each of the forest point cloud samples; Counting the total number of single tree instances of each tree species and the number of voxels corresponding to the single tree instances of each tree species in the voxels corresponding to the trees; According to the total number of single tree instances of each tree species and the number of voxels corresponding to the single tree instances of each tree species, the average number of voxels corresponding to the single tree instances of different tree species is determined.

3. The method according to claim 1, characterized in that In the offset space, the target voxels are clustered by ball query and breadth-first search algorithm, and a rough instance mask corresponding to each single tree in the forest point cloud to be segmented is determined according to the predetermined average number of voxels corresponding to single tree instances of different tree species; In the offset space, clustering the target voxels by ball query and breadth-first search algorithm to obtain instance masks corresponding to each clustering result; According to the instance masks corresponding to each of the clustering results and the predetermined average number of voxels corresponding to the single tree instances of different tree species, the single tree instances included in the forest point cloud for single tree instance segmentation and the coarse instance masks corresponding to each single tree instance are determined.

4. The method according to claim 1, characterized in that The constructing the offset space according to the offset of each target voxel comprises: An offset space is constructed according to the coordinates of each target voxel obtained through voxelization and the offset of each target voxel.

5. The method according to claim 1, characterized in that The method of obtaining a segmentation result corresponding to the forest point cloud to be segmented into single trees based on the pre-trained TransformerDecoder model according to the super-point features, the super-point spatial coordinates and each of the initial query vectors includes: By using a pre-trained Transformer Decoder model, according to the superpoint space coordinates, the features of each of the initial query vectors are optimized to obtain a target query vector corresponding to each of the initial query vectors, and a semantic label and offset of each of the target query vectors are predicted; Determining the dot product similarity between each of the target query vectors and the super point features respectively; Determining, according to the dot product similarity, an instance binary mask corresponding to each of the target query vectors; According to the instance binary mask and the semantic label corresponding to each of the target query vectors, a segmentation result corresponding to the forest point cloud to be segmented into single trees is obtained.

6. The method according to claim 1, characterized in that The step of determining the coordinates and features of the initial query vector corresponding to each single tree instance according to the rough instance mask corresponding to each single tree instance in the forest point cloud to be segmented into single tree instances comprises: Determine, according to the rough instance mask corresponding to each single tree instance in the forest point cloud to be segmented, the coordinates and features of the voxels corresponding to each single tree instance in the forest point cloud to be segmented, in the offset space; The coordinates and features of the voxels corresponding to each single tree instance in the forest point cloud to be segmented into single tree instances in the offset space are respectively average pooled to obtain the coordinates and features of the initial query vector corresponding to each single tree instance.

7. The method according to claim 5, characterized in that Based on the pre-trained TransformerDecoder model, the segmentation result corresponding to the forest point cloud to be segmented into single trees is obtained according to the super point features, the super point spatial coordinates and each of the initial query vectors, which is expressed as: Q,K,V=MLP(Q f ,Q f ,Q f ), SP'=SP f +embedding(SP l ), Q out =FFN(Q”), Q offset ,Q label =MLP(Q out ), Q' c =Q c +Q offset , Predscore=(Q out ×Q offset )Sigmoid, Predmask=(Predscore>0.5), Among them, Q represents the Transformer self-attention Query vector, K represents the Transformer self-attention Key vector, V represents the Transformer self-attention Value vector, MLP() represents the multi-layer perceptron, Q f represents the query vector feature, SP' represents the super-point fusion feature, SP f represents super-point features, embedding() represents Fourier encoding, SP l represents the superpoint coordinates, Q' represents the Transformer cross-attention Query vector, Self-Attention() represents self-attention, Cross-Attention() represents cross-attention, softmax() represents the normalized exponential function, d k represents the number of attention channels, the superscript T represents transposition, Q' represents the query vector after cross-attention optimization, Q out represents the query vector output after optimization by the feedforward neural network FFN(), FFN() represents the feedforward neural network, Q offset represents the offset of the query vector, Q label Represents the semantic label of the query vector, Q' c represents the coordinates of the query vector in the offset space, Q c Represents the initial original coordinates of the query vector, Predscore represents the dot product similarity, Predmask represents the instance binary mask, and Sigmoid represents the S-type function.

8. The method according to claim 1, characterized in that Determine the superpoint features and superpoint space coordinates, including: Extracting the features and spatial coordinates of the voxels corresponding to each of the superpoints; Average pooling is performed on the features and spatial coordinates of the voxels corresponding to each superpoint to obtain the superpoint features and the superpoint spatial coordinates.

9. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, for implementing any of the methods described in claims 1-8 when executing a program stored in a memory.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images