Large-scale forest single tree instance segmentation method and device based on lidar point cloud

Abstract

The application provides a large-scale forest single-tree instance segmentation method and device based on a laser radar point cloud, a plurality of target voxels are obtained by voxelizing a forest point cloud to be subjected to single-tree instance segmentation; voxel features, offsets and semantic labels of the target voxels are obtained according to a pre-trained sparse convolutional neural network backbone model; an offset space is constructed according to the offsets; rough instance masks corresponding to each single-tree instance in the forest point cloud to be subjected to single-tree instance segmentation are determined according to the target voxels and average voxel numbers corresponding to different single-tree instances of different tree species which are determined in advance; coordinates and features of initial query vectors corresponding to each single-tree instance are determined according to the rough instance masks; super points corresponding to the forest point cloud to be subjected to single-tree instance segmentation, super point features and super point space coordinates are determined; and a segmentation result is obtained based on a pre-trained model, the super point features, the super point space coordinates and the initial query vectors, so that high-quality segmentation is achieved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, in particular to a large-scale forest single tree instance segmentation method and device based on laser radar point cloud. BACKGROUND

[0002] In the global range, the monitoring and management of forest resources play a crucial role in maintaining ecological balance, responding to climate change and protecting biodiversity. Large-scale forests, due to their dense tree distribution and wide coverage, pose great challenges to traditional monitoring methods. Traditional forest survey methods rely on manual field measurement, which is not only time-consuming and labor-intensive, but also costly, making it difficult to achieve rapid monitoring on a large scale. With the development of remote sensing technology, it has become possible to use satellite or aerial platform image data for forest monitoring.

[0003] The existing large-scale single tree segmentation technology usually splits the large-scale plot into small-scale plots, and then performs reasoning and splicing respectively when dealing with forest scenes. The current popular deep learning methods mainly include two categories: clustering model and super point model. The clustering model, such as the point cloud offset-based clustering method, predicts the offset of each point in the point cloud, and then uses sphere query and breadth-first search algorithm to cluster and segment the dense point cloud. This method can effectively segment single trees into independent individuals when dealing with large-scale data sets. The super point model, such as the super point-based deep learning model, first divides the point cloud into super points according to the attributes of the point cloud itself, and then calculates the similarity between the initialized query vector and the super points to obtain the instance segmentation result. This method converts point cloud data into super point representation, reducing the computational complexity and improving the segmentation efficiency.

[0004] However, both methods have certain limitations. The clustering model relies on traditional point cloud clustering methods when segmenting single trees. One of the main problems of this method is that it is highly sensitive to parameter settings, and it consumes a large amount of time and memory resources when dealing with large-scale data sets. In order to solve the problem of resource consumption, it is usually necessary to divide the plot into smaller parts for processing, and then splice the results. However, this segmentation and splicing method will result in low-quality single tree instances, which cannot meet the demand of high-precision segmentation. On the other hand, the super point model reduces computational consumption by over-segmenting forest point clouds into super points. Although this method improves the computational efficiency to some extent, it also requires the storage of a large number of gradient and optimizer parameters during the model training phase, which also requires the plot to be divided for processing. This segmentation strategy limits the generalization ability of the model, resulting in a significant decrease in reasoning accuracy on large-scale plots. SUMMARY

[0005] In order to solve the above problems existing in the prior art, the present application provides a large-scale forest single tree instance segmentation method and device based on laser radar point cloud, specifically comprising:

[0006] In the first aspect, the present application provides a large-scale forest single tree instance segmentation method based on laser radar point cloud, comprising:

[0007] The forest point cloud to be subjected to single tree instance segmentation is voxelized to obtain a plurality of target voxels;

[0008] The plurality of target voxels are input into a pre-trained sparse convolutional neural network backbone model to obtain voxel features, offsets and semantic labels of each target voxel corresponding to the forest point cloud to be subjected to single tree instance segmentation, the semantic labels being used to identify tree species;

[0009] An offset space is constructed according to the offsets of the target voxels;

[0010] In the offset space, the target voxels are clustered through a sphere query and a breadth-first search algorithm, and a rough instance mask corresponding to each single tree instance in the forest point cloud to be subjected to single tree instance segmentation is determined according to a pre-determined average number of voxels corresponding to single tree instances of different tree species;

[0011] The coordinates and features of an initial query vector corresponding to each single tree instance are determined according to the rough instance mask corresponding to each single tree instance in the forest point cloud to be subjected to single tree instance segmentation;

[0012] The forest point cloud to be subjected to single tree instance segmentation is subjected to over-segmentation to obtain super points corresponding to the forest point cloud to be subjected to single tree instance segmentation, and super point features and super point space coordinates are determined;

[0013] Based on a pre-trained Transformer Decoder model, a segmentation result corresponding to the forest point cloud to be subjected to single tree instance segmentation is obtained according to the super point features, the super point space coordinates and the initial query vectors.

[0014] In the second aspect, the present application further provides an electronic device comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory complete communication with each other through the communication bus;

[0015] The memory is used to store a computer program;

[0016] The processor is used to execute the program stored on the memory to implement any method provided in the first aspect.

[0017] In a third aspect, the present application provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement any method provided in the first aspect.

[0018] Advantages of the present application:

[0019] The method and device for large-scale forest single tree instance segmentation based on laser radar point cloud provided by the present application, comprising: voxelizing a forest point cloud to be subjected to single tree instance segmentation to obtain a plurality of target voxels; inputting the plurality of target voxels into a pre-trained sparse convolutional neural network backbone model to obtain voxel features, offsets and semantic labels of each target voxel corresponding to the forest point cloud to be subjected to single tree instance segmentation, the semantic labels being used to identify tree species; constructing an offset space according to the offsets of the target voxels; clustering the target voxels in the offset space through a sphere query and a breadth-first search algorithm, and determining a rough instance mask corresponding to each single tree instance in the forest point cloud to be subjected to single tree instance segmentation according to a pre-determined average number of voxels corresponding to different single tree instances of different tree species; determining the coordinates and features of an initial query vector corresponding to each single tree instance according to the rough instance mask corresponding to each single tree instance in the forest point cloud to be subjected to single tree instance segmentation; performing over-segmentation on the forest point cloud to be subjected to single tree instance segmentation to obtain super points corresponding to the forest point cloud to be subjected to single tree instance segmentation, and determining super point features and super point space coordinates; and obtaining a segmentation result corresponding to the forest point cloud to be subjected to single tree instance segmentation based on a pre-trained Transformer Decoder model according to the super point features, the super point space coordinates and the initial query vectors. Through the application of voxelization processing and feature extraction technology, the generated single tree instance mask is more accurate in spatial position, and the segmentation quality is significantly improved. By using the Transformer Decoder model to further optimize the features of the query vector, the generalization ability and reasoning accuracy of the model on different plots are enhanced. The automatic processing process reduces manual intervention, simplifies the operation steps, reduces the cost, and realizes significant improvement of large-scale forest single tree segmentation.

[0020] The present application will be further described in detail below with reference to the accompanying drawings and embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0021] Figure 1 The process of the large-scale forest single tree instance segmentation method based on laser radar point cloud provided by the present application;

[0022] Figure 2 The voxel schematic diagram provided by the present application. DETAILED DESCRIPTION

[0023] The application will be described in further detail below with reference to specific embodiments, but the embodiments of the application are not limited thereto.

[0024] To solve the problems in the prior art, the application provides a large-scale forest single tree instance segmentation method based on a laser radar point cloud, as shown in the formula (I): Figure 1 As shown in the formula (I), the method comprises the following steps.

[0025] S101, voxelizing a forest point cloud to be subjected to single tree instance segmentation to obtain a plurality of target voxels.

[0026] As shown in the formula (I), a voxel is a cubic unit in a three-dimensional space, similar to a pixel in a two-dimensional image. Voxelization is to divide point cloud data into a series of small cubes, and the points in each voxel can be represented by their centroids or average values. Voxelization helps to reduce the amount of data and maintain the spatial structure. Figure 2

[0027] S102, inputting the plurality of target voxels into a pre-trained sparse convolutional neural network backbone model to obtain voxel features, offset values and semantic labels of each target voxel corresponding to the forest point cloud to be subjected to single tree instance segmentation.

[0028] The sparse convolutional neural network backbone model MinkUNet34C is a U-Net architecture variant based on the Minkowski sparse convolutional network, which is used to process high-dimensional sparse data. The high-dimensional sparse data obtains the features of each voxel after passing through the MinkUNet34C backbone network, and predicts the offset distance from the voxel to the instance center according to the voxel features. After offsetting all voxels, each category is clustered in the spatial coordinate system according to their corresponding semantic categories.

[0029] The method for constructing the sparse convolutional neural network backbone model can refer to the method disclosed in the document "Choy C, Gwak J Y, Savarese S. 4d spatio-temporal convnets: Minkowski convolutional neural networks [C] / / Proceedings of the IEEE / CVF conference on computer vision and pattern recognition. 2019: 3075-3084."

[0030] In order to realize the prediction of the offset value and the semantic label, the application adds two layers of multilayer perceptron (MLP) layers to the original sparse convolutional neural network backbone model for predicting the offset value and the semantic label.

[0031] ​The semantic label is used to identify the tree species.

[0032] By constructing a sparse convolutional neural network, large-scale point cloud data can be efficiently processed without sacrificing memory and time resources, and fast single tree segmentation is achieved.

[0033] S103, constructing an offset space according to the offset of each target voxel.

[0034] Specifically, the offset space is constructed according to the coordinates of each target voxel obtained by voxelization processing and the offset of each target voxel.

[0035] For example, the offset space is constructed by adding the coordinates of each target voxel and the offset of each target voxel.

[0036] S104, in the offset space, the target voxel is clustered by a sphere query and a breadth-first search algorithm, and according to a predetermined average number of voxels corresponding to different tree species single tree instances, a rough instance mask corresponding to each single tree instance in the forest point cloud to be segmented is determined. Instance.

[0037] In one possible implementation, in the offset space, the target voxel is clustered by a sphere query and a breadth-first search algorithm, and according to a predetermined average number of voxels corresponding to different tree species single tree instances, a rough instance mask corresponding to each single tree in the forest point cloud to be segmented is determined. Instance, including the following steps A1 and A2:

[0038] A1, in the offset space, the target voxel is clustered by a sphere query and a breadth-first search algorithm, and each clustering result is obtained. The instance mask corresponding to the mask.

[0039] The specific clustering method can refer to the method disclosed in the document "Vu T, Kim K, Luu T M, et al. Softgroup for 3d instance segmentation on point clouds [C] / / Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition. 2022: 2708-2717."

[0040] A2, according to the instance mask corresponding to each clustering result and the predetermined average number of voxels corresponding to different tree species single tree instances, the single tree instance included in the forest point cloud to be segmented is determined, and the rough instance mask corresponding to each single tree instance is determined.

[0041] In a possible implementation, determining the average voxel number corresponding to single-tree instances of different tree species includes B1-B3:

[0042] B1, voxelizing each of the labeled forest point cloud samples to obtain voxels corresponding to trees in each of the forest point cloud samples.

[0043] B2, counting the total number of single-tree instances of each tree species and the number of voxels corresponding to single-tree instances of each tree species in the voxels corresponding to the trees.

[0044] B3, determining the average voxel number corresponding to single-tree instances of different tree species according to the total number of single-tree instances of each tree species and the number of voxels corresponding to single-tree instances of each tree species.

[0045] S105, determining the coordinates and features of the initial query vector corresponding to each single-tree instance according to the coarse instance mask corresponding to each single-tree instance in the forest point cloud to be segmented.

[0046] In a possible implementation, determining the coordinates and features of the initial query vector corresponding to each single-tree instance according to the coarse instance mask corresponding to each single-tree instance in the forest point cloud to be segmented includes steps C1 and C2:

[0047] C1, determining the coordinates and features of the voxels corresponding to each single-tree instance in the forest point cloud to be segmented in the offset space according to the coarse instance mask corresponding to each single-tree instance in the forest point cloud to be segmented.

[0048] C2, performing average pooling processing on the coordinates and features of the voxels corresponding to each single-tree instance in the forest point cloud to be segmented in the offset space, respectively, to obtain the coordinates and features of the initial query vector corresponding to each single-tree instance.

[0049] For example, the offset P of the instance corresponding voxel is found according to the correspondence between each instance mask and voxel l The average pooling is performed on each instance corresponding voxel to obtain the initial query vector coordinates And the query vector features Wherein And represent the coordinates and features of the nth query vector.

[0050] S106, performing over-segmentation on the forest point cloud to be segmented to obtain super-points corresponding to the forest point cloud to be segmented, and determining the super-point features and super-point space coordinates.

[0051] The super-point can be regarded as a constant component in the geometric and radiometric point feature surface segmentation constant approximation, and the super-point index can be obtained by over-segmentation.

[0052] The over-segmentation method can refer to the point cloud over-segmentation method disclosed in the literature "Robert D, Raguet H, Landrieu L. Efficient 3D semantic segmentation with superpoint transformer [C] / / Proceedings of the IEEE / CVF International Conference on Computer Vision. 2023: 17195-17204."

[0053] For example, the process of point cloud over-segmentation computing superpoints usually involves the following key steps:

[0054] 1. Preprocessing: denoising the original point cloud to reduce the interference of outliers in subsequent algorithms.

[0055] 2. Voxelization: dividing the entire point cloud space into voxels according to a voxel size, generating a voxel cloud. Each voxel may contain multiple original point cloud data. The resolution of voxelization will affect the shape and size of the subsequent superpoints.

[0056] 3. Establish adjacency graph: in the voxel space, establish an adjacency graph according to the spatial relationship between voxels, usually considering 26-neighborhood.

[0057] 4. Superpoint growth: starting from the seed voxel, use a flow-constrained clustering algorithm to traverse adjacent voxels layer by layer, and calculate the similarity distance between adjacent voxels and the current superpoint. According to the preset threshold, voxels are clustered into uniform smooth superpoints adhering to the target boundary.

[0058] 5. Superpoint feature extraction: extract geometric features such as normal vector and centroid for each superpoint. Measure the similarity between adjacent superpoints by normalizing the distance value.

[0059] 6. Superpoint graph construction: construct a superpoint graph, where nodes represent superpoints and edges represent the adjacency relationship between superpoints.

[0060] 7. Post-processing: for over-segmented point clouds, calculate the concave-convex relationship between different superpoints to merge adjacent superpoints. Use CC (Convexity Criterion) and SC (Sanity Criterion) and other criteria to evaluate the geometric relationship between superpoints.

[0061] This process can effectively segment large-scale point cloud data into meaningful superpoints, providing preprocessing for subsequent semantic segmentation, object recognition, and other tasks. Through this method, the processing efficiency can be improved while maintaining the details of the point cloud.

[0062] In a possible implementation, the determining of the super point feature and the super point spatial coordinate comprises: extracting features and spatial coordinates of voxels corresponding to each super point; and performing average pooling processing on the features and the spatial coordinates of the voxels corresponding to each super point respectively to obtain the super point feature and the super point spatial coordinate.

[0063] S107, based on the pre-trained Transformer Decoder model, obtaining, according to the super point feature, the super point spatial coordinate and each initial query vector, a segmentation result corresponding to the forest point cloud to be subjected to single tree instance segmentation.

[0064] In a possible implementation, based on the pre-trained Transformer Decoder model, obtaining, according to the super point feature, the super point spatial coordinate and each initial query vector, a segmentation result corresponding to the forest point cloud to be subjected to single tree instance segmentation, comprises D1-D4.

[0065] D1, through the pre-trained Transformer Decoder model, optimizing features of each initial query vector according to the super point spatial coordinate to obtain a target query vector corresponding to each initial query vector, and predicting a semantic label and an offset of each target query vector, the semantic label being used to identify a tree species.

[0066] D2, respectively determining dot product similarities of each target query vector and the super point feature.

[0067] D3, according to the dot product similarity, determining an instance binary mask corresponding to each target query vector.

[0068] D4, according to the instance binary mask corresponding to each target query vector and the semantic label, obtaining the segmentation result corresponding to the forest point cloud to be subjected to single tree instance segmentation.

[0069] Optionally, based on the pre-trained Transformer Decoder model, obtaining, according to the super point feature, the super point spatial coordinate and each initial query vector, a segmentation result corresponding to the forest point cloud to be subjected to single tree instance segmentation, is represented as:

[0070] Q, K, V = MLP(Q f , Q f , Q f ),

[0071] SP' = SP f + embedding(SP l ),

[0072]

[0073] Q out = FFN(Q"),

[0074] Q offset ,Q label =MLP(Q out ),

[0075] Q' c =Q c +Q offset ,

[0076] Predscore=(Q out ×Q offset )Sigmoid,

[0077] Predmask=(Predscore>0.5),

[0078] wherein Q represents a Transformer self-attention Query vector, K represents a Transformer self-attention Key vector, V represents a Transformer self-attention Value vector, MLP() represents a multi-layer perceptron, Q f represents a query vector feature, SP' represents a super point fusion feature, SP f represents a super point feature, embedding() represents a Fourier encoding, SP l represents a super point coordinate, Q' represents a Transformer cross-attention Query vector, Self-Attention() represents self-attention, Cross-Attention() represents cross-attention, softmax() represents a normalized exponential function, d k represents a number of attention channels, a superscript T represents a transpose, Q'' represents an optimized query vector of cross-attention, Q out represents an output of an optimized query vector of feedforward neural network FFN(), FFN() represents a feedforward neural network, Q offset represents an offset of a query vector, Q label represents a semantic label of a query vector, Q' c represents a coordinate of a query vector in an offset space, Q c represents an initial original coordinate of a query vector, Predscore represents a dot product similarity, Predmask represents an instance binary mask, and Sigmoid represents a sigmoid function.

[0079] By calculating the dot product similarity between the super point feature and the query vector, the single tree instance can be accurately identified and segmented, and the accuracy of instance segmentation is improved.

[0080] Optionally, the method further comprises, before S101, dividing the forest into a plurality of plots, and obtaining laser radar point clouds corresponding to each plot.

[0081] Correspondingly, steps S101 to S107 are performed for each plot, and then all plots are traversed to complete the single tree instance segmentation of the entire forest.

[0082] Optionally, the method further comprises: using a mean average precision (mAP) as a general evaluation index of instance segmentation, and setting an average IoU threshold to be from 50% to 95% with a step of 5%.

[0083] Using the mean average precision as the evaluation index can comprehensively and accurately evaluate the model performance under various IoU thresholds.

[0084] The large-scale forest single tree instance segmentation method based on a laser radar point cloud provided by the application comprises the following steps: voxelizing a forest point cloud to be subjected to single tree instance segmentation to obtain a plurality of target voxels; inputting the plurality of target voxels into a pre-trained sparse convolutional neural network backbone model to obtain voxel features, offsets and semantic labels of each target voxel corresponding to the forest point cloud to be subjected to single tree instance segmentation, the semantic labels being used to identify tree species; constructing an offset space according to the offsets of the target voxels; clustering the target voxels in the offset space through a sphere query and a breadth-first search algorithm, and determining a rough instance mask corresponding to each single tree instance in the forest point cloud to be subjected to single tree instance segmentation according to a pre-determined average number of voxels corresponding to different single tree instances of tree species; determining the coordinates and features of an initial query vector corresponding to each single tree instance according to the rough instance mask corresponding to each single tree instance in the forest point cloud to be subjected to single tree instance segmentation; performing over-segmentation processing on the forest point cloud to be subjected to single tree instance segmentation to obtain super points corresponding to the forest point cloud to be subjected to single tree instance segmentation, and determining super point features and super point space coordinates; and obtaining a segmentation result corresponding to the forest point cloud to be subjected to single tree instance segmentation based on a pre-trained Transformer Decoder model according to the super point features, the super point space coordinates and the initial query vectors.

[0085] In order to further prove the beneficial effects of the application, a set of experimental data is provided as follows:

[0086] 1. Simulation conditions

[0087] The present application is based on an Inter(R) Core(TM) i7-13700KF 5.4GHz CPU, an NVIDIA A6000 GPU, an Ubuntu 20.04 operating system, a deep learning framework pytorch version 1.13, a CUDA version 11.8, and a python version 3.8.

[0088] The method compared in the experiment is as follows:

[0089] One is a clustering model, referred to as Softgroup in the experiment, and the reference is Vu T, Kim K, Luu T M, et al. Softgroup for 3d instance segmentation on point clouds [C] / / Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition. 2022: 2708-2717.

[0090] The other is a super point model, referred to as SPFormer in the experiment, and the reference is Sun J, Qing C, Tan J, et al. Superpoint transformer for 3d scene instance segmentation [C] / / Proceedings of the AAAI Conference on Artificial Intelligence. 2023, 37(2): 2393-2401.

[0091] 2. Simulation content

[0092] According to the specific embodiment of the present application, the instance segmentation accuracy is calculated and compared with the accuracy, inference time and video memory consumption of the Softgroup method and the SPFormer method, and the results are shown in Table 1.

[0093] Table 1 Forest single tree instance segmentation accuracy, inference time and video memory consumption

[0094] Method Softgroup SPFormer The invention mAP 50 (%)]] 21.61 24.39 45.73 Inference time (it / s) 89.2 20.4 38.1 Video memory consumption (GB) 24.7 46.3 37.8

[0095] As can be seen from Table 1, due to the use of the present application based on clustering initialization query vector and then refined by the Transformer decoder, higher recognition effect can be achieved, verifying the advancement of the present application.

[0096] The application further provides a structure of an electronic device, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory complete communication with each other through the communication bus,

[0097] The memory is used for storing a computer program.

[0098] The processor is used for executing the program stored on the memory to realize the steps provided in the method embodiments.

[0099] The communication interface is used for communication between the electronic device and other devices.

[0100] The method provided by the embodiments of the application can be applied to an electronic device. Specifically, the electronic device can be a desktop computer, a portable computer, a smart mobile terminal, a server and the like. Herein, no limitation is made, and any electronic device capable of realizing the application falls within the protection scope of the application.

[0101] The application further provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program is executed by a processor to realize the steps provided in the method embodiments.

[0102] For the electronic device / storage medium embodiments, the description is relatively simple because they are basically similar to the method embodiments, and the specific contents, beneficial effects and the like are described in the method embodiments.

[0103] The terms "first", "second" are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined with "first", "second" can explicitly or implicitly include one or more of the features. In the description of the application, the meaning of "a plurality of" is two or more, unless otherwise specifically limited.

[0104] The above is a further detailed description of the application in combination with specific preferred embodiments, and the specific implementation of the application cannot be limited to these descriptions. For ordinary skilled persons in the art to which the application belongs, some simple deductions or replacements can be made without departing from the concept of the application, and all of them should be regarded as falling within the protection scope of the application.

Claims

1. A large-scale forest single tree instance segmentation method based on laser radar point cloud, characterized in that, The method comprises the following steps: voxelizing a forest point cloud to be subjected to single-tree instance segmentation to obtain a plurality of target voxels; inputting the plurality of target voxels into a pre-trained sparse convolutional neural network backbone model to obtain voxel features, offsets and semantic labels of each target voxel corresponding to the forest point cloud to be subjected to single-tree instance segmentation, wherein the semantic labels are used to identify tree species; constructing an offset space according to the offsets of each target voxel; in the offset space, clustering the target voxels by means of a sphere query and a breadth-first search algorithm, and determining a coarse instance mask corresponding to each single-tree instance in the forest point cloud to be subjected to single-tree instance segmentation according to a pre-determined average number of voxels corresponding to single-tree instances of different tree species; determining the coordinates and features of an initial query vector corresponding to each single-tree instance according to the coarse instance mask corresponding to each single-tree instance in the forest point cloud to be subjected to single-tree instance segmentation; performing over-segmentation on the forest point cloud to be subjected to single-tree instance segmentation to obtain hyperpoints corresponding to the forest point cloud to be subjected to single-tree instance segmentation, and determining hyperpoint features and hyperpoint space coordinates; obtaining a segmentation result corresponding to the forest point cloud to be subjected to single-tree instance segmentation based on a pre-trained Transformer Decoder model according to the hyperpoint features, the hyperpoint space coordinates and each initial query vector.

2. The method of claim 1, wherein, The method for determining the average number of voxels corresponding to single-tree instances of different tree species comprises the following steps: voxelizing a plurality of labeled forest point cloud samples to obtain voxels corresponding to trees in each forest point cloud sample; counting the total number of single-tree instances of each tree species and the number of voxels corresponding to single-tree instances of each tree species in the voxels corresponding to the trees; determining the average number of voxels corresponding to single-tree instances of different tree species according to the total number of single-tree instances of each tree species and the number of voxels corresponding to single-tree instances of each tree species.

3. The method of claim 1, wherein, The method for determining the average number of voxels corresponding to single-tree instances of different tree species according to the total number of single-tree instances of each tree species and the number of voxels corresponding to single-tree instances of each tree species comprises the following steps: in the offset space, clustering the target voxels by means of a sphere query and a breadth-first search algorithm, and determining a coarse instance mask corresponding to each single-tree instance in the forest point cloud to be subjected to single-tree instance segmentation according to a pre-determined average number of voxels corresponding to single-tree instances of different tree species; in the offset space, clustering the target voxels by means of a sphere query and a breadth-first search algorithm to obtain an instance mask corresponding to each clustering result; 4. The method of claim 1, wherein, determining single-tree instances included in the forest point cloud to be subjected to single-tree instance segmentation and a coarse instance mask corresponding to each single-tree instance according to the instance mask corresponding to each clustering result and the pre-determined average number of voxels corresponding to single-tree instances of different tree species. The method for constructing an offset space according to the offsets of each target voxel comprises the following steps:

5. The method of claim 1, wherein, constructing an offset space according to the coordinates of each target voxel obtained by voxelization and the offsets of each target voxel. The method for obtaining a segmentation result corresponding to the forest point cloud to be subjected to single-tree instance segmentation based on a pre-trained Transformer Decoder model according to the hyperpoint features, the hyperpoint space coordinates and each initial query vector comprises the following steps: The Transformer Decoder model is pre-trained, and the features of the initial query vectors are optimized according to the hyperpoint space coordinates to obtain target query vectors corresponding to the initial query vectors, and the semantic labels and offsets of the target query vectors are predicted. The dot product similarity of each target query vector and the hyperpoint feature is determined respectively. According to the dot product similarity, the instance binary mask corresponding to each target query vector is determined. According to the instance binary mask and the semantic label corresponding to each target query vector, the segmentation result corresponding to the forest point cloud to be segmented into single tree instances is obtained.

6. The method of claim 1, wherein, The coordinates and features of the initial query vectors corresponding to each single tree instance are determined according to the coarse instance mask corresponding to each single tree instance in the forest point cloud to be segmented into single tree instances, including: The coordinates and features of the voxels corresponding to each single tree instance in the forest point cloud to be segmented into single tree instances in the offset space are determined according to the coarse instance mask corresponding to each single tree instance in the forest point cloud to be segmented into single tree instances. The coordinates and features of the voxels corresponding to each single tree instance in the forest point cloud to be segmented into single tree instances in the offset space are averaged and pooled to obtain the coordinates and features of the initial query vectors corresponding to each single tree instance.

7. The method of claim 5, wherein, The Transformer Decoder model is pre-trained, and the segmentation result corresponding to the forest point cloud to be segmented into single tree instances is obtained according to the hyperpoint feature, the hyperpoint space coordinates and the initial query vectors, represented as: Q, K, V = MLP(Q f Q f Q f ), SP' = SP f + embedding(SP l ), Q out = FFN(Q"), Q offset ,Q label = MLP(Q out ), Q' c = Q c + Q offset , Predscore = (Q out x Q offset ) Sigmoid, Predmask = (Predscore > 0.5), wherein Q denotes a Transformer self-attention Query vector, K denotes a Transformer self-attention Key vector, V denotes a Transformer self-attention Value vector, MLP() denotes a multi-layer perceptron, Q f denotes a query vector feature, SP' denotes a super point fusion feature, SP f denotes a super point feature, embedding() denotes a Fourier encoding, SP l denotes a super point coordinate, Q' denotes a Transformer cross-attention Query vector, Self-Attention() denotes a self-attention, Cross-Attention() denotes a cross-attention, softmax() denotes a normalized exponential function, d k denotes a number of attention channels, superscript T denotes a transpose, Q" denotes an optimized Query vector of cross-attention, Q out denotes an output of an optimized Query vector of feed-forward neural network FFN(), FFN() denotes a feed-forward neural network, Q offset denotes an offset of a Query vector, Q label denotes a semantic label of a Query vector, Q' c denotes a coordinate of a Query vector in an offset space, Q c denotes an initial original coordinate of a Query vector, Predscore denotes a dot product similarity, Predmask denotes an instance binary mask, Sigmoid denotes a sigmoid function.

8. The method of claim 1, wherein, The hyperpoint feature and the hyperpoint space coordinates are determined, including: The features and spatial coordinates of the voxels corresponding to each hyperpoint are extracted. The features and spatial coordinates of the voxels corresponding to each hyperpoint are averaged and pooled to obtain the hyperpoint feature and the hyperpoint space coordinates.

9. An electronic device, comprising: The processor, the communication interface, the memory and the communication bus are connected through the communication bus; The memory is used to store the computer program; The processor is used to execute the program stored in the memory to realize the method of any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, The computer program is stored in the computer readable storage medium, and the processor executes the computer program to realize the method of any one of claims 1-8. The computer program is stored in the computer readable storage medium, and the processor executes the computer program to realize the method of any one of claims 1-8.

Images