Eucalyptus single knot segmentation method, device, system, and storage medium

Abstract

The present invention discloses a method, device, system, and storage medium for eucalyptus tree segmentation, comprising: step S1, acquiring historical UAV data of eucalyptus trees; step S2, training an improved Mask R-CNN model based on the historical UAV data to generate a tree segmentation model; and step S3, inputting the UAV data of eucalyptus trees in a target area into the tree segmentation model to extract the number of individual eucalyptus trees and their crown area. The technical solution of the present invention improves recognition accuracy in complex scenarios and improves transferability across different scenarios.

Description

Technical Field

[0001] The present invention belongs to the technical field of data processing, and in particular relates to a method and device, a system, and a storage medium for splitting single eucalyptus trees. Background Art

[0002] Eucalyptus plantations, known as one of the world's three fastest-growing plantations, along with pine and poplar, are widely planted in tropical and subtropical regions. Not only do they possess excellent wood properties, making them the most important source of short fibers for pulp and paper production, they also play a vital role in ecological security, regulating global climate change, and ensuring national timber security. Traditional eucalyptus resource surveys are often obtained through field measurements. While accurate, these results are time-consuming, labor-intensive, costly, and limited in scope. The data update frequency is low, making it difficult to meet the data needs for monitoring the dynamic changes in short-rotation eucalyptus plantations. The successful application of remote sensing technology has, to a certain extent, made up for the shortcomings of traditional surveys, not only saving data survey costs but also significantly reducing survey time and increasing data update frequency.

[0003] Drones, with their advantages of low flight altitude, flexible operation, and low cost, have become a primary means of acquiring high-resolution imagery, laying the foundation for extracting structural parameters of individual trees in eucalyptus plantations. Tree crown segmentation is crucial for extracting structural parameters of individual trees from high-resolution remote sensing imagery. Deep learning algorithms can learn and adaptively extract features from large amounts of data, making them suitable for processing a wide range of data types. They are not only highly adaptable to different types of landforms and vegetation, but can also learn complex features and contextual information. They can also automatically learn patterns and regularities in the data, eliminating the need for manual intervention and saving significant labor costs and time. They are widely used for individual tree segmentation in forests and achieve excellent results. Currently, there are two main categories of commonly used deep learning algorithms for individual tree segmentation. One is semantic segmentation, which aims to label each pixel in an image with a corresponding semantic category, thereby achieving pixel-level understanding and analysis. Representative algorithms include DeepLabv3+, PSPNet, and UNet. Because these algorithms focus on the content of each location in the image rather than the boundaries of individual objects, semantic segmentation results are prone to artifacts, making complete segmentation difficult. The other type is instance segmentation, which achieves individual tree segmentation by accurately segmenting each object instance and assigning a unique identifier to each instance. Representative algorithms include the Mask R-CNN algorithm and the Yolact algorithm. Unlike semantic segmentation algorithms, instance segmentation algorithms focus not only on the object categories in the image but also on distinguishing between different instances of the same category, making individual tree segmentation less prone to overlap. The Mask R-CNN algorithm, a representative example of instance segmentation, can segment images at the pixel level, providing accurate segmentation results that include object boundaries and shape information. It can also be applied to objects of varying sizes and proportions, making it suitable for segmentation tasks in a variety of scenarios and environments. It has been widely used for individual tree segmentation. For example, Zhenbang Hao et al. used the Mask R-CNN algorithm to detect individual tree crowns in a young Chinese fir forest based on drone imagery. The results achieved good individual tree segmentation results, with an accuracy of 84.68%. However, further verification is needed to determine the performance of this algorithm in individual tree segmentation for forests of other age groups. Hancong Fu et al. used the Mask R-CNN algorithm to detect the crowns of Pinus sylvestris var. mongolica. Experimental results showed that this method achieved a maximum crown extraction accuracy of 89.6%, but its application in high-density forests still presents challenges. Robiah Hamzah et al. used the Mask R-CNN algorithm to segment and identify trees in tropical rainforests, achieving an accuracy of 75%. However, the segmentation results showed that many trees were not identified. Although the Mask R-CNN algorithm can accurately segment individual trees, its recognition accuracy is low in complex scenes, resulting in poor segmentation results and difficulty in transferring to other environments. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a method and device, system and storage medium for segmenting single eucalyptus trees, so as to improve the recognition accuracy in complex scenes and the migration adaptability in different scenes.

[0005] To achieve the above object, the present invention adopts the following technical solutions:

[0006] A method for splitting a single eucalyptus tree, comprising:

[0007] Step S1, obtaining historical eucalyptus UAV data;

[0008] Step S2: training an improved Mask R-CNN model based on historical eucalyptus UAV data to obtain a single tree segmentation model; wherein the improved Mask R-CNN model includes: sequentially connecting a backbone network, a feature pyramid network FPN, a region proposal network RPN, a RoI Align layer, a classifier module, and a mask module; wherein three enhanced selective kernel convolution modules ESK are set in the feature pyramid network FPN, and each enhanced selective kernel convolution module ESK includes: a channel attention unit, a spatial attention unit, and a multi-scale convolution unit;

[0009] Step S3: Input the UAV data of eucalyptus trees in the target area into the single tree segmentation model to extract the number of single eucalyptus trees and the crown area of single trees.

[0010] Preferably, step S1 further includes preprocessing the historical eucalyptus UAV data, which includes:

[0011] Trimming of historical eucalyptus UAV data;

[0012] The eucalyptus crown edge contours were marked based on the clipped historical eucalyptus UAV data.

[0013] The present invention also provides a eucalyptus single wood splitting device, comprising:

[0014] Acquisition module, used to obtain historical eucalyptus UAV data;

[0015] A training module is used to train an improved Mask R-CNN model based on historical eucalyptus UAV data to obtain a single tree segmentation model. The improved Mask R-CNN model includes: a backbone network, a feature pyramid network (FPN), a region proposal network (RPN), a RoI alignment layer, a classifier module, and a mask module connected in sequence. Three enhanced selective kernel convolution modules (ESK) are set in the feature pyramid network (FPN), and each enhanced selective kernel convolution module (ESK) includes: a channel attention unit, a spatial attention unit, and a multi-scale convolution unit.

[0016] The extraction module is used to input the UAV data of eucalyptus trees in the target area into the single tree segmentation model to extract the number of single eucalyptus trees and the crown area of single trees.

[0017] Preferably, a pre-processing module is further included for pre-processing the historical eucalyptus UAV data, which includes:

[0018] A clipping unit, used to clip the historical eucalyptus UAV data;

[0019] The annotation unit is used to annotate the edge contours of eucalyptus crowns based on the clipped historical eucalyptus UAV data.

[0020] The present invention also provides a eucalyptus single tree segmentation system, comprising: a memory and a processor, wherein the memory stores a computer program executed by the processor, and the computer program executes a eucalyptus single tree segmentation method when executed by the processor.

[0021] The present invention also provides a storage medium, on which a computer program is stored, and the computer program executes the eucalyptus single tree segmentation method when running.

[0022] This paper adds three enhanced selective kernel convolution modules (ESK) to the Mask R-CNN model's FPN network, enhancing focus on key semantic features of eucalyptus trees. Furthermore, the feature pyramid network (FPN) outputs three effective feature layers, rather than five layers of varying sizes, reducing the number of network model parameters. This technical solution improves recognition accuracy in complex scenes and improves transfer adaptability across diverse scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0024] Figure 1 This is a flow chart of a method for segmenting a single eucalyptus tree according to an embodiment of the present invention;

[0025] Figure 2 This is the structural diagram of the feature pyramid network FPN in the improved Mask R-CNN model. DETAILED DESCRIPTION

[0026] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0027] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0028] Example 1:

[0029] like Figure 1 As shown, an embodiment of the present invention provides a method for segmenting a single eucalyptus tree, comprising:

[0030] Step S1, obtaining historical eucalyptus UAV data;

[0031] Step S2: training an improved Mask R-CNN model based on historical eucalyptus UAV data to obtain a single tree segmentation model;

[0032] Step S3: Input the UAV data of eucalyptus trees in the target area into the single tree segmentation model to extract the number of single eucalyptus trees and the crown area of single trees.

[0033] As an implementation method of the embodiment of the present invention, step S1 further includes preprocessing the historical eucalyptus UAV data, which includes:

[0034] Trimming of historical eucalyptus UAV data;

[0035] The eucalyptus crown edge contours were marked based on the clipped historical eucalyptus UAV data.

[0036] As an implementation method of an embodiment of the present invention, the improved Mask R-CNN model includes: a backbone network, a feature pyramid network FPN, a region proposal network RPN, a RoI Align layer, a classifier module, and a mask module connected in sequence; wherein, three enhanced selection kernel convolution modules ESK are set in the feature pyramid network FPN, and each enhanced selection kernel convolution module ESK includes: a channel attention unit, a spatial attention unit and a multi-scale convolution unit.

[0037] like Figure 2 As shown, in step S2, the backbone network extracts features from the historical eucalyptus UAV data to generate feature layers of different sizes from C2 to C5. After the feature layers of C2 to C5 undergo a Conv2D convolution with a convolution kernel size of 1X1, the feature layers of C2 to C4 are input into the ESK feature module for enhanced feature extraction. The extracted features are jump-connected with the features after the previous convolution, and the corresponding feature layers P2 to P5 are obtained after upsampling. P2, P3 and P4 are taken as effective feature layers; wherein; the ESK module has different channel attention units, spatial attention units and multi-scale convolutions with different kernel sizes; the channel attention module and spatial attention module in ESK can calibrate the target image features from the spatial and channel dimensions at the same time; the multi-convolution layers with different kernel sizes in ESK can provide receptive fields of different scales, thereby improving the adaptability of the network to different input images and improving the target detection accuracy. The RPN network is used to predict the P2 to P4 feature layers to generate candidate regions in the image that may contain the target, and generate region proposal boxes. The region proposal boxes are filtered, and the filtered regions are feature aligned in the RoI Align layer to generate features corresponding to the candidate regions and map them back to the feature map. The feature results obtained after the RoI Align operation are resized. In the classifier module, the clipped content is resized to 7x7x256, and then a convolution operation is performed to decode the proposal box to obtain the final prediction box. In the mask module, the clipped content is resized to 14x14x256, and a convolution operation is also performed to obtain the semantic segmentation result within the prediction box. This completes the training of the improved Mask R-CNN model and obtains the single tree segmentation model.

[0038] Furthermore, the process of training the improved Mask R-CNN model includes:

[0039] 1. The historical eucalyptus UAV data was input into the Restnet101 backbone network for feature extraction, and four feature layers of different sizes were obtained, namely C2, C3, C4, and C5. These feature layers were then convolved with a kernel size of 1x1 and a channel number of 256.

[0040] 2. The convolutional C2 to C4 feature layers are introduced into the ESK module, and feature extraction is performed in the channel attention unit and spatial attention unit respectively. The feature channels and spatial positions that are more important to the current task are selectively focused on from the channel dimension and spatial dimension, focusing on the important feature information of the image; the extracted features are then concatenated with the original features to obtain the total feature output.

[0041] 3. After the features extracted by the ESK module are concatenated with the features obtained by the previous convolution, they are upsampled to obtain the P2, P3, P4, and P5 feature layers respectively.

[0042] 4. Given the dense eucalyptus plantations and the small size of their crowns in the image, the original feature pyramid FPN outputs five feature layers of varying sizes, replacing them with three valid feature layers. Since the corresponding prior frames for layers P2 through P5 increase in size, the P2, P3, and P4 feature layers are used. Accordingly, the RPN prior frame lengths are changed from [32, 64, 128, 256, 512] to [32, 64, 128], corresponding to the P2, P3, and P4 feature layers, respectively.

[0043] 5. The obtained P2, P3, and P4 feature layers are input into the RPN proposal network to obtain the prior frame adjustment parameters and detect whether the prior frame contains an object. Specifically, the input feature layer first undergoes a 3x3 convolution with a channel count of 512. Then, based on the RPN prior frame parameters [32, 64, 128], the input image is divided into grids of different sizes: 128x128x256, 64x64x256, and 32x32x256. Each grid has three prior frames by default. Two more convolutions are then performed to determine whether the prior frame contains an object and adjust it, respectively, to obtain a new frame, the proposal frame.

[0044] 6. After further adjustment and decoding, the obtained suggestion box is further screened through non-maximum suppression to select the suggestion box with the highest confidence.

[0045] 7. In the RoI Align layer, the suggestion box is used to intercept the shared feature layer (P2, P3, P4), and the feature layer is aligned with the suggestion box. The size of the suggestion box is used to find which feature layer the suggestion box belongs to, and the features corresponding to the candidate region are generated and mapped back to the feature map.

[0046] 8. In the classifier module, a 7x7 convolution with a channel number of 1024 and a 1x1 convolution with a channel number of 1024 are used to convolve the 7x7x256 area obtained by ROI Align. The two convolutions with a channel number of 1024 are used to simulate two 1024 full connections. Then, full connections are performed separately to obtain the objects in the suggestion box and the adjustment parameters of the suggestion box.

[0047] 9. In the mask module, the resized local feature layer is first subjected to four 3x3 256-channel convolutions, followed by one deconvolution, and then another convolution with the number of channels equal to the number of types of objects being inspected, denoted as n. The final result represents the class of each pixel; the final shape is 28x28xn, representing the category of each pixel.

[0048] 10. The prediction result of the classifier module is the type of object inside the proposal box and the position of the prediction box. The result of the adjusted proposal box is the final prediction box. The decoding process of the prediction box includes: 1) extracting the proposal box that does not belong to the background and has a score greater than 0.7; 2) decoding using the proposal box and the prediction result of the classifier module to obtain the position of the final prediction box; 3) using the score and the position of the final prediction box to perform non-maximum suppression to prevent duplicate detection.

[0049] 11. Use the final prediction box as the area clipping part of the mask module, and use the final prediction box to clip the common feature layer used in the mask module; after clipping, use the mask module to classify the pixels and obtain the semantic segmentation result.

[0050] 12. The historical eucalyptus UAV data was trained on the improved Mask R-CNN model with a batch size of 1, an epoch size of 300, an Adam optimization method, a momentum parameter of 0.9, a weight decay parameter of 0, and an initial learning rate of 0.0001.

[0051] Example 2:

[0052] The embodiment of the present invention further provides a eucalyptus single wood splitting device, comprising:

[0053] Acquisition module, used to obtain historical eucalyptus UAV data;

[0054] A training module is used to train an improved Mask R-CNN model based on historical eucalyptus UAV data to obtain a single tree segmentation model. The improved Mask R-CNN model includes: a backbone network, a feature pyramid network (FPN), a region proposal network (RPN), a RoI alignment layer, a classifier module, and a mask module connected in sequence. Three enhanced selective kernel convolution modules (ESK) are set in the feature pyramid network (FPN), and each enhanced selective kernel convolution module (ESK) includes: a channel attention unit, a spatial attention unit, and a multi-scale convolution unit.

[0055] The extraction module is used to input the UAV data of eucalyptus trees in the target area into the single tree segmentation model to extract the number of single eucalyptus trees and the crown area of single trees.

[0056] As an implementation manner of an embodiment of the present invention, a pre-processing module is further included, which is used to pre-process the historical eucalyptus UAV data, including:

[0057] A clipping unit, used to clip the historical eucalyptus UAV data;

[0058] The annotation unit is used to annotate the edge contours of eucalyptus crowns based on the clipped historical eucalyptus UAV data.

[0059] Example 3:

[0060] An embodiment of the present invention further provides a eucalyptus tree segmentation system, comprising: a memory and a processor, wherein the memory stores a computer program executed by the processor, and the computer program executes a eucalyptus tree segmentation method when executed by the processor.

[0061] Example 4:

[0062] An embodiment of the present invention further provides a storage medium having a computer program stored thereon, and the computer program executes a method for segmenting a single eucalyptus tree when running.

[0063] The embodiments described above are merely descriptions of preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by persons skilled in the art should fall within the scope of protection defined by the claims of the present invention.

Claims

1. A method for splitting a single eucalyptus tree, characterized in that: include: Step S1, obtaining historical eucalyptus UAV data; Step S2: training an improved Mask R-CNN model based on historical eucalyptus UAV data to obtain a single tree segmentation model; wherein the improved Mask R-CNN model includes: sequentially connecting a backbone network, a feature pyramid network FPN, a region proposal network RPN, a RoIAlign layer, a classifier module, and a mask module; wherein three enhanced selective kernel convolution modules ESK are set in the feature pyramid network FPN, and each enhanced selective kernel convolution module ESK includes: a channel attention unit, a spatial attention unit, and a multi-scale convolution unit; Step S3: inputting the UAV data of eucalyptus trees in the target area into the single tree segmentation model to extract the number of single eucalyptus trees and the crown area of single trees; The process of training the improved Mask R-CNN model includes: The historical eucalyptus UAV data was input into the Restnet101 backbone network for feature extraction, and four feature layers of different sizes were obtained, namely C2, C3, C4, and C5. Then, a convolution operation with a kernel size of 1x1 and a channel number of 256 was performed on each feature layer. The convolutional C2 to C4 feature layers are introduced into the ESK module, and feature extraction is performed in the channel attention unit and spatial attention unit respectively. The feature channels and spatial positions that are more important to the current task are selectively focused on from the channel dimension and spatial dimension. The extracted features are then concatenated with the original features to obtain the total feature output. The features extracted by the ESK module are concatenated with the features obtained by the previous convolution, and then upsampled to obtain the P2, P3, P4, and P5 feature layers respectively; Based on the characteristics of dense eucalyptus planting and its crown as a small target in the image, the original feature pyramid FPN outputs 5 feature layers of different sizes, which are changed to output 3 effective feature layers. Since the corresponding prior frames from P2 to P5 are getting larger and larger, the P2, P3, and P4 feature layers are taken, and the parameters of the RPN prior frame length [32, 64, 128, 256, 512] are changed to [32, 64, 128], corresponding to the P4, P3, and P2 feature layers respectively; The obtained P4, P3, and P2 feature layers are input into the RPN proposal box network to obtain the prior box adjustment parameters and detect whether the prior box contains an object. The specific operation is as follows: the input feature layer is first subjected to a 3x3 convolution with a channel number of 512. Then, according to the parameters of the RPN prior box [32, 64, 128], the input image is divided into grids of different sizes: 32x32x256, 64x64x256, and 128x128x256. Each grid has three prior boxes by default. Then, two convolutions are performed to determine whether the prior box contains an object and adjust it to obtain a new box, which is defined as the proposal box. After further adjustment and decoding, the obtained suggestion boxes are further screened through non-maximum suppression to select the suggestion box with the highest confidence; In the RoIAlign layer, the shared feature layer (P2, P3, and P4) is intercepted using the suggestion box, and the feature layer is aligned with the suggestion box. The feature layer to which the suggestion box belongs is found by the size of the suggestion box, and the features corresponding to the candidate region are generated and mapped back to the feature map. In the classifier module, a 7x7 convolution with a channel number of 1024 and a 1x1 convolution with a channel number of 1024 are used to convolve the 7x7x256 area obtained by ROIAlign. Two convolutions with a channel number of 1024 are used to simulate two 1024 full connections. Then, full connections are performed separately to obtain the objects in the suggestion box and the adjustment parameters of the suggestion box. In the mask module, the resized local feature layer is first subjected to four 3x3 256-channel convolutions, followed by one deconvolution, and then another convolution with the number of channels equal to the number of types of objects being inspected, denoted as n. The final result represents the class of each pixel; the final shape is 28x28xn, representing the category of each pixel. The prediction result based on the classifier module is the type of object inside the proposal box and the position of the prediction box. The result after the proposal box is adjusted is the final prediction box. The decoding process of the prediction box includes: 1) extracting the proposal box that does not belong to the background and has a score greater than 0.7; 2) decoding using the proposal box and the prediction result of the classifier module to obtain the position of the final prediction box; 3) using the score and the position of the final prediction box to perform non-maximum suppression to prevent duplicate detection; The final prediction box is used as the area interception part of the mask module, and the common feature layer used in the mask module is intercepted using the final prediction box. After interception, the mask module is used to classify the pixels to obtain the semantic segmentation result. The historical eucalyptus UAV data was trained on the improved Mask R-CNN model.

2. The method for splitting a single eucalyptus tree according to claim 1, wherein: Step S1 also includes pre-processing the historical eucalyptus UAV data, which includes: Trimming of historical eucalyptus UAV data; The eucalyptus crown edge contours were marked based on the clipped historical eucalyptus UAV data.

3. A eucalyptus single wood splitting device, using the method according to any one of claims 1-2, characterized in that: include: Acquisition module, used to obtain historical eucalyptus UAV data; A training module is used to train an improved Mask R-CNN model based on historical eucalyptus UAV data to obtain a single tree segmentation model. The improved Mask R-CNN model includes: a backbone network, a feature pyramid network (FPN), a region proposal network (RPN), a RoIAlign layer, a classifier module, and a mask module connected in sequence. Three enhanced selective kernel convolution modules (ESK) are set in the feature pyramid network (FPN), and each enhanced selective kernel convolution module (ESK) includes: a channel attention unit, a spatial attention unit, and a multi-scale convolution unit. The extraction module is used to input the UAV data of eucalyptus trees in the target area into the single tree segmentation model to extract the number of single eucalyptus trees and the crown area of single trees.

4. The eucalyptus single wood splitting device according to claim 3, characterized in that: A pre-processing module is also included for pre-processing the historical eucalyptus UAV data, which includes: A clipping unit, used to clip the historical eucalyptus UAV data; The annotation unit is used to annotate the edge contours of eucalyptus crowns based on the clipped historical eucalyptus UAV data.

5. A eucalyptus single wood splitting system comprising: A memory and a processor, wherein the memory stores a computer program executed by the processor, and when the computer program is executed by the processor, the method for splitting a single eucalyptus tree according to any one of claims 1 to 2 is executed.

6. A storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the method for segmenting a single eucalyptus tree according to any one of claims 1 to 2 is executed.

Images