Single-plant crown segmentation method and system based on remote sensing image

Abstract

The invention relates to the technical field of forestry remote sensing, in particular to a single-plant crown segmentation method based on a remote sensing image. Tree RGB images are collected through remote sensing images, three data sets of man-made forests, urban forests and natural forests are created, and data are preprocessed at the same time, including the steps of data enhancement, noise removal, image distortion correction, image size and color space unification and the like; a deep neural network TCSRCNN is used for training; and deploying the optimal model in an actual application scene, and performing automatic segmentation of the single-tree crown on a new remote sensing image. According to the method, the irregular edges of the crowns can be accurately segmented, and the problems of adhesion between single-tree crowns, insufficient generalization ability of the algorithm and the like are effectively solved. Accurate data support is provided for forest resource monitoring, and powerful technical support is provided for the fields of ecological environment assessment, forestry planning and management and the like.

Description

Technical Field

[0001] The invention relates to the field of forestry remote sensing technology, and in particular to a single tree crown segmentation method and system based on remote sensing images. Background Art

[0002] Current crown segmentation algorithms face a series of complex problems when facing practical applications. First, the complex and changing background and overlapping crowns make it difficult to accurately identify individual crowns, especially in dense forest environments. Second, the crown shapes and sizes of different tree species and growth stages vary greatly, which requires the algorithm to be highly flexible and adaptable to cope with various changes. However, the influence of lighting conditions and shadows further increases the difficulty of crown segmentation, because different lighting conditions will change the performance characteristics of the crown in the image. The generalization ability of the algorithm is also an important issue. Due to the differences in forest environments, tree species and growth conditions in different regions, the crown segmentation algorithm is required to have good generalization ability to adapt to different application scenarios. However, this often requires a large amount of labeled data to train the algorithm, and manually labeling the crown is a time-consuming and difficult task, which limits the number of samples that can be used for algorithm training.

[0003] The challenge of tree crown segmentation lies in the complexity of the forest itself, the diversity of tree species, and the irregularity of the spatial structure of the tree crown. These factors make many existing segmentation methods only applicable to specific tree species and within a specific resolution range. When dealing with tree species diversity, traditional algorithms often have difficulty capturing the morphological differences and texture features between different tree species, resulting in inaccurate segmentation results. For the irregularity of the spatial structure of the tree crown, including occlusion, overlap, and changes in lighting conditions between tree crowns, traditional methods may not be able to effectively distinguish between foreground and background, which in turn affects the accuracy and robustness of segmentation.

[0004] With the continuous advancement of remote sensing technology, the development of high spatial resolution remote sensing images and information extraction methods has provided a new solution for the extraction of single tree crowns. Compared with traditional manual delineation, this crown detection method based on remote sensing technology not only greatly improves work efficiency and reduces costs, but also realizes efficient monitoring and management of large areas. Summary of the invention

[0005] In view of the above problems existing in the prior art, the present invention is proposed.

[0006] To solve the above technical problems, the present invention provides the following technical solutions: a single tree crown segmentation method based on remote sensing images, comprising: collecting tree RGB images through remote sensing images, and creating three data sets of artificial forests, urban forests, and natural forests; preprocessing the data, including data enhancement, noise removal, image distortion correction, and unifying image size and color space; using a deep neural network TCSRCNN for training, the core of the TCSRCNN model lies in its feature extraction, region proposal generation, RoIAlign layer, and the collaborative work of the three heads of classification, regression, and segmentation; deploying the optimal model to actual application scenarios to automatically segment single tree crowns in new remote sensing images.

[0007] As a preferred solution of the single tree crown segmentation method based on remote sensing images described in the present invention, wherein: the acquisition of tree RGB images through remote sensing images includes manually marking the outline boundary of the tree crown, digitizing the crown layer, and associating it with data, including the number, species name, and year of each tree;

[0008] The preprocessing includes preprocessing the data using steps such as data enhancement, noise removal, image distortion correction, and image size and color space unification;

[0009] When dividing the dataset, the drone images were cut into square slices, and the minimum coverage area of ​​the manually labeled canopy was set to 60%. Among the image slices that met the criteria, 15% of the slices were randomly selected for the test set.

[0010] As a preferred solution of the single tree crown segmentation method based on remote sensing images described in the present invention, the use of the deep neural network TCSRCNN includes feature extraction, region proposal generation, RoIAlign layer, and the collaborative work of three heads of classification, regression and segmentation.

[0011] As a preferred solution of the single tree crown segmentation method based on remote sensing images described in the present invention, the training using the deep neural network TCSRCNN includes optimizing the backbone network, improving it to ResNeXt and embedding the SENet module; introducing the BiFPN-CBAM module; and optimizing the mask loss function to the Boundary-Dice loss function.

[0012] As a preferred solution of the single tree crown segmentation method based on remote sensing images described in the present invention, the introduced BiFPN-CBAM module is expressed as:

[0013]

[0014] in, is the middle feature in the right-to-left channel, is the output of the fifth layer. w1 and w2 are learnable weights, w3, w4, and w5 are learned weights, and ∈ is a fixed coefficient of 0.0001.

[0015] As a preferred solution of the single tree crown segmentation method based on remote sensing images described in the present invention, the mask loss function is optimized to be a Boundary-Dice loss function, which is expressed as:

[0016]

[0017] in, is a representation of the boundary of the ground truth region G, s θ represents the softmax probability output of the segmentation network, φ G It's the border The level set representation of .

[0018]

[0019] Among them, P is the predicted segmentation area and G is the actual segmentation area.

[0020] As a preferred solution of the single tree crown segmentation method based on remote sensing images described in the present invention, the automatic segmentation of single tree crowns of new remote sensing images includes integrating the model into the system and extracting single tree crown information from the remote sensing image through automated processing to support forest monitoring, forestry management and environmental protection applications.

[0021] Another object of the present invention is to provide a single tree crown segmentation system based on remote sensing images. The remote sensing image preprocessing module collects tree RGB images and creates three data sets of artificial forests, urban forests, and natural forests to perform data preprocessing, thereby improving data quality, optimizing data sets, and accelerating the training process; the TCSRCNN network training module uses deep neural networks for training to achieve accurate segmentation of single tree crowns, enhance the model's anti-interference ability, and reduce the complexity of artificial feature design through efficient learning; the crown segmentation application deployment module deploys the optimal model to actual scenarios, realizes automated processing and real-time monitoring of remote sensing images, and improves work efficiency. At the same time, the wide applicability of the model provides strong technical support for forest resource monitoring and management.

[0022] As a preferred solution of the remote sensing image-based single tree crown segmentation system described in the present invention, it includes a remote sensing image preprocessing module, a TCSRCNN network training module, and a crown segmentation application deployment module;

[0023] The remote sensing image preprocessing module collects RGB images of trees through remote sensing images, creates three data sets of artificial forests, urban forests, and natural forests, and preprocesses the data;

[0024] The TCSRCNN network training module uses the deep neural network TCSRCNN for training;

[0025] The crown segmentation application deployment module deploys the optimal model to the actual application scenario and automatically segments the crowns of individual trees in new remote sensing images.

[0026] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and wherein when the processor executes the computer program, the processor implements the steps of any one of the methods for segmenting a single tree crown based on remote sensing images.

[0027] A computer-readable storage medium having a computer program stored thereon, characterized in that when the computer program is executed by a processor, the steps of any one of the methods for segmenting a single tree crown based on remote sensing images are implemented.

[0028] Beneficial effects of the invention: The invention can accurately segment the irregular edges of tree crowns, effectively deal with the problems of adhesion between single tree crowns and insufficient generalization ability of the algorithm. It not only provides accurate data support for forest resource monitoring, but also provides strong technical support for ecological environment assessment, forestry planning and management and other fields. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0030] Figure 1 A schematic flow chart of a method for segmenting a single tree crown based on remote sensing images is provided as an embodiment of the present invention.

[0031] Figure 2 A schematic diagram of a TCSRCNN framework of a tree crown segmentation algorithm for a single tree crown segmentation method based on remote sensing images provided in one embodiment of the present invention.

[0032] Figure 3 A schematic diagram of the SE-ResNeXt structure of a single tree crown segmentation method based on remote sensing images provided by an embodiment of the present invention.

[0033] Figure 4A schematic diagram of the BiFPN-CBAM structure of a single tree crown segmentation method based on remote sensing images provided by an embodiment of the present invention.

[0034] Figure 5 A total loss curve diagram of a tree crown segmentation algorithm TCSRCNN for a single tree crown segmentation method based on remote sensing images provided in one embodiment of the present invention.

[0035] Figure 6 The segmentation effects of TCSRCNN on three data sets for a single tree crown segmentation method based on remote sensing images provided in one embodiment of the present invention.

[0036] Figure 7 An embodiment of the present invention provides a method for segmenting a single tree crown based on remote sensing images, which compares the segmentation effects of multiple algorithms. DETAILED DESCRIPTION

[0037] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in the art without creative work should fall within the scope of protection of the present invention.

[0038] Example 1, reference Figure 1-Figure 5 , which is the first embodiment of the present invention, and provides a method for segmenting a single tree crown based on remote sensing images, such as Figure 1 As shown, including:

[0039] S1. Collect RGB images of trees through remote sensing images and create three data sets of artificial forests, urban forests, and natural forests; preprocess the data, including data enhancement, noise removal, image distortion correction, and unification of image size and color space.

[0040] Specifically, S1 includes:

[0041] S11. Use the M350RTK UAV to collect data on the trees in the experimental area. First, configure the UAV and set its high-precision RTK positioning system to ensure that the acquired images have accurate geographic location information. Then plan the flight route and set the appropriate shooting height and angle to cover the entire experimental area and obtain image data with sufficient resolution. During the flight, the UAV will automatically capture a large number of high-quality remote sensing images, and the RTK system will record the precise location of each image. After completing the flight mission, collect and process these image data, and perform subsequent image stitching, correction, and analysis to extract detailed information on the trees in the experimental area.

[0042] S12. Perform data augmentation to expand the sample size and improve the generalization ability of the model, including operations such as rotation, flipping, cropping, and scaling. Next, noise removal is to improve image quality through filtering techniques or algorithms to eliminate random noise that may interfere with analysis. Then correct image distortion and correct image geometric distortion caused by shooting angle or equipment problems. Unify the image size to ensure that all images have the same resolution for subsequent processing and analysis. Finally, convert the image to a unified color space, such as RGB or grayscale, to ensure data consistency.

[0043] S13. Manually mark the outline boundary of the crown, then digitize the crown and link it to the data, including the number, species name, year, etc. of each tree. In the further experimental process, a total of 3538 digitized crowns and corresponding data were used. These steps ensure the accuracy and reliability of crown segmentation based on real scene data, providing a reliable basis for subsequent research. When dividing the data set, the drone imagery is cut into square slices, and the minimum coverage area of ​​the manually labeled crown is set to 60%, which prevents the crown in the data set from being too sparse, thereby affecting the sensitivity of the algorithm and the amount of available data. Among all the image slices that meet the criteria, 15% of the slices are randomly selected for the test set.

[0044] S2, use the deep neural network TCSRCNN for training, such as Figure 2 The TCSRCNN model framework shown in the figure, the core of the TCSRCNN model lies in its feature extraction, region proposal generation, RoIAlign layer, and the collaborative work of the three heads of classification, regression and segmentation.

[0045] S21, SE-ResNeXt model is an efficient image classification network built on the basis of ResNeXt model and combined with SE (Squeeze-and-Excitation) module. This model enhances the representation ability of the network by explicitly modeling the interdependence between feature channels. The SE module learns the importance of each feature channel and recalibrates these feature channels, so that the model automatically enhances useful features and suppresses redundant features during training.

[0046] like Figure 3The SE-ResNeXt model structure is shown in Figure 1. In SE-ResNeXt, each ResNeXt base block embeds a SE module. Specifically, the SE module first compresses the spatial dimension of the input feature map through a global average pooling operation to obtain a feature vector representing global information. The feature vector is then fed into two fully connected layers to learn the nonlinear relationship between channels and output the important weights of each channel. Finally, these weights are used to recalibrate the channels of the original feature map to achieve redistribution of feature channels.

[0047] S22. Bidirectional Feature Pyramid Network (BiFPN) is an efficient multi-scale feature fusion network that significantly optimizes the traditional feature pyramid network (FPN).

[0048] Taking P5 feature fusion as an example, the weighted bidirectional pyramid network structure can be expressed by the formula:

[0049]

[0050] in, is the middle feature in the right-to-left channel, is the output of the fifth layer. w1 and w2 are learnable weights, w3, w4, and w5 are learned weights, and ∈ is a fixed coefficient of 0.0001.

[0051] However, when fusing features, BiFPN treats each channel and spatial position relatively evenly, without highlighting key features in a targeted manner. Therefore, the purpose of introducing the CBAM module is to further enhance the model's sensitivity to important features through channel and spatial attention mechanisms. Figure 4 The structure of BiFPN and BiFPN-CBAM is shown in Figure 1. In this improvement, the output of BiFPN at each layer is processed by the CBAM module, which brings multiple advantages. First, through the channel attention mechanism, the network adaptively focuses on the more discriminative channels in the current task and reduces the interference of irrelevant features. Second, the spatial attention mechanism allows the network to pay more attention to important areas in the image, such as the edges, details or key information areas of the target, while ignoring redundant information in the background. This feature enhancement mechanism is particularly suitable for tasks that require high-precision object detection or detailed segmentation.

[0052] S23, boundary loss is mainly used for instance segmentation tasks. It improves the accuracy of segmentation boundaries by minimizing the difference between predicted boundaries and actual boundaries. In crown segmentation, the edges of crowns are often dynamic and fuzzy, so it is crucial to accurately identify the boundaries of crowns. Dice Loss (DLoss) is a loss function for image segmentation tasks, which is calculated based on the Dice coefficient. DLoss has significant advantages in dealing with class imbalance problems and can effectively handle situations where the number of positive and negative samples in segmentation tasks is large.

[0053] The BoundaryLoss and DLoss formulas are as follows:

[0054]

[0055] in, is a representation of the boundary of the ground truth region G, s θ represents the softmax probability output of the segmentation network, φ G It's the border The level set representation of .

[0056]

[0057] Among them, P is the predicted segmentation area and G is the actual segmentation area.

[0058] Combining Boundary Loss and DLoss takes full advantage of both. Boundary loss focuses on the accuracy of the boundary, while DLoss focuses on the balance of categories. The combination of the two ensures both the accurate identification of the crown boundary and the correctness and importance of the crown area in the overall segmentation result. This combined method enables the segmentation model to show higher performance and robustness in the crown segmentation task.

[0059] S3. Deploy the optimized model to actual application scenarios to automatically segment single tree crowns in new remote sensing images, aiming to achieve efficient and accurate crown identification. This process includes integrating the model into the system and extracting single crown information from remote sensing images through automated processing to support applications such as forest monitoring, forestry management, and environmental protection, ultimately improving data processing efficiency and decision-making quality.

[0060] Example 2, reference Figure 6 and Figure 7 , which is the second embodiment of the present invention, and provides a method for segmenting a single tree crown based on remote sensing images;

[0061] This example conducts visual evaluation and quantitative analysis on the above three data sets (poplar plantation, urban mixed forest, and natural forest) to quantitatively demonstrate the superiority in terms of completeness, correctness, and effectiveness. Figure 6 It shows the segmentation effect of TCSRCNN on three datasets. Figure 7 The segmentation results of various algorithms are compared, showing the superiority of TCSRCNN. TCSRCNN performs best on the artificial forest dataset, with most tree crowns being segmented and a lower missed detection rate. This is related to the high image resolution and orderly arrangement of trees in the artificial forest dataset. Since the image resolution of the urban forest dataset is slightly lower, the segmentation effect is slightly worse, some tree crowns are difficult to segment, and some shadows may be mistaken for tree crowns. On the natural forest dataset, the segmentation results are satisfactory, but the segmentation results are reduced due to the interference of dense tree crowns and ground weeds.

[0062] In order to more intuitively demonstrate the performance of the improved algorithm, four algorithms, Depth Anything, FastSAM, ZoeDepth, and SAM, are selected for qualitative analysis. Figure 6 As shown in the figure, TCS R-CNN has good performance on all three datasets. There are too many tree crowns in DepthAnything and ZoeDepth, and the effect is not good for areas with dense tree crowns. FastSAM and the original Mask R-CNN have certain segmentation capabilities, but there are many missed detections and false detections. SAM has the strongest segmentation ability, but it performs poorly when segmenting more complex tree crowns.

[0063] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

[0064] Embodiment 3, the third embodiment of the present invention, is different from the first two embodiments in that:

[0065] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.

[0066] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0067] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0068] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0069] Embodiment 4, this embodiment provides a single tree crown segmentation system based on remote sensing images, characterized by: comprising a remote sensing image preprocessing module, a TCSRCNN network training module, and a tree crown segmentation application deployment module;

[0070] The remote sensing image preprocessing module collects RGB images of trees through remote sensing images, creates three data sets of artificial forests, urban forests, and natural forests, and preprocesses the data;

[0071] The TCSRCNN network training module uses the deep neural network TCSRCNN for training;

[0072] The crown segmentation application deployment module deploys the optimal model to the actual application scenario and automatically segments the crowns of individual trees in new remote sensing images.

[0073] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A method for segmenting a single tree crown based on remote sensing images, characterized in that: include, Collect RGB images of trees through remote sensing images, create three data sets of artificial forests, urban forests, and natural forests, and pre-process the data; Use the deep neural network TCSRCNN for training; The optimal model is deployed to actual application scenarios to automatically segment single tree crowns in new remote sensing images.

2. The method for segmenting a single tree crown based on remote sensing images according to claim 1, characterized in that: The collecting of tree RGB images through remote sensing images includes manually marking the outline boundary of the tree crown, digitizing the crown layer, and associating it with data, including the number, species name, and year of each tree; The preprocessing includes preprocessing the data using data enhancement, noise removal, image distortion correction, and image size and color space unification steps; When dividing the dataset, the drone images were cut into square slices, and the minimum coverage area of ​​the manually labeled canopy was set. Among the image slices that met the criteria, slices were randomly selected for the test set.

3. The method for segmenting a single tree crown based on remote sensing images according to claim 2, characterized in that: The use of the deep neural network TCSRCNN includes feature extraction, region proposal generation, RoIAlign layer, and the collaborative work of three heads: classification, regression, and segmentation.

4. The method for segmenting a single tree crown based on remote sensing images according to claim 1, characterized in that: The training using the deep neural network TCSRCNN includes optimizing the backbone network, improving it to ResNeXt and embedding the SENet module; introducing the BiFPN-CBAM module; and optimizing the mask loss function to the Boundary-Dice loss function.

5. The method for segmenting a single tree crown based on remote sensing images according to claim 4, characterized in that: The introduced BiFPN-CBAM module is expressed as: in, is the middle feature in the right-to-left channel, is the output of the fifth layer, w1 and w2 are learnable weights, w3, w4, and w5 are learned weights, and ∈ is a fixed coefficient of 0.0001.

6. The method for segmenting a single tree crown based on remote sensing images according to claim 1, characterized in that: The optimization of the mask loss function to the Boundary-Dice loss function is expressed as: in, is a representation of the boundary of the ground truth region G, s θ represents the softmax probability output of the segmentation network, φ G It's the border The level set representation is Among them, P is the predicted segmentation area and G is the actual segmentation area.

7. The method for segmenting a single tree crown based on remote sensing images according to claim 1, characterized in that: The automatic segmentation of individual tree crowns for new remote sensing images includes integrating the model into the system and extracting individual tree crown information from the remote sensing images through automated processing to support forest monitoring, forestry management, and environmental protection applications.

8. A system for segmenting a single tree crown based on remote sensing images according to any one of claims 1 to 7, characterized in that: Including remote sensing image preprocessing module, TCSRCNN network training module, and crown segmentation application deployment module; The remote sensing image preprocessing module collects RGB images of trees through remote sensing images, creates three data sets of artificial forests, urban forests, and natural forests, and preprocesses the data; The TCSRCNN network training module uses the deep neural network TCSRCNN for training; The crown segmentation application deployment module deploys the optimal model to the actual application scenario and automatically segments the crowns of individual trees in new remote sensing images.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images