A High-Detection-Rate Method for Individual Tree Detection in Unmanned Aerial Vehicle Remote Sensing Images of Complex Forest Land

Abstract

This invention relates to a high-detection-rate method for individual tree detection in complex forest land using UAV remote sensing images. Compared with existing technologies, it overcomes the shortcomings of difficulty in detecting individual trees under conditions of steep slopes and canopy occlusion. The invention includes the following steps: data acquisition and preprocessing; construction of a lightweight individual tree detection model; training of the lightweight individual tree detection model; acquisition of the UAV remote sensing image to be detected; and obtaining the individual tree detection results from the UAV remote sensing image. This invention achieves accurate extraction of individual trees even in complex environments such as mountainous areas with steep slopes, large differences in tree canopy size, and canopy image occlusion.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of unmanned aerial vehicle remote sensing image processing, in particular to a complex forest high-detection-rate unmanned aerial vehicle remote sensing image forest tree individual detection method. BACKGROUND

[0002] Affected by forest pests, there are many types of forest pests, and their main damage sites are also different. Although their damage methods are not the same, they will eventually affect the growth of trees and change the appearance and crown of trees. With the development of unmanned aerial vehicle aerial photography technology, artificial intelligence, big data and other technologies, the use of unmanned aerial vehicle remote sensing for suspected sick trees in mountainous forest areas can greatly improve the detection efficiency. Using unmanned aerial vehicles combined with automatic AI detection and recognition is an advanced technical means different from traditional forest inspection.

[0003] In the process of forest inspection, first, the single tree needs to be accurately positioned and recognized to obtain more accurate detection results. However, in the complex mountainous mixed forest environment, due to the large slope, the large number of forest tree species, the concentrated distribution state, and the size difference of the crown, the tree crown layer is distributed in a complex manner, and the tree crown layer image obtained by aerial remote sensing means is blocked, resulting in a large increase in single tree detection omission rate.

[0004] In order to solve the problem of missed detection caused by large slope and crown layer blocking and improve the detection accuracy and speed of large-area remote sensing image detection, a target automatic detection model needs to be designed to reduce the single tree missed detection problem caused by terrain and natural ecological growth, thereby laying a foundation for improving the accuracy and efficiency of later pest identification. SUMMARY

[0005] The purpose of the present application is to solve the problem of single tree detection under the condition of large slope and crown layer blocking in the prior art, and to provide a complex forest high-detection-rate unmanned aerial vehicle remote sensing image forest tree individual detection method to solve the above problems.

[0006] In order to achieve the above purpose, the technical scheme of the present application is as follows:

[0007] A complex forest high-detection-rate unmanned aerial vehicle remote sensing image forest tree individual detection method, comprising the following steps:

[0008] Data acquisition and preprocessing: using an unmanned aerial vehicle to carry a visible light camera to collect forest remote sensing images of complex forests, the forest remote sensing images including visible images and elevation information of the target area, and generating DEM, DSM and orthographic images of the target area through professional software and performing data preprocessing;

[0009] Construction of a lightweight forest individual tree detection model: the lightweight forest individual tree detection model includes an orthographic image reconstruction module and an SC-RetinaNet network, the SC-RetinaNet network introduces a shffleNetv2 module and a C3-CBAM module to improve the RetinaNet network;

[0010] Training of a lightweight forest individual tree detection model: input the preprocessed data into the lightweight forest individual tree detection model for training;

[0011] Acquisition of a to-be-detected unmanned aerial vehicle remote sensing image: acquire a to-be-detected unmanned aerial vehicle remote sensing image and preprocess it;

[0012] Obtaining of an unmanned aerial vehicle remote sensing image forest individual tree detection result: inputting the preprocessed unmanned aerial vehicle remote sensing image into the trained lightweight forest individual tree detection model to obtain an unmanned aerial vehicle remote sensing image forest individual tree detection result.

[0013] The data acquisition and preprocessing includes the following steps:

[0014] Visible images and elevation information of the target area are collected using a camera mounted on an unmanned aerial vehicle, and are spliced to generate DEM, DSM and orthographic images of the target area;

[0015] Image superposition is performed on the spliced DEM and DSM data to obtain normalized digital surface model data NDSM, which is calculated as follows:

[0016] NDSM = DSM - DEM;

[0017] The DEM data is subjected to terrain processing, and slope data is generated according to the slope calculation formula as follows:

[0018] [dz / dx] = ((c+2f+i)-(a+2d+g) / (8*x_cellsize))

[0019] [dz / dy] = ((g+2h+i)-(a+2d+c) / (8*y_cellsize))

[0020]

[0021] Wherein, letters a to i represent the center pixel (e) and its adjacent eight pixel values, and the adjacent eight pixels determine the horizontal and vertical increments;

[0022] x_cellsize represents the length of a pixel in the horizontal direction, y_cellsize represents the length of a pixel in the vertical direction, dz / dx is the rate of change of the surface in the horizontal direction from the center pixel, that is, the rate of change of the pixel e in the x direction, dz / dy is the rate of change of the surface in the vertical direction from the center pixel, that is, the rate of change of the pixel e in the y direction, ATAN is an inverse tangent function returning a numerical value, and slop_redians is the slope of the center pixel e.

[0023] The construction of the single tree detection model comprises the following steps:

[0024] The first part of the single tree detection model is set as an orthographic image reconstruction module, and the second part is an SC-RetinaNet network.

[0025] The orthographic image reconstruction module is set based on normalized digital surface model data NDSM, and the visible three-channel image and the NDSM single-channel image are reconstructed to obtain a fourth-channel reconstructed image.

[0026] The SC-RetinaNet network is set as follows:

[0027] The first part of the SC-RetinaNet network is set as a main feature extraction network A and a main feature extraction network B.

[0028] The main feature extraction network B comprises nine layers, the first layer is a convolutional layer, the second, fourth, sixth and eighth layers are shffleNetv2 modules, and the third, fifth, seventh and ninth layers are C3-CBAM modules.

[0029] The second part of the SC-RetinaNet network is set as a feature fusion layer, that is, an original RetinaNet network FPN layer, which comprises three layers, P3, P4 and P5.

[0030] The third part of the SC-RetNet network is set to three prediction, classification and regression subnetworks. The classification and regression subnetworks perform target classification on the three feature information output by the second part, and the regression subnetworks perform target boundary regression to obtain the target prediction box.

[0031] Build the ShffleNetv2 module;

[0032] ShuffleNetv2 is constructed from convolutional and downsampling modules. When extracting feature information, group convolution and channels are used to compress the model. The left branch of the ShuffleNetv2 module uses a 3×3 DW convolution and a 1×1 convolution, while the right branch uses a 1×1 quadratic convolution and a 3×3 DW convolution. After convolution, the channels are concatenated using Concat, doubling the number of output channels. Finally, the output feature layer is shuffled.

[0033] Construct a CBMA attention module;

[0034] Construct a C3-CBMA attention module and introduce the channel-space attention module into the C3 module to obtain the C3-CBAM module;

[0035] The C3-CBAM module performs branching operations on the input feature matrix. The left branch undergoes convolution and multiple bottleneck layers, while the right branch undergoes convolution and a channel spatial attention module to enhance feature extraction capabilities. The feature layers from the two branches are concatenated and then convolved for output.

[0036] The setting of the orthophoto reconstruction module includes the following steps:

[0037] The NDSM raster data is normalized using the maximum and minimum value formulas, as shown in the following formula;

[0038] The processed NDSM data and orthophotos were fused as bands, with the NDSM data being the reconstructed image from the fourth channel.

[0039]

[0040] It is x j The minimum value in the column. It is x j The maximum value in the column;

[0041] Trees in the orthophotos were labeled with individual rectangular bounding boxes. The dataset was created based on the DEM slope data according to the terrain slope standard and divided into two categories: steep and dangerous areas with a slope greater than 25 degrees and other categories. Each dataset includes orthophotos, NDSM images, and XML files of individual tree bounding box labels. The orthophotos and NDSM images correspond one-to-one, and the image size is 512×512.

[0042] The construction of the CBMA attention module includes the following steps:

[0043] Configure the channel attention module:

[0044] The channel attention module generates channel attention weight information by utilizing the channel relationships between image features. This module first uses average pooling and max pooling operations to aggregate the channel information, generating two distinct channel context information bundles: and These represent average pooling features and max pooling features, respectively; then these two information bundles are transferred to the same shared network, which consists of a multilayer perceptron (MLP) containing one hidden layer; element-wise summation is used to merge the output feature vectors to generate channel attention weight information M. c ∈R C*1*1 W1∈R C*C*16 W0∈R C*16*C ;

[0045] The CBMA attention module is configured to use average pooling and max pooling operations to aggregate channel information, generating two distinct bundles of channel context information: and They represent the average pooling feature and the max pooling feature, respectively;

[0046] These two information bundles are transferred to the same shared network, which consists of a multilayer perceptron (MLP) containing a hidden layer.

[0047] Element-wise summation is used to merge the output feature vectors to generate channel attention weight information.

[0048] M c ∈R C*1*1 W1∈R C*C*16 W0∈R C*16*C ,

[0049] Where σ is the sigmoid function;

[0050] The channel attention module is calculated as follows:

[0051]

[0052] Where σ is the sigmoid function, R is the feature map, c is the number of channels, MC, W1, W0, and F are sub-feature maps, MLP is a multilayer perceptron, AvgPool is the average pooling operation, and MaxPool is the max pooling operation. These represent the average pooling feature and the max pooling feature, respectively.

[0053] Setting up the spatial attention module:

[0054] The spatial attention module generates spatial attention weight information by utilizing the spatial relationships between image features. The spatial attention module focuses on the image spatial dimension feature information, which together with the channel feature information of the channel attention module constitutes the channel spatial attention module.

[0055] Spatial attention calculation first adopts the channel attention average pooling method. and max pooling operation They are then concatenated to generate an effective feature map, and then spatial attention weights M are generated through a 7x7 convolutional layer and an activation function σ. s (F)∈R H*W ;

[0056] Channel attention average pooling is used. and max pooling operation And connect them to generate an effective feature map;

[0057] Spatial attention weights M are generated using a 7x7 convolutional layer and an activation function σ. s (F)∈R H*W ,

[0058] The expression for the spatial attention module is as follows:

[0059]

[0060] Where σ is the sigmoid function, R is the feature map, s is the channel, AvgPool is the average pooling operation, and MaxPool is the max pooling operation. These represent average pooling and max pooling operations on s, respectively.

[0061] The training of the lightweight single-tree detection model includes the following steps:

[0062] The training iteration count was set to 100. The main network was frozen for the first 50 iterations with a batch size of 8. After 50 iterations, the training was unfrozen with a batch size of 4. The Adam optimizer was used with an initial learning rate of 0.0005. The learning rate decreased as the number of iterations increased, i.e., after two iterations, the learning rate was 0.9 times the original learning rate.

[0063] The preprocessed data is input into the single tree detection model. Image reconstruction is performed in the first part of the model. The orthophoto and NDSM image in the dataset are reconstructed to obtain a reconstructed image with dimensions of 512×512×4.

[0064] Training the SC-RetinaNet network:

[0065] First, the main feature extraction network A extracts features from the fourth channel of the reconstructed image, and the fifth layer outputs feature A1 with dimensions of 128×128×64.

[0066] The main feature extraction network B extracts features from the first three channels of the reconstructed image. The fifth layer output feature B1 has a dimension of 128×128×256, the seventh layer output feature B2 has a dimension of 64×64×512, and the ninth layer output feature B3 has a dimension of 32×32×1024.

[0067] The above features are then fused. Features A1 and B3 are input into the P3 layer of the FPN module, features A1 and B2 are input into the P4 layer of the FPN module, and features A1 and B1 are input into the P5 layer of the FPN module.

[0068] Finally, the three output features are used by the classification and regression sub-network to perform target classification, and the regression sub-network performs target boundary regression to obtain the target prediction box. After iteration, the loss value converges, and the pre-trained model is obtained.

[0069] Beneficial effects

[0070] This invention presents a high-detection-rate method for individual tree detection in complex forest land using UAV remote sensing images. Compared with existing technologies, this method achieves accurate extraction of individual trees even in complex environments such as steep slopes, significant differences in tree canopy size, and canopy image occlusion in mountainous areas. This invention reconstructs the remote sensing image through a normalized digital surface model and slope domain segmentation. Utilizing the SC-RetinaNet network, it achieves high-precision detection and localization of individual trees while improving detection efficiency and accelerating the detection process for individual trees in complex mountainous environments. Attached Figure Description

[0071] Figure 1 This is a sequence diagram of the method of the present invention;

[0072] Figure 2 This is a technical roadmap of the present invention;

[0073] Figure 3 This is a network model diagram of the present invention;

[0074] Figure 4 a is a structural diagram of the ShuffleNetv1 module in the network model of this invention;

[0075] Figure 4 b is a structural diagram of the ShuffleNetv2 module in the network model of this invention;

[0076] Figure 5 This is a structural diagram of the attention module in the network model of the present invention;

[0077] Figure 6 This is a structural diagram of the C3-CBAM module in the network model of this invention;

[0078] Figure 7 This is a sample image of the detection results of the present invention. Detailed Implementation

[0079] To provide a better understanding of the structural features and effects achieved by the present invention, a detailed description is provided below, accompanied by preferred embodiments and accompanying drawings:

[0080] like Figure 1 and Figure 2 As shown, the present invention provides a method for high-detection-rate single-tree detection in complex forest land using UAV remote sensing images, comprising the following steps:

[0081] The first step is data acquisition and preprocessing: using a drone equipped with a visible light camera, forest remote sensing images of complex woodlands are acquired. The forest remote sensing images include visible images and elevation information of the target area. The images are then stitched together using professional software to generate DEM, DSM and orthophotos of the target area and are then preprocessed.

[0082] (1) Use a drone equipped with a camera to collect visible images and elevation information of the target area, and stitch them together to generate DEM, DSM and orthophoto of the target area.

[0083] (2) The DEM and DSM data generated by the stitching are overlaid to obtain the normalized digital surface model data NDSM. The calculation of the normalized digital surface model data is as follows:

[0084] NDSM = DSM-DEM;

[0085] The DEM data undergoes terrain processing, and slope data is generated based on the slope calculation formula, which is as follows:

[0086] [dz / dx]=((c+2f+i)-(a+2d+g) / (8*x_cellsize))

[0087] [dz / dy]=((g+2h+i)-(a+2d+c) / (8*y_cellsize))

[0088]

[0089] In this context, letters a to i represent the values ​​of the center pixel (e) and its eight neighboring pixels, with the eight neighboring pixels determining the horizontal and vertical increments.

[0090] x_cellsize represents the length of a cell in the horizontal direction, y_cellsize represents the length of a cell in the vertical direction, dz / dx is the rate of change of the surface in the horizontal direction starting from the center cell, that is, the rate of change of cell e in the x direction, dz / dy is the rate of change of the surface in the vertical direction starting from the center cell, that is, the rate of change of cell e in the y direction, ATAN is the arctangent function that returns the number, and slop_redians is the slope of the center cell e.

[0091] The second step is the construction of a lightweight tree detection model: The lightweight tree detection model includes an orthophoto reconstruction module and SC-RetinaNet. SC-RetinaNet is an improvement on the SC-RetinaNet network by introducing shffleNetv2 and C3-CBAM.

[0092] In forests, trees overlap and obscure each other due to terrain, height, and age, and their canopy sizes vary. The original RetinaNet network has a high false negative rate for small-area and overlapping variegated trees, resulting in a relatively slow overall detection speed. To address these issues, this invention studies a deep learning image recognition method that combines Normalized Digital Surface Model (NDSM) data with an improved RetinaNet model.

[0093] NDSM data records the height information of all ground features above ground level, reflecting their true height. This information plays a crucial role in image classification for distinguishing different types of ground features (such as buildings and ground, trees and low vegetation). This method improves the feature richness of the RetinaNet model by incorporating the height information provided by NDSM, optimizing the recognition results, and eliminating minor misclassifications to achieve high-precision image recognition.

[0094] Therefore, improvements were made to the RetinaNet network in the main feature extraction module. First, the first part of the main network is used to fully extract features from the NDSM image. Then, the second part of the main network is used to extract features from the original visible image. Finally, the extracted features from the two parts are fused. The three extracted deep feature layers are fused with the FPN layer, and then predictions are performed on each of the fused feature layers. The last two prediction network branches are divided into classification and box regression parts. The classification sub-network performs target classification on the output of the backbone network, and the box regression sub-network performs target boundary regression. Multiple ShuffleNetv2 modules and a C3-CBAM module with an attention mechanism are introduced into the network to enhance feature extraction capabilities while accelerating detection speed. The C3-CBAM module is also introduced into the FPN (Feature Pyramid Network) layer to further enhance feature fusion capabilities and the accuracy of output features, resulting in more accurate detection precision for the classification and regression sub-networks. The SC-RetinaNet network structure is as follows: Figure 3 As shown.

[0095] The construction of the single-tree detection model includes the following steps:

[0096] (1) The first part of the single tree detection model is set as the orthophoto reconstruction module and the second part is the SC-RetinaNet network.

[0097] (2) Set up the orthophoto reconstruction module. Based on the normalized digital surface model data NDSM, set up the orthophoto reconstruction module to reconstruct the visible three-channel image and the NDSM single-channel image to obtain the reconstructed image of the fourth channel.

[0098] Setting up the orthophoto reconstruction module includes the following steps:

[0099] A1) Normalize the NDSM raster data using the maximum and minimum value formulas, as shown in the following formula;

[0100] The processed NDSM data and orthophotos were fused as bands, with the NDSM data being the reconstructed image from the fourth channel.

[0101]

[0102] It is x j The minimum value in the column. It is x j The maximum value in the column;

[0103] A2) Individual tree bounding boxes are labeled for trees in the orthophotos, and a dataset is created based on the DEM slope data according to the terrain slope standard. The dataset is divided into two categories: steep and dangerous areas with a slope greater than 25 degrees and other categories. Each dataset includes orthophotos, NDSM images, and individual tree bounding box label XML files. The orthophotos and NDSM images correspond one-to-one, and the image size is 512×512.

[0104] (3) Configure the SC-RetinaNet network.

[0105] B1) The first part of the SC-RetinaNet network is set as the main feature extraction network A and the main feature extraction network B. The main feature extraction network A consists of five layers: the first layer is a convolutional layer, the second and fourth layers are shffleNetv2 modules, and the third and fifth layers are C3-CBAM modules. The main feature extraction network A is used to extract features from the reconstructed image of the fourth channel, and the fifth layer outputs feature A1.

[0106] The main feature extraction network B consists of nine layers. The first layer is a convolutional layer, the second, fourth, sixth and eighth layers are shffleNetv2 modules, and the third, fifth, seventh and ninth layers are C3-CBAM modules. The main feature extraction network B is used to extract features from the visible three channels of the reconstructed image. The fifth, seventh and ninth layers output features B1, B2 and B3 respectively.

[0107] B2) The second part of the SC-RetinaNet network is set as the feature fusion layer, which is the original RetinaNet network FPN layer. There are three layers in total, namely P3, P4 and P5. The output of each layer is connected to a C3-CBAM module. Features A1 and B3 are input to the P3 layer in the FPN module, features A1 and B2 are input to the P4 layer in the FPN module, and features A1 and B1 are input to the P5 layer in the FPN module to perform feature fusion on the features extracted in the first part.

[0108] B3) The third part of the SC-RetNet network is set to three prediction, classification and regression subnetworks. The classification and regression subnetworks perform target classification on the three feature information output by the second part, and the regression subnetworks perform target boundary regression to obtain the target prediction box.

[0109] (4) Construct the ShffleNetv2 module.

[0110] The ShuffleNetv2 module is a lightweight neural network proposed to improve upon the ShuffleNetv1 module based on extensive experimental data. ShuffleNetV2 is constructed from convolutional and downsampling modules. When extracting feature information, it utilizes group convolution and channel (shuffle) to compress the model, significantly reducing computational cost. Its structure is as follows:Figure 4 As shown.

[0111] ShuffleNetv1 module utilizes Figure 4 In (a), the block unit segments the channels of the input feature matrix. The left branch can be considered as residual edges, used to deepen the network layers. The right branch uses a quadratic 1×1 convolution and a single 3×3 DW convolution. Channels are concatenated using Concat. The number of output channels is consistent with the number of input channels.

[0112] The ShuffleNetv2 module uses Figure 4 In (b), the block unit no longer splits the channels of the input feature matrix. The left branch uses one 3×3 DW convolution and one 1×1 convolution. The right branch uses a 1×1 quadratic convolution and one 3×3 DW convolution. After the convolution, the channels are concatenated using Concat, doubling the number of output channels. Finally, the output feature layer is shuffled to further enhance the feature extraction capability.

[0113] ShuffleNetv2 is constructed from convolutional and downsampling modules. When extracting feature information, group convolution and channels are used to compress the model. The left branch of the ShuffleNetv2 module uses a 3×3 DW convolution and a 1×1 convolution, while the right branch uses a 1×1 quadratic convolution and a 3×3 DW convolution. After convolution, the channels are concatenated using Concat, doubling the number of output channels. Finally, the output feature layer is shuffled.

[0114] (5) Construct the CBMA attention module.

[0115] The attention module includes a channel attention module and a spatial attention module, which are connected in a cascaded manner, such as... Figure 5 As shown.

[0116] The channel attention module generates channel attention weight information by utilizing the channel relationships between image features. This module first uses average pooling and max pooling operations to aggregate the channel information, generating two distinct bundles of channel context information: and These represent the average pooling feature and the max pooling feature, respectively. These two information bundles are then transferred to the same shared network, which consists of a multilayer perceptron (MLP) containing a hidden layer.

[0117] The spatial attention module generates spatial attention weights by utilizing the spatial relationships between image features. Unlike the channel attention module, the spatial attention module focuses on image spatial dimension features, which together with the channel features of the channel attention module constitute the channel spatial attention module. The calculation of spatial attention first follows the channel attention average pooling method. and max pooling operation They are then concatenated to generate an effective feature map, and then spatial attention weights M are generated through a 7x7 convolutional layer and an activation function σ. s (F)∈R H*W .

[0118] The construction of the CBMA attention module includes the following steps:

[0119] C1) Configure the channel attention module:

[0120] The channel attention module generates channel attention weight information by utilizing the channel relationships between image features. This module first uses average pooling and max pooling operations to aggregate the channel information, generating two distinct channel context information bundles: and These represent average pooling features and max pooling features, respectively; then these two information bundles are transferred to the same shared network, which consists of a multilayer perceptron (MLP) containing one hidden layer; element-wise summation is used to merge the output feature vectors to generate channel attention weight information M. c ∈R C*1*1 W1∈R C*C*16 W0∈R C*16*C ;

[0121] C2) Configure the CBMA attention module to use average pooling and max pooling operations to concentrate channel information and generate two different bundles of channel context information: and They represent the average pooling feature and the max pooling feature, respectively;

[0122] These two information bundles are transferred to the same shared network, which consists of a multilayer perceptron (MLP) containing a hidden layer.

[0123] Element-wise summation is used to merge the output feature vectors to generate channel attention weight information.

[0124] M c ∈R C*1*1 W1∈R C*C*16 W0∈R C*16*C ,

[0125] Where σ is the sigmoid function;

[0126] The channel attention module is calculated as follows:

[0127]

[0128] Where σ is the sigmoid function, R is the feature map, c is the number of channels, MC, W1, W0, and F are sub-feature maps, MLP is a multilayer perceptron, AvgPool is the average pooling operation, and MaxPool is the max pooling operation. These represent the average pooling feature and the max pooling feature, respectively.

[0129] C3) Configure the spatial attention module:

[0130] The spatial attention module generates spatial attention weight information by utilizing the spatial relationships between image features. The spatial attention module focuses on the image spatial dimension feature information, which together with the channel feature information of the channel attention module constitutes the channel spatial attention module.

[0131] Spatial attention calculation first adopts the channel attention average pooling method. and max pooling operation They are then concatenated to generate an effective feature map, and then spatial attention weights M are generated through a 7x7 convolutional layer and an activation function σ. s (F)∈R H*W ;

[0132] Channel attention average pooling is used. and max pooling operation And connect them to generate an effective feature map;

[0133] Spatial attention weights M are generated using a 7x7 convolutional layer and an activation function σ. s (F)∈R H*W ,

[0134] The expression for the spatial attention module is as follows:

[0135]

[0136] Where σ is the sigmoid function, R is the feature map, s is the channel, AvgPool is the average pooling operation, and MaxPool is the max pooling operation. These represent average pooling and max pooling operations on s, respectively.

[0137] (6) Construct the C3-CBMA attention module and introduce the channel space attention module into the C3 module to obtain the C3-CBAM module; the C3-CBAM module performs branching operations on the input feature matrix. The left branch performs convolution and multiple bottleneck layer operations, and the right branch is used to enhance feature extraction capability through convolution and the channel space attention module; the feature layers of the two branches are concatenated by Concat and then convolved for output.

[0138] To improve the detection efficiency of the RetinaNet network, the channel spatial attention module is introduced into the C3 module to obtain the C3-CBAM module, with the structure as follows: Figure 6 As shown, the input feature matrix undergoes branching operations. The left branch performs convolution and multiple bottleneck layers. Bottlenecks are typically used in deeper networks to reduce network parameters and computational cost. The right branch passes through convolution and a channel spatial attention module to enhance feature extraction capabilities. The feature layers from the two branches are concatenated using Concat and then convolved for output. The C3-CBAM module maintains consistency between the input and output feature layers and is primarily used to enhance feature extraction and reduce model parameters.

[0139] The third step is to train the lightweight tree detection model: input the preprocessed data into the tree detection model for training.

[0140] (1) Set the number of training iterations to 100. Freeze the main network for the first 50 iterations and batch process Bich_size to 8. After 50 iterations, unfreeze the training and set Bich_size to 4. Select the Adam optimizer and set the initial learning rate to 0.0005. As the number of iterations increases, the learning rate decreases, i.e., after two iterations, the learning rate is 0.9 times the original learning rate.

[0141] (2) Input the preprocessed data into the single tree detection model and perform image reconstruction in its first part. Reconstruct the orthophoto and NDSM image in the dataset to obtain a reconstructed image with dimensions of 512×512×4.

[0142] (3) Training the SC-RetinaNet network:

[0143] First, the main feature extraction network A extracts features from the fourth channel of the reconstructed image, and the fifth layer outputs feature A1 with dimensions of 128×128×64.

[0144] The main feature extraction network B extracts features from the first three channels of the reconstructed image. The fifth layer output feature B1 has a dimension of 128×128×256, the seventh layer output feature B2 has a dimension of 64×64×512, and the ninth layer output feature B3 has a dimension of 32×32×1024.

[0145] The above features are then fused. Features A1 and B3 are input into the P3 layer of the FPN module, features A1 and B2 are input into the P4 layer of the FPN module, and features A1 and B1 are input into the P5 layer of the FPN module.

[0146] Finally, the three output features are used by the classification and regression sub-network to perform target classification, and the regression sub-network performs target boundary regression to obtain the target prediction box. After iteration, the loss value converges, and the pre-trained model is obtained.

[0147] The fourth step is to acquire the remote sensing image of the UAV to be detected: acquire the remote sensing image of the UAV to be detected and preprocess it.

[0148] Step 5: Obtaining the detection results of individual trees in UAV remote sensing images: Input the preprocessed UAV remote sensing images into the trained individual tree detection model to obtain the detection results of individual trees in UAV remote sensing images.

[0149] To demonstrate the superiority of the proposed SC-RetinaNet network model, Faster R-CNN and RetinaNet network models were introduced for comparison. First, all three algorithms were trained using the same dataset, and the optimal training weight model was selected for single-plant testing. With an IOU threshold of 0.50 for all three, the average accuracy and overall accuracy of the three algorithms for single-plant detection, ranked from highest to lowest, are: SC-RetinaNet, RetinaNet, and Faster R-CNN. The SC-RetinaNet algorithm achieved an average accuracy of 85.69% and an overall accuracy of 82.58%, respectively, which is 2.74% and 4.39% higher than RetinaNet, and 5.37% and 7.88% higher than Faster R-CNN, respectively. This indicates that the SC-RetinaNet algorithm proposed in this patent outperforms the other two algorithms in single-plant detection. The SC-RetinaNet algorithm achieved a single-plant detection precision of 87.38%, representing improvements of 6.11% and 17.07%, respectively. This indicates that the SC-RetinaNet algorithm enhances feature extraction capabilities, improves the detection of stacked occluded single plants, and increases the number of correctly detected single plants. The SC-RetinaNet algorithm's recall is 2.95% higher than the RetinaNet algorithm, reducing the false positive rate when detecting single plants. The SC-RetinaNet algorithm reduces the single-image detection time by 0.22s and 0.46s compared to the other two algorithms, further improving detection efficiency. The evaluation metrics for single-plant detection by the three algorithms are shown in Table 1.

[0150] Table 1 Evaluation metrics of three algorithms for diseased tree detection

[0151]

[0152] like Figure 7 The image shown is a map illustrating the results of single-tree detection and identification in a mixed forest area with a slope of 25°–45°. Figure 7 As can be seen, good single-tree detection and recognition results were obtained for trees of different sizes. For trees that were partially covered due to the slope, the recognition results were also greatly improved, further verifying the superiority of this invention in single-tree detection and recognition in sloping areas.

[0153] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the claimed invention. The scope of protection claimed by the appended claims and their equivalents is defined.

Claims

1. A method for detecting individual trees in complex forest high detection rate UAV remote sensing images, characterized in that, The method comprises the following steps: 11) data acquisition and preprocessing: using a visible light camera carried by a UAV to collect forest remote sensing images of complex forest land, the forest remote sensing images comprising visible images and height information of a target area, and generating DEM, DSM and orthographic images of the target area by splicing and performing data preprocessing through professional software; 12) construction of a lightweight single tree detection model: the lightweight single tree detection model comprising an orthographic image reconstruction module and an SC-RetinaNet network; The construction of the single tree detection model comprises the following steps: 121) setting the first part of the single tree detection model as the orthographic image reconstruction module and the second part as the SC-RetinaNet network; 122) setting the orthographic image reconstruction module based on normalized digital surface model data NDSM, reconstructing a visible three-channel image and an NDSM single-channel image to obtain a fourth-channel reconstructed image; 123) setting the SC-RetinaNet network: 1231) setting the first part of the SC-RetinaNet network as a main feature extraction network A and a main feature extraction network B, the main feature extraction network A comprising five layers, the first layer being a convolutional layer, the second and fourth layers being shffleNetv2 modules, the third and fifth layers being C3-CBAM modules, the main feature extraction network A being used for feature extraction on the fourth-channel reconstructed image, and the fifth layer outputting a feature A1; the main feature extraction network B comprising nine layers, the first layer being a convolutional layer, the second, fourth, sixth and eighth layers being shffleNetv2 modules, and the third, fifth, seventh and ninth layers being C3-CBAM modules, the main feature extraction network B being used for feature extraction on the visible three-channel image of the reconstructed image, and the fifth, seventh and ninth layers outputting features B1, B2 and B3, respectively; 1232) setting the second part of the SC-RetinaNet network as a feature fusion layer, i.e., an original RetinaNet network FPN layer, comprising three layers, i.e., P3, P4 and P5, each layer being connected with a C3-CBAM module, the feature A1 and the feature B3 being input into the P3 layer in the FPN module, the feature A1 and the feature B2 being input into the P4 layer in the FPN module, and the feature A1 and the feature B1 being input into the P5 layer in the FPN module, and the features extracted by the first part being fused; 1233) setting the third part of the SC-RetNet network as three prediction classification regression sub-networks, the classification regression sub-networks performing target classification on the three feature information output by the second part, and the regression sub-networks performing target boundary regression to obtain a target prediction frame; 124) constructing a ShffleNetv2 module; The ShuffleNetv2 is constructed by a convolution module and a down-sampling module, when extracting feature information, group convolution and channel pair model are used for compression, the left branch of the ShuffleNetv2 module adopts one 3*3 DW convolution and one 1*1 convolution, the right branch adopts two 1*1 convolutions and one 3*3 DW convolution, after convolution, the channels are spliced by Concat, the number of output channels is doubled, and finally the output feature layer is channel shuffled; 125) constructing a CBMA attention module; 126) constructing a C3-CBMA attention module, introducing a channel spatial attention module into the C3 module to obtain a C3-CBAM module; The C3-CBAM module performs branch operation on the input feature matrix, the left branch performs convolution and multiple bottleneck layer Bottleneck operations, and the right branch passes through convolution and a channel spatial attention module to enhance the feature extraction capability; the two branch feature layers are spliced by Concat, and then convolution is performed for output; 13) training of the lightweight forest individual tree detection model: inputting the preprocessed data into the lightweight forest individual tree detection model for training; 14) obtaining of the unmanned aerial vehicle remote sensing image to be detected: obtaining the unmanned aerial vehicle remote sensing image to be detected and preprocessing the same; 15) obtaining of the unmanned aerial vehicle remote sensing image forest individual tree detection result: inputting the preprocessed unmanned aerial vehicle remote sensing image into the trained lightweight forest individual tree detection model to obtain the unmanned aerial vehicle remote sensing image forest individual tree detection result.

2. The method according to claim 1, wherein, The data acquisition and preprocessing includes the following steps: 21) using an unmanned aerial vehicle to carry a camera to collect visible images and elevation information of a target area, and splicing to generate DEM, DSM and orthographic image of the target area; 22) performing image superposition on the spliced DEM and DSM data to obtain normalized digital surface model data NDSM, and the normalized digital surface model data is calculated as follows: ； Performing terrain processing on the DEM data, and generating slope data according to a slope calculation formula as follows: ； ； ； Wherein, letters a to i represent the center pixel e and the values of the eight adjacent pixels, the eight adjacent pixels determine the horizontal increment and the vertical increment; x_cellsize represents the length of one pixel in the horizontal direction, y_cellsize represents the length of one pixel in the vertical direction, dz / dx is the change rate of the surface in the horizontal direction from the center pixel, that is, the change rate of the pixel e in the x direction, dz / dy is the change rate of the surface in the vertical direction from the center pixel, that is, the change rate of the pixel e in the y direction, ATAN is the inverse tangent value function of the returned number, and slop_redians is the slope of the center pixel e.

3. The method according to claim 1, wherein, The orthographic image reconstruction module includes the following steps: 31) performing NDSM grid data normalization processing on the normalized digital surface model NDSM data by using the maximum and minimum value formula, as shown in the following formula; Fusing the processed NDSM data and the orthographic image as a band, wherein the NDSM data is a fourth channel reconstruction image, ， is the minimum value of the column in which it is located, is the maximum value of the column in which it is located; 32) Single tree rectangular frame labeling is performed on the orthographic image, and the dataset is produced according to the terrain slope standard according to the DEM slope data, which is divided into two categories of labels, namely the steep and dangerous area category higher than 25 degrees and the other category; each category of dataset includes an orthographic image, an NDSM image, and a single tree rectangular frame label XML file, wherein the orthographic image and the NDSM image correspond to each other, and the image size is 512*512.

4. The method according to claim 1, wherein, The CBMA attention module includes the following steps: 41) Set the channel attention module: The channel attention module generates channel attention weight information using the channel relationship between image features. The module first uses average pooling and max pooling operations to concentrate channel information, generating two different channel context information beams: and , which represent the average pooled features and the max pooled features, respectively; then the two information beams are transferred to the same shared network, which is composed of a multi-layer perceptron (MLP) containing one hidden layer; the output feature vectors are combined using element-wise summation to produce channel attention weight information , , ; 42) Set the CBMA attention module to use average pooling and max pooling operations to concentrate channel information and generate two different channel context information bundles: and which represent the average pooling features and the max pooling features, respectively; The two information beams are transferred to the same shared network composed of a multi-layer perceptron (MLP) containing one hidden layer. Element summation is used to combine the output feature vectors to generate channel attention weight information 、 、 ， wherein is function; The channel attention module is calculated as shown below: ， wherein, is a sigmoid function, R is a feature map, c is a number of channels, MC, W1, W0, F are sub-feature maps, MLP is a multi-layer perceptron, AvgPool is an average pooling operation, MaxPool is a max pooling operation, , respectively represent the average-pooled feature and the max-pooled feature; 43) Set the spatial attention module: The spatial attention module generates spatial attention weight information using the spatial relationship between image features. The spatial dimension feature information focused on by the spatial attention module and the channel feature information of the channel attention together form the channel spatial attention module. The calculation of spatial attention first follows the channel attention average pooling and max pooling operations and connects them to generate an effective feature map, and then passes through a 7*7 convolution layer and an activation function to generate spatial attention weight information ; Channel attention average pooling and max pooling operations and concatenate them to generate an effective feature map; through a 7*7 convolutional layer and an activation function generating spatial attention weight information , The spatial attention module expression is as follows: ， wherein is a sigmoid function, R is a feature map, s is a channel, AvgPool is an average pooling operation, MaxPool is a max pooling operation, , denote an average pooling operation and a max pooling operation on s, respectively.

5. The method according to claim 1, wherein, The training of the lightweight forest single tree detection model includes the following steps: 51) Set the iteration number of training to 100, freeze the training main network for the first 50 times, and perform batch processing Bich_size for 8 times. After 50 times, unfreeze training, Bich_size is 4; select the Adam optimizer, the initial learning rate is 0.0005, and the learning rate decreases with the increase of the iteration number, that is, the learning rate is 0.9 times the original learning rate after two iterations; 52) Input the preprocessed data into the forest single tree detection model, and perform image reconstruction in the first part. The orthographic image and the NDSM image in the dataset are reconstructed to obtain a reconstructed image with a dimension of 512*512*4; 53) Training of the SC-RetinaNet network: First, the main feature extraction network A extracts features from the fourth channel of the reconstructed image. The fifth layer output feature A1 has a dimension of 128*128*64; The main feature extraction network B extracts features from the first three channels of the reconstructed image. The fifth layer output feature B1 has a dimension of 128*128*256, the seventh layer output feature B2 has a dimension of 64*64*512, and the ninth layer output feature B3 has a dimension of 32*32*1024; Then, the above features are fused. Features A1, B3 are input into the P3 layer of the FPN module, features A1, B2 are input into the P4 layer of the FPN module, and features A1, B1 are input into the P5 layer of the FPN module. Finally, the three output features are classified and regressed by the classification and regression subnetwork to obtain the target prediction box. After iteration, the loss value reaches convergence, and a pre-trained model is obtained.

Images