A deep learning-based unmanned aerial vehicle image tree detection method

Abstract

The scheme discloses a kind of unmanned aerial vehicle image tree detection method based on deep learning, the model constructed by the method is based on regression method, introduce attention mechanism and multi-scale feature fusion strategy, simultaneously propose double-level information fusion mode, fuse the preceding method and provide a brand-new tree detection model, this model can improve the generalization ability of complex scene without affecting the prediction speed, improve the recognition accuracy of model, solve the problem that existing model is prone to missing small trees and is poor in robustness in complex background scene.

Description

Technical Field

[0001] This invention belongs to the field of tree detection technology, and in particular relates to a method for constructing a tree detection model from UAV images and a detection method thereof. Background Technology

[0002] Trees are an integral part of forestry ecosystems, playing a vital role in improving human living environments, providing social services, and material production. To advance sustainable tree management, it is necessary to collect basic tree information (such as species and location). Obtaining detailed tree information is fundamental to sustainable tree management. By collecting this information, we can map and represent trees, which facilitates subsequent maintenance and management.

[0003] With the continuous advancement of aerospace engineering technology, unmanned aerial vehicles (UAVs) are being increasingly used to acquire low-altitude surface images due to their advantages such as ultra-high spatial resolution, relatively low operating costs, and near real-time image acquisition. Currently, tree detection methods can be divided into two categories: tree detection methods based on traditional image processing and tree detection methods based on deep learning.

[0004] Tree detection methods based on traditional image processing can be divided into two main categories. One category is pixel-based methods, which generally extract the maximum value of a local region as the tree's center point and combine it with algorithms such as region growing, watershed segmentation, and template matching to detect individual trees. These tree detection methods based on local maximum theory cannot classify trees according to their overall characteristics, and the accuracy of seed point selection in complex backgrounds will also decrease, thus affecting the accuracy of individual tree detection. The other category is object-based tree detection methods, which can better reflect the overall characteristics of trees and improve the accuracy of tree detection by incorporating machine learning algorithms. However, for different scenarios, it is necessary to manually specify the extracted features and limit the corresponding parameters, making it difficult to accurately detect individual trees without prior knowledge.

[0005] Compared to traditional image processing methods, deep learning-based tree detection methods do not rely on manual feature selection. Instead, they automatically learn relevant features from massive amounts of image data according to predefined labels, abstracting low-dimensional texture and color features of the image into high-dimensional semantic features, resulting in good classification performance. Deep learning-based tree detection algorithms mainly fall into two categories: candidate region-based algorithms and regression-based algorithms. Candidate region-based algorithms first acquire regions of interest on the feature map and then fine-tune them. These algorithms have high accuracy but also have drawbacks, requiring significant time to extract candidate regions, which limits the detection speed. Regression-based algorithms eliminate the candidate region extraction stage, directly generating anchor boxes at various locations on the feature map and performing target discrimination. Although this sacrifices some accuracy, the detection and recognition speed is improved. However, if the image resolution is insufficient, these methods may miss small trees. Overall, deep learning-based tree detection methods perform well in scenarios with fixed tree sizes and simple scenes, but their robustness is less ideal in scenarios with complex backgrounds and diverse tree shapes. Summary of the Invention

[0006] The purpose of this invention is to address the aforementioned problems by providing a deep learning-based method for tree detection in UAV images. This method constructs a model that, based on regression methods, introduces an attention mechanism and a multi-scale feature fusion strategy. It also proposes a two-level information fusion approach, combining the aforementioned methods to provide a novel tree detection model. This model can improve the generalization ability in complex scenes and enhance the model's recognition accuracy without affecting prediction speed, thus solving the problems of existing models easily missing small trees and having poor robustness in complex background scenes.

[0007] To achieve the above objectives, the present invention adopts the following technical solutions:

[0008] A deep learning-based method for tree detection in drone imagery, comprising:

[0009] S1. Perform the first size trimming on the input UAV image and perform two-level information fusion at the depth and width levels to output the first feature map;

[0010] S2. Perform downsampling and a second two-level information fusion on the first feature map to output the second feature map;

[0011] S3. Continue downsampling the second feature map and perform a third two-level information fusion to output the third feature map;

[0012] S4. Use the MSPP module to capture multi-scale information from the third feature map and output the fourth feature map;

[0013] S5. Trim the size of the fourth feature map and the second feature map respectively to make the two feature maps the same size and then superimpose them. Then perform double-level information fusion processing again to output the fifth feature map.

[0014] S6. Trim the size of the fifth feature map and the first feature map respectively to make the two feature maps the same size and then superimpose them. Then perform double-level information fusion processing again to output the sixth feature map.

[0015] S7. The sixth feature map is further resized to have the same size as the fifth feature map. Then, it is superimposed on the fifth feature map and subjected to two-level information fusion processing again before the seventh feature map is output.

[0016] S8. The seventh feature map is further resized to have the same size as the fourth feature map. Then, it is superimposed on the fourth feature map and subjected to double-level information fusion processing again to output the eighth feature map.

[0017] S9. The sixth, seventh, and eighth feature maps are processed using the AM Block module to ensure that the network focuses on key regions at each scale;

[0018] S10. Using Detector Head 1, Detector Head 2, and Detector Head 3 respectively, based on the three feature maps processed by the AM Block module, detect small, medium, and large targets respectively, and merge the detection results of the three detector heads to output the final detection result of the model.

[0019] The dimensions mentioned here refer to the width, height, and number of channels of the feature map. Consistent width and height allow for stacking along the channel dimension, while having the same number of channels ensures a consistent proportion of information from different scales.

[0020] Using a regression-based detection algorithm first avoids the time spent on candidate region extraction, ensuring the algorithm's detection speed. Furthermore, the algorithm performs two-level information fusion on feature maps at multiple levels, enhancing the network's ability to detect targets in complex scenes and overcoming the current problem of weak generalization in complex scenarios. MSPP is used to capture multi-scale information, and S5 and S6 are used to fuse information from multiple levels, enabling the network to detect targets of different sizes. Additionally, AM Block modules are used to focus on key regions for large, medium, and small targets, weakening irrelevant background areas, allowing the model to achieve good recognition results for large, medium, and small targets. These operations enable the model to effectively capture small targets without affecting prediction speed, overcoming the current problem of the inability to simultaneously achieve both prediction speed and small target capture.

[0021] In the above-mentioned deep learning-based UAV image tree detection method, in step S1, the input UAV image is a cropped input feature map with a specific size.

[0022] Step S1 includes two Basic Conv1 modules, one FE-1Block module, one MP2 module, and one FE-1Block module. The Basic Conv1 and MP2 modules perform the first size adjustment on the input image, and the two FE-1Block modules perform two-level information fusion at the depth and width levels.

[0023] In the aforementioned deep learning-based UAV image tree detection method, the FE-1Block module includes a Basic Conv2 module. The output of the Basic Conv2 module has two branches. After the first branch is processed by two Conv Block modules, it is further processed by a Basic Conv2 module operation. Then, it is added to the second branch, thus completing the deep-level information fusion. The width-level information fusion is implemented by the Conv Block module. Finally, the fused feature map is processed by a Basic Conv3 module and then output.

[0024] In the above-mentioned deep learning-based UAV image tree detection method, the Conv Block module has two branches. After the first branch is processed by a Basic Conv3 module, the resulting feature map is divided into four parts according to the number of channels, resulting in x1, x2, x3 and x4. No operation is performed on x1 to obtain y1. x2 is processed by the Basic Conv2 module to obtain y2. y2 and x3 are added together and then processed by the Basic Conv2 module to obtain y3. y3 and x4 are added together and then processed by the Basic Conv2 module to obtain y4.

[0025] Feature maps y1, y2, y3 and y4 are superimposed on the channel dimension to complete information fusion at the width level;

[0026] Next, the fused features are processed by a Basic Conv2 module, then skipped to branch 2, and the feature maps of the two parts are added together. Finally, the output is processed by a Basic Conv3 module.

[0027] The Conv Block module uses an inverted structure. It reduces the number of channels by 1×1 convolution, performs a series of operations, and then restores the original number of channels by 1×1 convolution. The change in the number of channels first decreases and then increases, just like an inverted shape. This inverted structure can greatly reduce the network parameters and computational load, ensuring network computation speed while capturing small targets.

[0028] In the above-mentioned deep learning-based UAV image tree detection method, the Basic Conv1 module, Basic Conv2 module, and Basic Conv3 module each include a 2D convolutional layer, a BN layer, and an activation function layer, and the 2D convolutional layers of the three modules have the same or different sizes and strides.

[0029] The introduction of the Basic Conv series of modules can reduce image resolution, decrease the overall computational load of the network, and thus improve the prediction speed of the network.

[0030] In the above-mentioned deep learning-based UAV image tree detection method, steps S2 and S3 both include an MP2 module and an FE-2Block module. The MP2 module downsamples the first feature map / second feature map, and the FE-2Block module performs second / third-level information fusion on the downsampled feature map.

[0031] The FE-2Block module includes a Basic Conv2 module. The output of the Basic Conv2 module has two branches. After the first branch is processed by four Conv Block modules, it is processed by a Basic Conv2 module. Then it is added to the second branch. Finally, the fused feature map is processed by a Basic Conv3 module and output.

[0032] In the above-mentioned deep learning-based UAV image tree detection method, the MSPP module in step S4 divides the input features into two branches. Branch one is further divided into five sub-branches after passing through the Basic Conv3 module: Sub-branch one uses MP3 for feature extraction; Sub-branch two uses MP5 for feature extraction; Sub-branch three uses MP9 for feature extraction; Sub-branch four uses MP13 for feature extraction; Sub-branch five is not processed.

[0033] The five small branches are superimposed in the channel dimension, and the superimposed feature map is processed by a Basic Conv3 module. Then, it is superimposed with branch two in the channel dimension after being processed by a Basic Conv3 module. Finally, it is processed by a Basic Conv3 module to output the fourth feature map.

[0034] In the above-mentioned deep learning-based UAV image tree detection method, in step S5, the second feature map is resized using a BasicConv3 module, the fourth feature map is resized using a Basic Conv3 module and an upsampling module, and the superimposed feature map is fused using a two-level information fusion process using an FE-1Block module.

[0035] In step S6, the first feature map is resized using a Basic Conv3 module, the fifth feature map is resized using a Basic Conv3 module and an upsampling module, and the superimposed feature map is fused using a two-level information fusion process using an FE-1Block module.

[0036] In step S7, the size of the sixth feature map is adjusted by a Basic Conv1 module so that it has the same size as the fifth feature map. Then, a FE-1Block module is used to perform two-level information fusion processing on the superimposed feature map.

[0037] In step S8, the seventh feature map is resized using a Basic Conv1 module to make it the same size as the fourth feature map, and then a FE-2Block module is used to perform two-level information fusion processing on the superimposed feature map.

[0038] In the above-mentioned deep learning-based UAV image tree detection method, in step S9, the AM Block module divides the input features into three branches. The first branch is processed by an AP2 module for feature extraction, and then sequentially processed by a Basic Conv3 module, a Basic Conv2 module, and a Basic Conv3 module, followed by upsampling.

[0039] Then, the output of branch one is added to that of branch two, and the value is normalized to the range (0, 1) by the Sigmoid function.

[0040] Finally, the normalized result is multiplied by branch three.

[0041] In the above deep learning-based UAV image tree detection method, the loss function used to train the model for processing steps S1-S10 is as follows:

[0042]

[0043] Where, λ coord λ is the penalty coefficient for coordinate prediction. noobj This is the penalty coefficient for confidence levels when the target is not included.

[0044] S 2 =S×S represents the number of grids after the input image is divided, and B represents the number of predicted bounding boxes for each grid; These represent whether the j-th bounding box in the i-th grid is responsible for detecting the object; x i y i w i h i These represent the horizontal and vertical coordinates, as well as the width and height of the prediction box, respectively. These represent the horizontal and vertical coordinates, as well as the width and height of the actual bounding box; c i and p represents the confidence scores of the i-th grid cell containing the predicted bounding box and the ground truth bounding box, respectively; i and These represent the probability values ​​of the predicted bounding box and the ground truth bounding box belonging to a certain category, respectively.

[0045] The advantages of this invention are:

[0046] Using FE-1 Block and FE-2 Block to fuse information from both depth and width levels improves the network's anti-interference ability and enhances its ability to detect targets in complex scenes;

[0047] The Conv Block module with an inverted structure can greatly reduce network parameters and computational load compared to traditional operations, ensuring network computation speed while capturing small targets;

[0048] Adding an AM Block module to the detection at each scale helps the network focus on key areas with dense targets while weakening irrelevant background areas, effectively solving the false detection problem faced by small target detection.

[0049] It overcomes the shortcomings of commonly used methods and has the advantages of fast inference speed, high accuracy and robustness of inference results. It can reduce the phenomenon of missing small trees without sacrificing inference speed and can achieve good detection results even in complex background scenes.

[0050] The FE-1Block module, FE-2Block module, skip connections, and MSPP module are used to achieve cross-module preservation of shallow features and increase the receptive field of the network, effectively improving the performance of small target detection. The AM Block module is used to focus on important information in the scene. By proposing, combining, and improving components such as the FE-1Block module, FE-2Block module, skip connections, MSPP module, and AM Block module, a tree detection model is obtained. This model has a fast inference speed, high detection accuracy, and strong generalization ability, which is beneficial to the application of the model in practical engineering. Attached Figure Description

[0051] Figure 1 This is a flowchart illustrating the overall process of the model built for tree detection in UAV imagery in this solution.

[0052] Figure 2 for Figure 1 Structure diagram of the Basic Conv1 module;

[0053] Figure 3 for Figure 1 Structure diagram of the FE-1Block module;

[0054] Figure 4 for Figure 3 Structure diagram of the Conv Block module;

[0055] Figure 5 for Figure 1 Structure diagram of the FE-2Block module;

[0056] Figure 6 for Figure 1 Structure diagram of the MSPP module in China;

[0057] Figure 7 for Figure 1 Structure diagram of the AM Block module. Detailed Implementation

[0058] This invention addresses the problems of existing models missing small trees and poor robustness in complex background scenes. The proposed solution is a tree detection model suitable for UAV imagery. Based on a regression method, this model incorporates an attention mechanism and a multi-scale feature fusion strategy, enabling it to effectively capture small targets without compromising prediction speed, and enhancing its generalization ability in complex scenes.

[0059] The complete process of this method is as follows:

[0060] S1. Acquire data, obtain drone images of the target area, and perform data preprocessing, including stitching, color balancing, cropping, etc., to finally obtain complete high-resolution drone images of the target area.

[0061] S2. Create tree samples by uniformly cropping the target area image to a size of 512×512 and using the Label Img annotation tool to annotate the cropped image with 2D bounding boxes to obtain the location information of the trees.

[0062] S3. Model building: Based on regression methods, design models using structural approaches such as residual structure, multi-scale information fusion strategies, and attention mechanisms.

[0063] S4. Model training: Use the created samples to train the model designed in Step 3.

[0064] S5. Tree Detection: Use a trained model to detect trees in high-resolution UAV imagery of the target area.

[0065] The proposed method in this paper is used for rapid and accurate detection and extraction of tree location information from UAV imagery. It overcomes various shortcomings of commonly used methods and boasts advantages such as fast inference speed, high accuracy, and robustness of inference results. The model construction and training are described in detail below. Figure 1 Here is the overall flowchart of the proposed model:

[0066] Step 1: The cropped drone image with a size of 512×512×3 is used as input. After processing by two Basic Conv1 modules, the width and height of the input image are downsampled from 512×512 to 128×128. At the same time, the three input channels (red, green, and blue) are expanded to 64 channels. Then, it is processed by an FE-1Block module to obtain a feature map after context information fusion. Then, it is downsampled by a Max Pooling (MP) with a stride of 2 and a size of 3×3 (MP2). At this time, the width and height of the feature map are changed from 128×128 to 64×64. Then, it is processed by another FE-1Block module, and the output feature map size is now 64×64×128.

[0067] The specific structure of the Basic Conv1 module is as follows: Figure 2 As shown, the network employs a 2D convolution with a stride of 2 and a size of 3×3, followed by a Batch Normalization (BN) layer and a SiLU activation function. This module reduces the image resolution, thereby decreasing the overall computational cost of the network. The introduction of the Batch Normalization layer enables more stable training, reduces the occurrence of gradient vanishing, and continuously enhances the network's nonlinear fitting ability with the application of this module.

[0068] FE-1Block module, such as Figure 3As shown, this module fuses feature information at both the depth and width levels. First, it passes through a Basic Conv2 module (the only difference from Basic Conv1 is that the convolutional layers in Basic Conv2 have a stride of 1 and a size of 3×3) to reduce the number of channels from 64 to 32, thus reducing the computational load of the network. Then, it splits into two branches. Branch one is processed by two consecutive Conv Block modules, then undergoes another Basic Conv2 module operation, and is subsequently added to branch two, thus completing the fusion of depth-level contextual information. The fused feature map is then subjected to a Basic Conv3 module operation (the only difference from Basic Conv1 is that the convolutional layers in Basic Conv3 have a stride of 1 and a size of 1×1), restoring the number of channels from 32 to the original 64. This inverted structure significantly reduces the network parameters, thereby reducing computational load and improving computational speed.

[0069] The structure of the Conv Block module is as follows: Figure 4 As shown, this module has two branches. Branch one goes through a Basic Conv3 module, at which point the feature map size becomes 128×128×32. Then, the resulting feature map is divided into four equal parts according to the number of channels, resulting in x1, x2, x3, and x4 (at this time, the feature map sizes of x1, x2, x3, and x4 are all 128×128×8). In the figure, y1 is obtained by not performing any operation on x1, y2 is obtained by processing x2 through a Basic Conv2 module, y3 is obtained by adding the two feature maps y2 and x3 and then performing a Basic Conv2 module operation, and y4 is obtained by adding y3 and x4 and then performing a Basic Conv2 module operation.

[0070] The feature maps y1, y2, y3, and y4 are then stacked along the channel dimension, completing the information fusion at the width level. At this point, the feature map size is restored to its original 128×128×32. Next, the fused features are processed by a Basic Conv2 module, then a skip connection is made with branch two to add the two feature maps together. Finally, the output passes through a Basic Conv3 module. After processing by the Conv Block module, the network can significantly increase the model's receptive field with minimal parameter consumption, thereby enhancing the network's ability to detect targets in complex scenes.

[0071] Step 2: Based on the feature map obtained in Step 1, downsampling is performed using Max Pooling with a step size of 2 and a size of 3×3, changing the size of the feature map to 32×32×128. After processing by an FE-2Block module, the output feature map size becomes 32×32×256.

[0072] The structure of the FE-2Block module is as follows: Figure 5 As shown, in general, the structure is similar to the FE-1Block module, but the FE-2Block module has more and deeper layers, and uses four Conv Block modules in succession to extract feature information, resulting in stronger non-linear fitting ability.

[0073] Step 3: Based on the feature map obtained in Step 2, downsampling is performed using a MaxPooling with a step size of 2 and a size of 3×3, changing the size of the feature map to 16×16×256. After processing by an FE-2Block module, the size of the output feature map becomes 16×16×512.

[0074] Step 4: Perform MSPP module operations on the feature map obtained in Step 3 to fuse feature information at different scales. At this time, the size of the output feature map becomes 16×16×256.

[0075] The specific structure of the MSPP module is as follows: Figure 6 As shown, it captures multi-scale information by using multiple Max Pooling operations of different sizes. This module has two branches. The right branch, after passing through a Basic Conv3 module, splits into five smaller branches: Branch 1 uses a 3×3 Max Pooling operation with a stride of 1 for feature extraction; Branch 2 uses a 5×5 Max Pooling operation with a stride of 1; Branch 3 uses a 9×9 Max Pooling operation with a stride of 1; Branch 4 uses a 13×13 Max Pooling operation with a stride of 1; Branch 5 is left unprocessed. These five smaller branches are then stacked along the channel dimension, and the stacked feature map is processed by a Basic Conv3 module before being combined with... Figure 6 The branch on the left side of the middle is processed by a BasicConv3 module and then superimposed on the channel dimension. Finally, it is processed by a Basic Conv3 module to become the final output, realizing the fusion of feature information at different scales.

[0076] Step 5: Perform a Basic Conv3 module operation on the feature map obtained after Step 4, reducing the number of channels from 256 to 128. Then, perform an upsampling operation to double the spatial resolution of the feature map to 32×32×128. Next, perform a Basic Conv3 module operation on the feature map obtained after Step 2, reducing the number of channels from 256 to 128. Then, stack this feature map with the upsampled feature map along the channel dimension, fusing the high-resolution information from the shallower parts of the network layer with the high-semantic information from the deeper parts, enabling the network to detect objects of different sizes. Finally, perform an FE-1 Block module operation, resulting in an output feature map size of 32×32×128.

[0077] Step Six: Perform a Basic Conv3 module operation on the feature map obtained after processing in Step Five, reducing the number of channels from 128 to 64. Then, perform an upsampling operation on it, resulting in a spatial resolution of 64×64×64 for the feature map. Next, perform a Basic Conv3 module operation on the feature map obtained after processing in Step One, reducing the number of channels from 128 to 64. Then, stack it with the upsampled feature map in the channel dimension. Finally, perform an FE-1 Block module operation on it, resulting in an output feature map size of 64×64×64.

[0078] Step 7: Perform AM Block module operations on the feature map obtained in Step 6, and then use the detection head Head1 to detect the target. This layer is mainly responsible for detecting small targets. Head1 is essentially a convolutional layer. After the Head1 operation, the feature map size becomes 64×64×18. This layer sets 3 anchor boxes for each sampling point on the feature map, allowing the network to automatically learn and regress the target's position information, thereby achieving the detection of the target object.

[0079] The structure of the AM Block module is as follows: Figure 7As shown, this module has three branches. The middle branch performs feature extraction using an Average Pooling operation with a stride of 2 and a size of 3×3. Then, it sequentially passes through a Basic Conv3 module (which compresses the number of channels, halving them), a Basic Conv2 module, and another Basic Conv3 module (which expands the number of channels to their original size). Next, it undergoes upsampling to restore the feature map resolution to its pre-Average Pooling size. This is then added to the feature map of the branch on the right, and the value is normalized to the range (0, 1) using the Sigmoid function. Finally, it is multiplied by the feature map of the branch on the left.

[0080] The AM Block module is designed to help the network focus on key areas with dense targets while weakening irrelevant background areas, effectively solving the false detection problem faced by small target detection.

[0081] Step 8: The feature map obtained in Step 6 is processed by a Basic Conv1 module to halve its width and height to 32×32. At the same time, the number of channels of the two feature maps is kept the same by manual settings. Then, it is superimposed on the feature map obtained in Step 5 in the channel dimension, and then processed by an FE-1Block module.

[0082] Step 9: Next, the AM Block module is used to process the feature map obtained in Step 8, so that the network can focus on important information in the image when extracting features. Then, the detection head Head2 is used to detect medium-sized targets. At this time, the feature map size is 32×32×18.

[0083] Step 10: The feature map obtained in Step 8 is processed by a Basic Conv1 module, and then superimposed on the feature map obtained in Step 4 in the channel dimension. After that, it is processed by an FE-2Block module, followed by an AMBlock module. Finally, the detection head Head3 is used to detect large targets. At this time, the feature map size is 16×16×18.

[0084] Step 11: Calculate the loss function and update the weights of each parameter in the network using backpropagation and gradient descent. The loss function of the model is shown in Equation (1), which mainly measures the coordinates of the target box, the confidence level, and the classification category. The mean square error (MSE) loss function is used in the target box regression process, and the binary cross-entropy loss function is used to measure the confidence level and the category.

[0085]

[0086] Where, λ coord λ is the penalty coefficient for coordinate prediction. noobj This is the penalty coefficient for confidence levels when the target is not included.

[0087] S 2 =S×S represents the number of grids after the input image is divided, and B represents the number of predicted bounding boxes for each grid; These represent whether the j-th bounding box in the i-th grid is responsible for detecting the object; x i y i w i h i These represent the horizontal and vertical coordinates, as well as the width and height of the prediction box, respectively. These represent the horizontal and vertical coordinates, as well as the width and height of the actual bounding box; c i and p represents the confidence scores of the i-th grid cell containing the predicted bounding box and the ground truth bounding box, respectively; i and These represent the probability values ​​of the predicted bounding box and the ground truth bounding box belonging to a certain category, respectively.

[0088] After the model is built and trained, the area to be detected is photographed using a drone. The drone images are then preprocessed and cropped to a size of 512×512×3. Finally, the cropped drone images are input into the trained model. Steps one through ten are used to process the input drone images to obtain the detection results for small, medium, and large targets. Finally, the detection results of the three detection heads are merged to output the final detection result of the model.

[0089] The specific embodiments described herein are merely illustrative of the spirit of the invention. Those skilled in the art to which this invention pertains may make various modifications or additions to the described specific embodiments or use similar methods to substitute them, without departing from the spirit of the invention or exceeding the scope defined by the appended claims.

Claims

1.A method for detecting trees in images captured by a drone based on deep learning, the method comprising: The method comprises: S1. First size correction is performed on the input UAV image, and two-level information fusion is performed at the depth and width levels to output a first feature map; S2. The first feature map is down-sampled and second two-level information fusion is performed to output a second feature map; S3. The second feature map is continuously down-sampled, and third two-level information fusion is performed to output a third feature map; S4. The third feature map is subjected to multi-scale information capture using an MSPP module to output a fourth feature map; S5. The fourth feature map and the second feature map are respectively subjected to size correction to correct the two feature maps to the same size and are subjected to superposition processing, and then two-level information fusion processing is performed again to output a fifth feature map; S6. The fifth feature map and the first feature map are respectively subjected to size correction to correct the two feature maps to the same size and are subjected to superposition processing, and then two-level information fusion processing is performed again to output a sixth feature map; S7. The sixth feature map is continuously subjected to size correction to have the same size as the fifth feature map, and then the sixth feature map is subjected to superposition processing with the fifth feature map, and two-level information fusion processing is performed again to output a seventh feature map; S8. The seventh feature map is continuously subjected to size correction to have the same size as the fourth feature map, and then the seventh feature map is subjected to superposition processing with the fourth feature map, and two-level information fusion processing is performed again to output an eighth feature map; S9. The sixth feature map, the seventh feature map, and the eighth feature map are respectively processed using an AM Block module to enable the network to focus on the key area at each scale; S10. A detection head one, a detection head two, and a detection head three are used to detect small targets, medium targets, and large targets based on the three feature maps processed by the AM Block module, respectively, and the detection results of the three detection heads are combined to output the final detection result of the model. 2.The deep learning-based unmanned aerial vehicle image tree detection method of claim 1, wherein, In step S1, the input UAV image is a cropped input feature map with a specific size; Step S1 comprises, in sequence, two Basic Conv1 modules, an FE-1 Block module, an MP2 module, and an FE-1 Block module. The input image is subjected to first size correction by the Basic Conv1 module and the MP2 module, and two-level information fusion at the depth and width levels is performed by the two FE-1 Block modules. 3.The deep learning-based unmanned aerial vehicle image tree detection method of claim 2, wherein, The FE-1 Block module comprises a Basic Conv2 module. The output of the Basic Conv2 module has two branches. After the first branch is continuously processed by two Conv Block modules, a Basic Conv2 module operation is performed on the first branch, and then the first branch is added to the second branch. At this point, the depth-level information fusion is completed, the width-level information fusion is realized by the Conv Block module, and finally the fused feature map is output after a Basic Conv3 module operation. 4.The deep learning-based unmanned aerial vehicle image tree detection method of claim 3, wherein, The Conv Block module has two branches, after the first branch is operated by a Basic Conv3 module, the feature map obtained is evenly divided into four parts according to the channel number to obtain x1, x2, x3 and x4, x1 is not operated to obtain y1, x2 is processed by a Basic Conv2 module to obtain y2, y2 and x3 are added and then operated by a Basic Conv2 module to obtain y3, y3 and x4 are added and then operated by a Basic Conv2 module to obtain y4; The feature maps y1, y2, y3 and y4 are stacked in the channel dimension, so as to complete the information fusion in the width level; Then, the fused features are processed by a Basic Conv2 module, and then are connected with the second branch by jumping, the feature map processed by the Basic Conv2 module is added to the feature map of the second branch, and then is output after being operated by a Basic Conv3 module. 5.The deep learning-based unmanned aerial vehicle image tree detection method of claim 4, wherein, The Basic Conv1 module, the Basic Conv2 module and the Basic Conv3 module each sequentially include a 2D convolution layer, a BN layer and an activation function layer, and the 2D convolution layers of the three have the same or different sizes and steps. 6.The deep learning-based unmanned aerial vehicle image tree detection method of claim 5, wherein, Steps S2 and S3 each include an MP2 module and an FE-2 Block module, the first feature map / second feature map is down-sampled by the MP2 module, and the down-sampled feature map is processed by the FE-2 Block module for second / third double-level information fusion; The FE-2 Block module includes a Basic Conv2 module, the output of the Basic Conv2 module has two branches, the first branch is continuously processed by four Conv Block modules, then is operated by a Basic Conv2 module, then is added to the second branch, and finally the fused feature map is output after being operated by a Basic Conv3 module. 7.The deep learning-based unmanned aerial vehicle image tree detection method of claim 6, wherein, The MSPP module in step S4 divides the input feature into two branches, the first branch is divided into five small branches after being operated by a Basic Conv3 module: the first small branch uses MP3 to extract features; the second small branch uses MP5 to extract features; the third small branch uses MP9 to extract features; The fourth small branch uses MP13 to extract features; The fifth small branch is not processed; The five small branches are stacked in the channel dimension, and the stacked feature map is operated by a Basic Conv3 module, then is stacked with the second branch after being operated by a Basic Conv3 module, and finally is output after being processed by a Basic Conv3 module. 8.The deep learning-based unmanned aerial vehicle image tree detection method of claim 7, wherein, In step S5, the second feature map is size-adjusted by a Basic Conv3 module, the fourth feature map is size-adjusted by a Basic Conv3 module and an up-sampling module, and the stacked feature map is processed by a FE-1 Block module for double-level information fusion processing; In step S6, the first feature map is size-adjusted by a Basic Conv3 module, the fifth feature map is size-adjusted by a Basic Conv3 module and an up-sampling module, and the stacked feature map is subjected to double-level information fusion processing by an FE-1 Block module; In step S7, the sixth feature map is size-adjusted by a Basic Conv1 module to have the same size as the fifth feature map, and then the stacked feature map is subjected to double-level information fusion processing by an FE-1 Block module; In step S8, the seventh feature map is size-adjusted by a Basic Conv1 module to have the same size as the fourth feature map, and then the stacked feature map is subjected to double-level information fusion processing by an FE-2 Block module. 9.The deep learning-based unmanned aerial vehicle image tree detection method of claim 8, wherein, In step S9, the input feature is divided into three branches by an AM Block module, the first branch is subjected to feature extraction by an AP2 module, and then sequentially subjected to operations of a Basic Conv3 module, a Basic Conv2 module and a Basic Conv3 module, and then subjected to an up-sampling operation; Then, the output of the first branch is added to the second branch, and the value is normalized to (0, 1) by a Sigmoid function; Finally, the normalized result is multiplied by the third branch. 10.The deep learning-based unmanned aerial vehicle image tree detection method of claim 1, wherein, The loss function used for training the model for processing steps S1-S10 is as follows: wherein, is a penalty coefficient for coordinate prediction, is a penalty coefficient for confidence without target location; This indicates the number of grid cells after the input image is divided. This indicates the number of target boxes predicted for each grid cell; , They represent the first The first grid Does each bounding box handle the detection of the target object? , , , These represent the horizontal and vertical coordinates, as well as the width and height of the prediction box, respectively. , , , These represent the horizontal and vertical coordinates, as well as the width and height of the actual bounding box, respectively. and They represent the first Each grid contains the confidence scores of the predicted bounding boxes and the ground truth bounding boxes; and These represent the probability values ​​of the predicted bounding box and the ground truth bounding box belonging to a certain category, respectively.

Images