A lightweight unmanned aerial vehicle aerial image target detection method

By designing the ERFNet backbone network and the ERF-PAN path aggregation network, and combining them with the lightweight detection head ERF-Head, the problems of occlusion and hardware resource limitations in small target detection in UAV aerial images were solved, achieving efficient and real-time target recognition.

CN116597331BActive Publication Date: 2025-12-05BEIJING UNION UNIVERSITY
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
CN202310639688.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-06-01
Publication Date
2025-12-05
Estimated Expiration
2043-06-01

AI Technical Summary

Technical Problem

Existing target detection methods for UAV aerial images suffer from occlusion and hardware resource limitations when detecting small targets, making it difficult to achieve a balance between real-time performance and accuracy. Furthermore, existing lightweight methods have significant computational overhead.

Method used

A lightweight backbone network, ERFNet, is designed using an Effective Receptive Field (ERF) module. It is combined with an Effective Receptive Field Path Aggregation Network (ERF-PAN) and a lightweight detection head (ERF-Head) to expand the receptive field and reduce parameters. The feature extraction capability is improved through residual connections and dilated convolutional blocks (D-blocks).

Benefits of technology

While reducing parameters and computational load, it improves the performance of target detection in UAV aerial images, achieving real-time high-precision target recognition, which is suitable for natural disaster relief and military reconnaissance.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN116597331B_ABST
    Figure CN116597331B_ABST
Patent Text Reader

Abstract

The present application provides a kind of lightweight unmanned aerial vehicle aerial image target detection method, including obtaining original unmanned aerial vehicle aerial image, also include the following steps: the original unmanned aerial vehicle aerial image is input into effective receptive field network, the size of original unmanned aerial vehicle aerial image is adjusted, and adjusted unmanned aerial vehicle aerial image is generated;Unmanned aerial vehicle aerial image is input into effective receptive field module to carry out down-sampling and feature extraction;Repeat 5 times, output {C3,C4,C5} scale;{C3,C4,C5} scale is input into effective receptive field path aggregation network to carry out feature fusion, and output {P3,P4,P5} feature map;{P3,P4,P5} feature map is input into effective receptive field detection head to carry out detection, and 1×1 convolution is used to output prediction result.In the case where local details are maintained, the receptive field of the convolution kernel module can be increased, and a lightweight backbone is designed using the effective receptive field module, effectively expanding the receptive field of the backbone, and avoiding the addition of additional context modules to expand the network receptive field.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of UAV image processing technology, and in particular to a lightweight UAV aerial image target detection method. Background Technology

[0002] Drones, with their powerful maneuverability, are widely used in military, agriculture, and environmental monitoring fields. Military drones can be used for battlefield reconnaissance and border patrols. In agriculture, drones can spray pesticides precisely, efficiently, and environmentally friendly. In environmental monitoring, they are used to observe air conditions, collect environmental data of the surrounding area, and monitor factory emissions in real time. Computer vision technology provides technical support for the development of drones. Object detection is a fundamental task of computer vision, capable of identifying all targets and their locations in drone aerial images. In recent years, deep learning based on convolutional neural networks has become the mainstream method for object detection. Two-stage methods, such as the R-CNN series, typically have high detection accuracy but slow inference speed. Single-stage algorithms, such as the SSD series and YOLO series, have fast inference speed but slightly lower accuracy. Although these methods have achieved advanced performance on natural scene images (MS COCO), they fail to achieve satisfactory detection results when detecting objects in drone aerial images or videos.

[0003] The following challenges exist in target detection in drone aerial photography scenarios: (1) Unlike natural scene images, due to the characteristics of drone images taken at low altitudes, the images contain many densely packed small targets. Distant targets are very small, and dense scenes can cause occlusion, making target detection more difficult. (2) Due to hardware resource limitations, general target detection algorithms reduce the size of high-resolution drone aerial images (e.g., 320×320 or 640×640) and input them into the model, which can lead to missed detection of small targets. (3) Target detection in drone aerial images needs to meet real-time requirements, so a trade-off between network parameters and accuracy is necessary, and there is an irreconcilable contradiction between the two.

[0004] Chinese patent application CN115810157A discloses a drone target detection method based on lightweight feature fusion. First, it redesigns the neck feature extraction module of YOLOv4-tiny using Depthically Separable Convolution (DSC) and Coordinate Attention (CA). Then, it uses an SPPF feature extraction module to increase the receptive field information of the extracted features while effectively maintaining the model's computational lightweight nature. Finally, it uses a Decoupled Head detection module to give the extracted feature information stronger spatial coherence. The disadvantages of this method are: ① The CA module introduces additional computational overhead, increasing network latency and leading to a decrease in detection speed; ② While SPPF effectively increases the receptive field information of the network, it also adds additional parameters and computational complexity.

[0005] Chinese patent application CN 112101113A discloses a lightweight method for small target detection in UAV images. The method processes each frame of the input UAV image video in a time sequence, including: scaling the image and inputting it into a Revised MobileNetV2 feature extractor to output a feature map; inputting the feature map into a synchronous upsampling and detection module to obtain the target center point position and corresponding scale, thus obtaining the bounding boxes of all predicted targets within the frame; after processing all frames of the video, applying fast sequence non-maximum suppression to the prediction results of all frames, and outputting the target detection result of the video. The drawback of this method is that while adding a binary classification center point target-oriented branch reduces the false detection rate of small targets to some extent, it increases computational overhead and consumes more hardware resources. Summary of the Invention

[0006] To address the aforementioned technical issues, this invention proposes a lightweight target detection method for UAV aerial images, which utilizes an ERF module to increase the receptive field of convolutional kernels while preserving local details. Furthermore, a lightweight backbone is designed using the ERF module, effectively expanding the receptive field of the backbone and avoiding the need to add additional context modules to broaden the network's receptive field.

[0007] The purpose of this invention is to provide a lightweight method for target detection in drone aerial images, including acquiring original drone aerial images and the following steps:

[0008] Step 1: Input the original drone aerial image into the Effective Receptive Field Network (ERFNet), adjust the size of the original drone aerial image, and generate an adjusted drone aerial image.

[0009] Step 2: Input the adjusted UAV aerial image into the Effective Receptive Field (ERF) module with a stride of 2 for downsampling and feature extraction;

[0010] Step 3: Repeat the method in Step 2 5 times to output three scales {C3, C4, C5};

[0011] Step 4: Input the three scales {C3, C4, C5} into the Effective Receptive Field Path Aggregation Network ERF-PAN for feature fusion, and output three feature maps {P3, P4, P5}.

[0012] Step 5: Input the three feature maps {P3, P4, P5} into the effective receptive field detection head ERF-Head for detection, and use 1×1 convolution to output the prediction result.

[0013] Preferably, step 1 involves generating an adjusted drone aerial image of size 320×320×32 from the original drone aerial image, which is 640×640×3.

[0014] In any of the above schemes, it is preferred that the size of the downsampled UAV aerial image becomes 160×160×32.

[0015] In any of the above schemes, step 2 preferably includes feature extraction using the effective receptive field module (ERF module) with a stride of 1, and the size of the image after feature extraction remains unchanged.

[0016] In any of the above schemes, it is preferred that when the stride of the effective receptive field module (ERF module) is 1, the number of input channels is divided into two parts. The left branch is used for residual connection to prevent gradient vanishing and reduce overfitting, while the right branch expands the network's receptive field by stacking n dilated convolutional blocks (D blocks) without losing local details.

[0017] In any of the above schemes, it is preferred that when the stride of the effective receptive field module (ERF module) is 2, the left branch learns features through dilated convolutional blocks (D blocks), the right branch learns features through maxpooling, the features learned by the left and right branches are concatenated, and the number of channels is restored using a 1×1 convolution.

[0018] In any of the above schemes, it is preferred that the sizes of the three scales {C3, C4, C5} are 80×80×128, 40×40×256, and 20×20×512, respectively.

[0019] In any of the above solutions, step 4 preferably includes the following sub-steps:

[0020] Step 41: Apply a 1×1 convolution to scale C5 to obtain scale C5′. Upsample scale C5′ to 40×40×256, and then fuse it with scale C4 for features.

[0021] Step 42: Use the effective receptive field module (ERF module) with a stride of 1 to extract features and obtain a scale C4_upsample with a size of 40×40×512;

[0022] Step 43: Apply a 1×1 convolution to the scale C4_upsample to obtain scale C4′, upsample scale C4′ to 80×80×128, and then fuse it with scale C3 for features;

[0023] Step 44: Use the effective receptive field module (ERF module) with a stride of 1 to extract features and obtain feature map P3. The size of feature map P3 is 80×80×128.

[0024] Step 45: Feature map P3 is downsampled and fused with scale C4′ using the effective receptive field module ERF module with a stride of 2, and then feature is extracted using the effective receptive field module ERF module with a stride of 1 to obtain feature map P4. The size of feature map P4 is 40×40×256.

[0025] Step 46: Feature map P4 is downsampled by the effective receptive field module ERF module with a stride of 2 and fused with scale C5′. Then, feature map P5 is extracted by the effective receptive field module ERF module with a stride of 1. The size of feature map P5 is 20×20×512.

[0026] Step 47: Output three feature maps {P3, P4, P5}, with sizes of 80×80×128, 40×40×256, and 20×20×512, respectively.

[0027] In any of the above schemes, step 5 preferably includes using 1×1 convolution to compress the number of channels {P3,P4,P5} to keep the number of channels consistent.

[0028] In any of the above schemes, step 5 preferably includes stacking two dilated convolutional blocks (Dblocks) to improve the feature extraction capability of the detection head.

[0029] This invention proposes a lightweight target detection method for UAV aerial images, which can effectively improve detection performance while reducing parameters. In natural disaster relief operations, rescuers can operate UAVs to transmit aerial images in real time and generate panoramic maps, providing situational awareness at the disaster site. Attached Figure Description

[0030] Figure 1 This is a flowchart of a preferred embodiment of the lightweight UAV aerial image target detection method according to the present invention.

[0031] Figure 2 This is a schematic diagram of the YOLO-ERF structure, representing a preferred embodiment of the lightweight UAV aerial image target detection method according to the present invention.

[0032] Figure 3 This is a diagram of the effective receptive field (ERF) module of another preferred embodiment of the lightweight UAV aerial image target detection method according to the present invention.

[0033] Figure 4 For example, the lightweight UAV aerial image target detection method according to the present invention Figure 3 A schematic diagram of the YOLO-ERF-Head structure in the embodiment shown. Detailed Implementation

[0034] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0035] Example 1

[0036] like Figure 1 As shown, a lightweight target detection method for UAV aerial images is implemented by step 100, which acquires the original UAV aerial image.

[0037] In step 110, the original UAV aerial image is input into the Effective Receptive Field Network (ERFNet), and the size of the original UAV aerial image is adjusted to 640×640×3. After passing through a Focus module, the adjusted UAV aerial image with a size of 320×320×32 is generated.

[0038] In step 120, the adjusted UAV aerial image is input into the effective receptive field module ERFmodule with a stride of 2 for downsampling. The size of the adjusted UAV aerial image after downsampling becomes 160×160×32. Feature extraction is then performed using the effective receptive field module ERFmodule with a stride of 1, and the image size remains unchanged after feature extraction.

[0039] When the stride of the effective receptive field module (ERF module) is 1, the number of input channels is divided into two parts. The left branch is used for residual connections to prevent gradient vanishing and reduce overfitting, while the right branch expands the network's receptive field by stacking n dilated convolutional blocks (Dblocks) without losing local details.

[0040] When the stride of the effective receptive field module (ERF module) is 2, the left branch learns features through dilated convolutional blocks (Dblock), and the right branch learns features through maxpooling. The features learned by the left and right branches are concatenated, and the number of channels is restored using a 1×1 convolution.

[0041] Execute step 130 to determine if the method of step 120 has been repeated 5 times. If it has not been repeated 5 times, then execute step 120 again; if it has been repeated 5 times, then execute step 140 to output three scales {C3, C4, C5}, where the sizes of the three scales {C3, C4, C5} are 80×80×128, 40×40×256, and 20×20×512, respectively.

[0042] Step 150 involves inputting the three scales {C3, C4, C5} into the Effective Receptive Field Path Aggregation Network (ERF-PAN) for feature fusion, outputting three feature maps {P3, P4, P5}, including the following sub-steps:

[0043] In step 151, a 1×1 convolution is applied to scale C5 to obtain scale C5′. Scale C5′ is then upsampled to become 40×40×256, and then features are fused with scale C4.

[0044] Perform step 152, using the effective receptive field module (ERF module) with a stride of 1 to extract features and obtain a scale C4_upsample of size 40×40×512;

[0045] Step 153 is executed, and a 1×1 convolution is applied to the scale C4_upsample to obtain scale C4′. The scale C4′ is then upsampled to become 80×80×128, and then features are fused with scale C3.

[0046] Execute step 154, use the effective receptive field module (ERF module) with a stride of 1 to extract features and obtain feature map P3, the size of feature map P3 is 80×80×128;

[0047] In step 155, feature map P3 is downsampled and fused with scale C4′ using the effective receptive field module ERF module with a stride of 2, and then feature is extracted using the effective receptive field module ERF module with a stride of 1 to obtain feature map P4. The size of feature map P4 is 40×40×256.

[0048] In step 156, feature map P4 is downsampled and fused with scale C5′ using the effective receptive field module ERF module with a stride of 2, and then feature is extracted using the effective receptive field module ERF module with a stride of 1 to obtain feature map P5. The size of feature map P5 is 20×20×512.

[0049] Execute step 157 to output three feature maps {P3, P4, P5}, with sizes of 80×80×128, 40×40×256, and 20×20×512, respectively.

[0050] In step 160, the three feature maps {P3, P4, P5} are input into the effective receptive field detection head ERF-Head for detection. A 1×1 convolution is used to compress the number of channels in {P3, P4, P5} to keep the number of channels consistent. Two dilated convolutional blocks (D blocks) are stacked to improve the feature extraction capability of the detection head. A 1×1 convolution is used to output the prediction result.

[0051] Example 2

[0052] This invention proposes a novel Effective Receptive Field (ERF) module that preserves local details while increasing the receptive field of convolutional kernels. The backbone is redesigned using the ERF module. A new backbone network, ERFNet, is proposed, which expands the network's receptive field while reducing network parameters, without requiring additional context modules to expand the backbone's receptive field. Simultaneously, the Path Aggregation Network (PAN) structure is improved using the ERF module, named the Effective Receptive Field Path Aggregation Network ERF-PAN. This improves the network's feature extraction capabilities and reduces parameters. Finally, a novel lightweight detection head, ERF-Head, suitable for UAV aerial photography scenarios, is designed.

[0053] This invention first proposes an ERF module that increases the receptive field of convolutional kernels while preserving local details. Simultaneously, a lightweight backbone is designed using the ERF module, effectively expanding the receptive field of the backbone and avoiding the need to add additional context modules to broaden the network's receptive field. Furthermore, the ERF module is used to improve the PAN structure, reducing the number of parameters while enhancing the network's feature extraction capabilities. Finally, a novel lightweight detection head is designed for small targets in complex backgrounds. The YOLO-ERF structure is as follows: Figure 2 As shown. The backbone is ERFNet, which inputs the extracted feature maps {C3, C4, C5} into the neck region. The neck region is the Effective Receptive Field Path Aggregation Network ERF-PAN, which fuses the three input feature maps and outputs three feature maps {P3, P4, P5}.

[0054] 1. Effective receptive field module

[0055] Residual connection structures and dense connection structures are widely used in convolutional neural networks. Residual connection structures alleviate the gradient vanishing problem caused by increasing the depth of neural networks. Dense connection structures enhance feature propagation and reuse features more effectively, exhibiting good performance in object detection tasks. This invention proposes a novel Effective Receptive Field (ERF) module for the model of this invention by using residual connections and dense connections. It not only preserves local details but also increases the receptive field of the convolutional kernel.

[0056] Figure 3 (a) illustrates the dilated convolutional block D of the present invention. First, 1×1 convolutions are used to increase nonlinearity and cross-channel information interaction. Then, during grouped convolutions, the dilated convolutional block D uses one dilation rate (d1) for one half of the groups and another dilation rate (d2) for the other half. This effectively expands the receptive field of the convolutional kernel while saving parameters. Furthermore, the SE module emphasizes channel information and can effectively weight network channels, achieving better performance with only a small increase in computation. Therefore, the dilated convolutional block D utilizes the SE module to enhance the features of important channels and weaken the features of unimportant channels. Finally, 1×1 convolutions are used to increase nonlinearity without sacrificing resolution. The number of groups in the D block g = 4, d1 is set to 1 to preserve local details, and the reduction ratio of the SE is set to 4.

[0057] The ERF module proposed in this invention is as follows: Figure 3 As shown. When the step size is 1 ( Figure 3(b) divides the input channels into two parts. The left branch is used for residual connections to prevent gradient vanishing and reduce overfitting, while the right branch expands the network's receptive field without losing local details by stacking n dilated convolutional blocks (D blocks). To balance accuracy and parameters, in YOLO-ERF, the ERF module stacks 3 dilated convolutional blocks (D blocks). Figure 3 (c) The structure is downsampled. The left branch learns features through dilated convolutional blocks (D blocks), and the right branch learns features through maxpooling. The features learned by the left and right branches are then concatenated, and finally, a 1×1 convolution is used to restore the number of channels.

[0058] 2. ERFNet and Efficient Receptive Field Path Aggregation Network ERF-PAN

[0059] This invention uses the proposed ERF module to construct the backbone. The backbone is named ERFNet, as shown in Table 1. In Table 1, *Channels indicates the number of output channels, and the number of input channels is inferred from the previous module. *Repeat indicates the number of times each module is used. For YOLO-ERF, a 32-channel Focus module with a stride of 1 is used first, which reduces the number of parameters without losing information. Then, an ERF module with a stride of 2 is used, followed by another ERF module with a stride of 1. By stacking four ERF modules, the resolution of the input image is downsampled to 1 / 32, resulting in a final number of 512 channels. The number of input channels is inferred from the previous module.

[0060]

[0061]

[0062] Table 1 ERFNet Operation Procedure

[0063] The Path Aggregation Network (PAN) structure is widely used in the neck regions of YOLOv4, YOLOv5, YOLOX, and YOLOv7, therefore this invention uses the PAN structure. This invention names the neck region constructed using the ERF module the Effective Receptive Field Path Aggregation Network ERF-PAN, and obtains multi-layer feature maps through top-down and bottom-up feature fusion. The specific structure is as follows... Figure 2 As shown.

[0064] 3. Effective sensing field detection head ERF-Head

[0065] The detection head is used to handle target location regression and classification tasks. In UAV aerial images, the background of densely packed small targets is complex and diverse. To improve the feature extraction capability of the detection head, this paper adopts the proposed dilated convolutional block (Dblock) to focus on regions of interest. The dilated convolutional block (Dblock) is a lightweight module that can be easily embedded into the detection head, while increasing the receptive field of the convolutional kernels while preserving local details. Therefore, this module is used to improve the feature representation of the detection head. The structure of the detection head is as follows: Figure 4 As shown.

[0066] A 1×1 convolution is used to compress the number of channels {P3, P4, P5} to maintain a consistent channel count. Then, two dilated convolutional blocks (D blocks) are stacked to improve the feature extraction capability of the detector head. Finally, a 1×1 convolution is used to output the prediction result.

[0067] Example 3

[0068] The lightweight UAV aerial image target detection method proposed in this patent has broad prospects, effectively improving detection performance while reducing parameters. In natural disaster relief operations, rescuers can control UAVs to transmit aerial images in real time and generate panoramic views, providing situational awareness at the disaster site. Timely understanding of the disaster situation in the affected area and real-time information acquisition enable rescue work, compensating for the shortcomings of manual search and rescue, improving search and rescue efficiency, and reducing casualties. Simultaneously, using UAVs to photograph large areas of farmland and land allows for a comprehensive understanding of various indicators such as crop growth environment and cycle from aerial images and video data. From irrigation to soil variation, and even to diseases, pests, and bacterial invasions invisible to the naked eye, problematic areas can be identified, facilitating better field management for farmers. Furthermore, in the military field, UAVs serve as an auxiliary means of targeting sensitive targets, enabling real-time tracking of enemy mobile forces and moving targets. Real-time data transmission through aerial photography can provide intelligence support for precision strike weapons when necessary. This patent was experimentally compared with other state-of-the-art single-stage detectors on the UAV dataset Visdrone2019, and the results are shown in Table 2. AP0.5 (AP50) represents the average accuracy of all target categories when the IOU threshold is 0.5. AP0.5:0.95 (AP) represents the average accuracy of all target categories when the IOU threshold is between 0.5 and 0.95, with a step size of 0.05. Higher values ​​for AP0.5 and AP0.5:0.95 are better, while lower parameter and computational costs are better.

[0069]

[0070] Table 2 compares the accuracy of different target detectors on VisDrone 2019.

[0071] As shown in Table 2, YOLO-ERF-T reduces parameters by 58.8% and computation by 53.4%, and achieves 2.7% higher AP and 5.3% higher AP50 than YOLOX-S. Simultaneously, it reduces parameters by 48.6% and computation by 24.2%, achieving 1.9% higher AP and 4.2% higher AP50 than YOLOv5-S. Experimental results demonstrate that this patent, while reducing parameters and computation, achieves better detection performance than other state-of-the-art single-stage detectors in UAV aerial imagery. Therefore, this patent has high application value in target detection in UAV aerial imagery.

[0072] Example 4

[0073] The method described in this patent is a lightweight, single-stage target detection network for drone aerial imagery. The specific implementation steps of this method are as follows:

[0074] 1. Input the original drone aerial image into ERFNet, which is 640×640×3 in size. After passing through a Focus module, its size is reduced to 320×320×32.

[0075] 2. Then, downsampling is performed using an ERF module with a step size of 2, resulting in a size of 160×160×32;

[0076] Then, an ERF module with a stride of 1 is used for feature extraction, while keeping the size unchanged.

[0077] 3. Repeat step 2 five times. The final output of ERFNet is three scales {C3, C4, C5}, with sizes of 80×80×128, 40×40×256, and 20×20×512, respectively.

[0078] 4. Then, {C3, C4, C5} are input into the Effective Receptive Field Path Aggregation Network (ERF-PAN) for feature fusion. A 1×1 convolution is applied to C5 to obtain C5′, which is then upsampled to a size of 40×40×256 and fused with C4. Afterwards, an ERF module with a stride of 1 is used for feature extraction to obtain C4_upsample, which has a size of 40×40×512.

[0079] 5. Apply a 1×1 convolution to C4_upsample to obtain C4′. Upsample C4′ to make its size 80×80×128, and then fuse it with C3 for features. Then use an ERF module with a stride of 1 to extract features to obtain P3, which is now 80×80×128.

[0080] 6. P3 is downsampled and fused with C4′ using an ERF module with a stride of 2, and then P4 is obtained by feature extraction using an ERF module with a stride of 1. At this time, the size is 40×40×256.

[0081] 7. P4 is downsampled and fused with C5′ using an ERF module with a stride of 2, and then P5 is obtained by feature extraction using an ERF module with a stride of 1. At this time, the size is 20×20×512.

[0082] 8. ERF-PAN outputs three feature maps {P3, P4, P5} through feature fusion, with sizes of 80×80×128 respectively.

[0083] 40×40×256, 20×20×512.

[0084] Finally, {P3, P4, P5} are input into the ERF-Head for detection. A 1×1 convolution is used to compress the number of channels in {P3, P4, P5} to maintain a consistent channel count. Then, two dilated convolutional blocks (D blocks) are stacked to improve the feature extraction capability of the detection head. Finally, a 1×1 convolution is used to output the prediction result.

[0085] To better understand this invention, specific embodiments have been described in detail above, but these are not intended to limit the invention. Any simple modifications made to the above embodiments based on the technical essence of this invention still fall within the scope of this invention. Each embodiment in this specification focuses on its differences from other embodiments; similar or identical parts between embodiments can be referred to mutually. For system embodiments, since they basically correspond to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

Claims

1. A lightweight unmanned aerial vehicle aerial image target detection method, comprising obtaining an original unmanned aerial vehicle aerial image, characterized in that, Further comprising the following steps: Step 1: input the original UAV aerial image into the effective receptive field network ERFNet, adjust the size of the original UAV aerial image, and generate an adjusted UAV aerial image; Step 2: input the adjusted UAV aerial image into the effective receptive field module ERF module with a stride of 2 for downsampling and feature extraction, and use the effective receptive field module ERF module with a stride of 1 for feature extraction, and the image size after feature extraction remains unchanged; When the stride of the effective receptive field module ERF module is 1, the input channel number is evenly divided into two parts, the left branch is used for residual connection to prevent gradient disappearance and reduce overfitting, and the right branch is expanded by stacking n hollow convolution blocks D block to expand the network receptive field without losing local details; When the stride of the effective receptive field module ERF module is 2, the left branch learns features through the hollow convolution block D block, and the right branch learns features through the maxpooling operation, and the features learned by the left and right branches are spliced, and the channel number is restored using a 1×1 convolution; Step 3: repeat the method of step 2 for 5 times, output {C3, C4, C5} three scales; Step 4: input the {C3, C4, C5} three scales into the effective receptive field path aggregation network ERF-PAN for feature fusion, output {P3, P4, P5} three feature maps, including the following substeps: Step 41: use a 1×1 convolution on scale C5 to obtain scale C5', and make scale C5' become 40×40×256 through upsampling, and then perform feature fusion with scale C4; Step 42: use an effective receptive field module ERF module with a stride of 1 to extract features to obtain a scale C4_upsample with a size of 40×40×512; Step 43: use a 1×1 convolution on the scale C4_upsample to obtain a scale C4', and make the scale C4' become 80×80×128 through upsampling, and then perform feature fusion with the scale C3; Step 44: use an effective receptive field module ERF module with a stride of 1 to extract features to obtain a feature map P3, and the size of the feature map P3 is 80×80×128; Step 45: the feature map P3 is downsampled by an effective receptive field module ERF module with a stride of 2, and is fused with the scale C4', and then an effective receptive field module ERF module with a stride of 1 is used for feature extraction to obtain a feature map P4, and the size of the feature map P4 is 40×40×256; Step 46: the feature map P4 is down-sampled by an effective receptive field module ERF module with a stride of 2 and fused with the scale C5', and then a feature extraction is performed using the effective receptive field module ERF module with a stride of 1 to obtain a feature map P5, and the size of the feature map P5 is 20x20x512; Step 47: output three feature maps {P3, P4, P5} with sizes of 80x80x128, 40x40x256 and 20x20x512 respectively; Step 5: input the three feature maps {P3, P4, P5} into an effective receptive field detection head ERF-Head for detection, and output a prediction result using a 1x1 convolution. 2.The lightweight UAV aerial image target detection method of claim 1, wherein: The step 1 includes that the original unmanned aerial vehicle aerial image has a size of 640x640x3, and a focus module is used to generate the adjusted unmanned aerial vehicle aerial image with a size of 320x320x32. 3.The lightweight UAV aerial image target detection method of claim 2, wherein: The size of the down-sampled adjusted unmanned aerial vehicle aerial image becomes 160x160x32. 4.The lightweight UAV aerial image target detection method of claim 3, wherein: The sizes of the three scales {C3, C4, C5} are 80x80x128, 40x40x256 and 20x20x512 respectively. 5.The lightweight UAV aerial image target detection method of claim 4, wherein: The step 5 further includes compressing the channel numbers of {P3, P4, P5} using a 1x1 convolution to keep the channel numbers consistent. 6.The lightweight UAV aerial image target detection method of claim 5, wherein: The step 5 further includes stacking two D blocks for improving the feature extraction capability of the detection head.

Citation Information

Patent Citations

  • Lightweight unmanned aerial vehicle image small target detection method

    CN112101113A

  • Unmanned aerial vehicle target detection method based on lightweight feature fusion

    CN115810157A

  • Aerial image small target detection method based on deep learning

    CN116012734A