Unmanned aerial vehicle image target detection network based on double-branch attention

Abstract

The application discloses a kind of unmanned aerial vehicle image target detection networks based on double branch attention, belongs to unmanned aerial vehicle image target detection network design technical field, is composed of Backbone, Neck and Head three parts;Backbone part includes two paths, respectively as the branch of multistage feature extraction and the branch of supplementary original detail information.The application, by increasing a parallel branch and the interaction of feature extraction branch in different layers, supplementary feature information lost in multistage down-sampling process.In addition, BCMR-YOLO utilizes region-channel mixed convolution module to focus on the relationship between regions and channels respectively, filter background and interference information, and strengthen the positioning ability of small target pixels in the network.Finally, BCMR-YOLO captures more rich small target feature information by increasing a high-resolution detection head, and improves the detection accuracy of small target.

Description

Technical Field

[0001] This invention belongs to the field of UAV image target detection network design technology, specifically relating to a UAV image target detection network based on bi-branch attention. Background Technology

[0002] Unmanned aerial vehicle (UAV) aerial image target detection technology has been widely applied in various scenarios, especially in civilian and military fields. However, because UAV aerial images typically have large and complex backgrounds, targets captured by UAVs at high altitudes occupy relatively small pixel areas, and targets are densely packed with each other and subject to significant occlusion from the UAV's perspective. Therefore, traditional target detection networks suffer from the following drawbacks when processing UAV aerial images:

[0003] Traditional object detection network designs, such as feature extraction and receptive field size, are insufficient for detecting small targets;

[0004] Traditional object detection networks suffer from multiple downsampling layers, leading to severe loss of detailed features.

[0005] Traditional object detection networks are complex and have a large number of parameters and computational loads in order to cope with general scenarios.

[0006] Therefore, we propose a UAV image target detection network based on bi-branch attention. Summary of the Invention

[0007] The purpose of this invention is to provide a UAV image target detection network based on dual-branch attention. Based on the YOLOv12 model, it adds a small-sized target detection head layer and incorporates the original detailed information of the input image into different layers of the backbone network to supplement lost detailed features. At the same time, it uses a region-channel hybrid convolution module (ACBlock module and depthwise separable convolution are cascaded) to filter background and noise, achieving efficient feature fusion across multiple branches and scales. This improves the localization capability of small target pixel regions and reduces model computation costs. It can help improve several aspects of existing UAV target detection networks: insufficient accuracy in detecting small targets and the technical problems of severe loss of detailed information due to multi-level downsampling.

[0008] To achieve the above objectives, the present invention provides the following technical solution: a UAV image target detection network based on dual-branch attention, which consists of three parts: Backbone, Neck, and Head;

[0009] The backbone consists of two paths: a multi-level feature extraction branch and a supplementary original detail information branch. In the multi-level feature extraction branch, the input image is first downsampled and features are extracted by two-level Conv modules. Then, in the supplementary original detail information branch, the input feature map is physically segmented and channel transformed to ensure the integrity of the original feature map information. The C3k2 module then re-extracts the feature information from the two branches. Finally, the ACMC module focuses and filters the fused feature information to extract rich semantic information and small target features.

[0010] The Neck section processes the path in two parts, top-down and bottom-up, to determine the size and position of the target in the image;

[0011] The Head part is used to detect multi-scale feature maps at different levels generated by the Neck part, and uses convolutional layers and prediction layers to generate the category, location and confidence of each detection result.

[0012] Preferably, the Neck section uses a two-part processing path—top-down and bottom-up—to determine the size and position of the target in the image, specifically including:

[0013] The top-down approach involves stitching together upsampled and high-resolution, coarse-grained feature maps, and then adapting and fusing contextual feature information at different scales using the ACMC module. The bottom-up approach, on the other hand, uses the Conv module and the ACMC module to fuse feature maps at different levels. By combining multi-scale and contextual information, the size and location of the target in the image can be determined.

[0014] Preferably, the fusion of feature maps at different levels using the Conv module and the ACMC module specifically includes:

[0015] First, channel transformation is performed using 1x1 convolutions. Then, multi-level feature extraction is carried out using the ACBlock module. Simultaneously, depthwise separable convolutions are cascaded to increase the receptive field. Residual connections are made between different layers of features obtained from the input feature map to improve feature reuse. Thus, the feature loss and degradation problems in the process of cascaded feature propagation and interaction are alleviated to a certain extent. Finally, features from different levels are concatenated, and a learnable scaling factor is added to further filter redundant information and introduce residual connections from input to output.

[0016] Preferably, the ACBlock module includes a region attention module, a channel attention module, and an MLP module; the region attention module focuses on the target location and suppresses background information and noise, while the channel attention module enhances key semantic information and weakens noise channels;

[0017] In its multi-level feature extraction process, the ACBlock module utilizes both region attention and channel attention modules to extract features from the input feature map in parallel, then concatenates and fuses them. This parallel fusion of region and channel attention modules enables adaptive feature calibration across both spatial and channel dimensions. The concatenated feature map is then subjected to a second extraction by the region attention module to further filter out any interference that may arise after fusion, increasing the weight of small target features. Since the output weight map lacks cross-regional semantic relationships, and when detecting large objects, there may be issues where related information is fragmented into different regions, the problem persists. Finally, the MLP module extracts long-distance dependencies between different spatial locations, breaking the limitations of local receptive fields and enabling global context modeling.

[0018] Compared with the prior art, the beneficial effects of the present invention are:

[0019] This invention addresses the issue of numerous small targets in UAV aerial images. To overcome the shortcomings of existing detection networks, such as insufficient accuracy in small target detection, significant loss of detail information during multi-level downsampling, and model complexity, a novel UAV image target detection network based on dual-branch attention, BCMR-YOLO, is proposed. BCMR-YOLO adds a parallel branch to interact with the feature extraction branch across different layers, supplementing the feature information lost during multi-level downsampling. Furthermore, BCMR-YOLO utilizes a region-channel hybrid convolution module (a cascaded structure of an ACBlock module and depthwise separable convolutions) to focus on the connections between regions and channels, filtering background and interference information, and enhancing the network's ability to locate small target pixels. Attached Figure Description

[0020] Figure 1 This is a diagram of the BCMR-YOLO network framework of the present invention.

[0021] Figure 2 This is a structural diagram of the ACMC module of the present invention.

[0022] Figure 3 This is a structural diagram of the ACBlock module of the present invention.

[0023] Figure 4 These are visualizations of the comparative experimental results of this invention on the VisDrone2019 test dataset. (a) BCMR-YOLO detection results; (b) YOLOv8-s detection results; (c) YOLOv7-n detection results; (d) YOLOv11-n detection results. Detailed Implementation

[0024] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0025] The overall framework of the BCMR-YOLO network proposed in this invention is as follows: Figure 1 As shown, it includes three important parts:

[0026] We propose a downsampling parallel branch, which splits the input image into two branches. One branch serves as the backbone to extract multi-layer feature maps, while the other branch uses image segmentation and channel transformation to extract the original information of the input image and concatenates it with the backbone at different levels, thereby compensating for the loss of detail information caused by the backbone during multi-level downsampling.

[0027] Adding a small target detection head is beneficial because drone aerial images often contain multiple small targets, which are easily obscured by the large-scale features on high-level feature maps. Therefore, by adding an additional detection head in the shallow layer, more local information can be utilized to improve the detection accuracy of small targets.

[0028] (3) A region-channel mixed convolution module (ACMC) is proposed, such as... Figure 2 As shown, this module cascades a region-channel attention module (ACBlock) with depthwise separable convolutions, fusing them across different layers. By utilizing ACBlock, it focuses on the feature relationships between regions and channels in the upper layers, enhancing feature aggregation capabilities, filtering irrelevant noise, and reducing computational costs.

[0029] This invention employs the VisDrone2019 dataset for experiments to validate the performance of BCMR-YOLO. The VisDrone2019 dataset, used for object detection in images, contains 10209 still images from various drone cameras, covering a wide range of locations including different cities, environments, objects, and densities. The training set consists of 6471 images, the validation set of 548 images, and the test set of 1610 images. The dataset includes the following categories: “ignored regions,” “pedestrian,” “people,” “bicycle,” “car,” “van,” “truck,” “tricycle,” “awning-tricycle,” “bus,” “motor,” and “others,” where “ignored regions” and “others” represent non-valid target regions. To comprehensively and intuitively evaluate the performance of BCMR-YOLO, different evaluation metrics are used for network complexity and detection accuracy. For network complexity, the number of network parameters and the number of floating-point operations are used for quantitative analysis, expressed by the following formulas: Network Parameters Number of floating-point operations In the formula, , , These represent the depth, height, and width of the convolutional kernel, respectively. , The number of input and output channels; , , These represent the depth, height, and width of the image, respectively. For object detection accuracy, precision (P), recall (R), average precision (AP), and mean average precision (mAP) are used for evaluation.

[0030] Precision (P) represents the percentage of all detected targets that are correctly detected, and is expressed by the following formula:

[0031]

[0032] Where TP represents the number of correctly detected targets; FP represents the number of incorrectly detected targets.

[0033] Recall (R) represents the percentage of all real-world targets that are correctly detected, and is expressed by the following formula:

[0034]

[0035] Here, FN represents the number of targets that actually exist but were not correctly detected.

[0036] Mean precision (AP) represents the area enclosed by the curves representing precision and recall. This metric includes three different average precisions: AP0.5, AP0.75, and AP0.95. AP0.5 indicates that a prediction is considered correct only when the intersection-union ratio (IU) between the predicted and ground truth bounding boxes is greater than 0.5; AP0.75 indicates that a prediction is considered correct only when the IU is greater than 0.75; and AP0.95 represents the average AP within the IU threshold range of 0.5 to 0.95 (with a step size of 0.05). It is expressed by the following formula:

[0037]

[0038] Mean precision (mAP) represents the average detection precision across all sample categories. This metric includes three different mean precision values: mAP 0.5, mAP 0.75, and mAP 0.95. It is expressed by the following formula:

[0039]

[0040] This invention utilizes the PyTorch deep learning framework. The server operating system is Ubuntu, the server CPU is an Intel(R) Xeon(R) Platinum 8352V CPU @ 2.10GHz, and the server GPU is a vGPU-32GB with 32GB of video memory. In this invention's experiments, the SGD optimizer was selected, and the initial learning rate was set to 1. 10 -2 Momentum was set to 0.937, batch size to 32, and weight decay factor to 5. 10 -4 The maximum number of iterations is set to 300. During the preprocessing stage of the training set images, the input images are first subjected to letterboxing, i.e., proportionally scaled, centered, and padded with filler. Other steps include converting from BGR to RGB format, normalization, channel number transformation, and adding batch dimension.

[0041] Table 1 shows the module names and parameter settings for each layer of the BCMR-YOLO network; Table 2 shows the ablation experiment results of BCMR-YOLO. The results show that, using YOLOv12n as the baseline network, by adding a branch consisting of physical cutting and channel transformation, the original information of the input image is preserved to the maximum extent without significantly increasing the number of network parameters. Furthermore, by stitching the YOLOv12n network with the baseline network backbone at different levels, the details lost during multi-level downsampling of the baseline network backbone are supplemented, preserving more fine-grained target features at high-resolution levels and improving target detection accuracy. When an additional high-resolution detector head is added, the number of network parameters increases slightly. By stitching with deep feature maps, the deep network can obtain richer shallow feature information of small targets, improving the detection accuracy of small targets. The table shows a significant increase in both mAP50 and mAP50-95. When the ACMC module is added, a hybrid approach of region attention and channel attention is used to focus on the feature information of small targets when extracting multi-scale feature information. This suppresses the impact of severe background interference and sparse small target features in UAV images. Furthermore, depthwise separable convolutions are used in cascades to increase the receptive field while reducing the number of network parameters. When branch paths, detector heads, and ACMC modules are added simultaneously, the network BCMR-YOLO of this invention achieves optimal overall performance at mAP50 and mAP50-95 distances, with only a 0.27M increase in network parameters compared to the baseline network.

[0042] Figure 4 The results of comparative experiments on the VisDrone2019 test dataset are presented, showcasing BCMR-YOLO, YOLOv8-s, YOLOv7-n, and YOLOv11-n. As shown in the upper left black box of the figure, BCMR-YOLO detected more "pedestrians," closest to the ground truth label, while YOLOv8-s and YOLOv7-n detected only one "pedestrian," indicating poor accuracy; YOLOv11-n did not detect any "pedestrians." In the lower right black box, BCMR-YOLO accurately detected two "people," closest to the ground truth label, while YOLOv8-s and YOLOv7-n detected only one "people" with low confidence; YOLOv11-n did not detect any "people." These comparative experimental results demonstrate that the BCMR-YOLO network of this invention significantly improves overall detection accuracy, especially for small targets such as "pedestrians."

[0043] Table 1 BCMR-YOLO Network Parameters

[0044]

[0045] Table 2. Overall network ablation experimental results on the VisDrone2019 test dataset.

[0046]

[0047] Table 3 shows the experimental results comparing the network with other networks on the VisDrone test dataset.

[0048]

[0049] This invention addresses the issue of numerous small targets in UAV aerial images. To overcome the shortcomings of existing detection networks, such as insufficient accuracy, significant loss of detail information during multi-level downsampling, and model complexity, a novel UAV image target detection network, BCMR-YOLO, based on dual-branch attention, is proposed. BCMR-YOLO adds a parallel branch to interact with the feature extraction branch across different layers, supplementing the feature information lost during multi-level downsampling. Furthermore, BCMR-YOLO utilizes a region-channel hybrid convolution module (a cascaded region-channel attention module and depthwise separable convolution) to focus on the connections between regions and channels, filtering background and interference information and enhancing the network's ability to locate small target pixels. Finally, BCMR-YOLO adds a high-resolution detection head to capture richer small target feature information, improving detection accuracy. Experimental results on the VisDrone2019 dataset demonstrate that BCMR-YOLO, using 2.83M parameters, significantly improves the accuracy of small target detection in UAV aerial images while maintaining a lightweight architecture.

[0050] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0051] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A method for constructing a UAV image target detection network based on dual-branch attention, characterized in that, The UAV image target detection network consists of three parts: Backbone, Neck, and Head. The backbone consists of two paths: a multi-level feature extraction branch and a supplementary original detail information branch. In the multi-level feature extraction branch, the input image is first downsampled and features are extracted by two-level Conv modules. Then, in the supplementary original detail information branch, the input feature map is physically segmented and channel transformed to ensure the integrity of the original feature map information. The C3k2 module then re-extracts the feature information from the two branches. Finally, the ACMC module focuses and filters the fused feature information to extract rich semantic information and small target features. The Neck section processes the path in two parts, top-down and bottom-up, to determine the size and position of the target in the image; The Head part is used to detect multi-scale feature maps at different levels generated by the Neck part, and uses convolutional layers and prediction layers to generate the category, location and confidence of each detection result; The Neck section uses a two-part processing path—top-down and bottom-up—to determine the size and position of the target in the image, specifically including: The top-down approach involves stitching together upsampled and high-resolution, coarse-grained feature maps, and then adapting and fusing contextual feature information at different scales using the ACMC module. The bottom-up approach, on the other hand, uses the Conv module and the ACMC module to fuse feature maps at different levels. By combining multi-scale and contextual information, the size and location of the target in the image can be determined. Specifically, the fusion of feature maps at different levels through the Conv module and the ACMC module includes: First, channel transformation is performed using 1x1 convolution. Then, multi-level feature extraction is performed using a three-level ACBlock module, and feature extraction is also performed using a two-level depthwise separable convolution DSConv to increase the receptive field. After each level of feature extraction, the features extracted by the two convolutions are fused. The fused features are then fed into the next level of ACBlock, and the output of the first level DSConv is fed into the second level DSConv. Finally, the features extracted from different levels of ACBlock and the output features of the 1x1 convolution are concatenated, and a learnable scaling factor is added to further filter redundant information and introduce residual connections from input to output. The ACBlock module includes a region attention module, a channel attention module, and an MLP module; the region attention module focuses on the target location and suppresses background information and noise, while the channel attention module enhances key semantic information and weakens noisy channels. In the multi-level feature extraction process, the ACBlock module utilizes both the region attention module and the channel attention module to extract features from the input feature map in parallel, and then concatenates and fuses them. The parallel fusion of the region attention module and the channel attention module enables adaptive feature calibration in both spatial and channel dimensions. The concatenated feature map is then subjected to a second extraction by the region attention module to further filter out interference information generated after fusion and increase the weight of small target feature information. Finally, the MLP module extracts long-distance dependencies between different spatial locations, breaking the limitations of the local receptive field and enabling global context modeling.

Images