A lightweight unmanned aerial vehicle aerial image small target detection method and system based on TFA-YOLO11

Abstract

The application discloses a kind of lightweight unmanned aerial vehicle aerial image small target detection method and system based on TFA-YOLO11, it is related to target detection field.Method includes: data set acquisition and data enhancement;TFA-YOLO11 network model is constructed;The neural network is trained using the data set after data enhancement, obtain the neural network of well-trained;Using the model of well-trained to detect the image to be detected.The application is based on multi-head self-attention mechanism, it is focused on to excavate the complementary information between adjacent features, a feature aggregation module is designed, different scale feature maps can be better fused spatial, semantic information;The application is based on high-resolution feature map, a very small target detection module is designed, can enhance the perception ability of network to very small target;The application is based on SPD convolution and designs lightweight main stem, neck network, can realize down-sampling with smaller parameter amount and calculation amount without information loss.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of image processing, and particularly relates to a lightweight unmanned aerial vehicle aerial image small target detection method and system based on TFA-YOLO11. BACKGROUND

[0002] With the rapid development of unmanned aerial vehicle technology, unmanned aerial vehicles have become an important tool in modern monitoring, investigation, rescue and other fields. Unmanned aerial vehicles are flexible and efficient, and can obtain large-scale, high-resolution image data to provide valuable information for various applications. Small targets refer to targets with a size of less than 32*32 pixels in an image. Small target detection refers to identifying and locating small objects such as pedestrians and vehicles in an image. In unmanned aerial vehicle aerial images containing rich environmental information, small targets often have great detection difficulty due to their small size, low target-to-background contrast, and severe occlusion.

[0003] With the rapid development of deep learning, neural network-based target detection algorithms have become the mainstream method in the field of target detection. Deep learning-based target detection technology can be generally divided into two categories: single-stage (One-stage) and two-stage (Two-stage). Two-stage target detection algorithms first use candidate region generation techniques to generate candidate regions that may contain targets, and then perform target position regression and class prediction on these candidate regions. Representative methods include Faster R-CNN, FPN, etc. Single-stage target detection does not require candidate region extraction, and only one forward pass is required to obtain target class and position. Representative methods include SSD, YOLO, etc. Two-stage target detection follows a coarse-to-fine processing mode, with high detection accuracy but slow speed, making it rarely used in engineering. On the contrary, single-stage target detection usually has slightly lower accuracy but better real-time performance. YOLO has become the preferred target detection algorithm at present due to its fast inference speed and small model size.

[0004] However, these algorithms mainly focus on regular size and regular perspective target detection tasks, and are often difficult to deploy in unmanned aerial vehicle target detection tasks. The main reasons are as follows: on the one hand, ground target detection based on unmanned aerial vehicle video faces more problems such as small target size, complex and variable background, overlap, occlusion, and scale variation, making it difficult for conventional target detection algorithms to accurately detect specific objects; on the other hand, since the inference process needs to be implemented in unmanned aerial vehicle devices, it is difficult to balance high detection accuracy and model lightweight. These factors hinder the development of deep learning methods in the field of small target detection under the perspective of unmanned aerial vehicles. SUMMARY

[0005] To solve the problems in the prior art, the application provides a lightweight unmanned aerial vehicle aerial image small target detection method and system based on TFA-YOLO11.

[0006] The technical scheme adopted by the application is as follows:

[0007] In a first aspect, the application discloses a lightweight unmanned aerial vehicle aerial image small target detection method based on TFA-YOLO11, comprising the following steps:

[0008] Step 1): Obtain the unmanned aerial vehicle aerial image dataset and perform data enhancement;

[0009] Step 2): Construct a TFA-YOLO11 network model;

[0010] The TFA-YOLO11 network model comprises a backbone network, a neck network, a feature aggregation module based on a Transformer, a small target detection module and a detection head module; the backbone network is used for multi-level down-sampling and feature extraction of an input image to obtain a plurality of layers of feature maps; the feature aggregation module is used for feature aggregation of adjacent layers of feature maps to obtain a plurality of high-quality aggregated feature maps of different scales; the neck network and the small target detection module jointly constitute a feature pyramid structure, and the feature pyramid structure is used for feature fusion of the high-quality aggregated feature maps of different scales, so that the obtained final feature map has the feature information of the other scale aggregated feature maps; in addition to the part constituting the feature pyramid structure, the small target detection module further comprises a small target detection head, which is used for small target detection of the final feature map with the highest spatial resolution; in addition to the final feature map with the highest spatial resolution, the other final feature maps are subjected to target detection through the detection head module;

[0011] Step 3): Train the TFA-YOLO11 network model by using the data set after data enhancement to obtain a trained TFA-YOLO11 network model;

[0012] Step 4): Detect the unmanned aerial vehicle aerial image to be detected by using the trained TFA-YOLO11 network model to obtain the positions and categories of all targets on the image to be detected. In a second aspect, the application discloses a lightweight unmanned aerial vehicle aerial image small target detection system based on TFA-YOLO11, which comprises:

[0013] A data enhancement module is used for data enhancement of the unmanned aerial vehicle aerial image dataset;

[0014] A network model construction module is used for constructing a TFA-YOLO11 network model;

[0015] A neural network training module is used to train the TFA-YOLO11 network model using the data-enhanced dataset to obtain a trained TFA-YOLO11 network model.

[0016] The UAV aerial image detection module is used to detect the UAV aerial images to be detected using the trained TFA-YOLO11 network model to obtain the location and category of all targets in the image to be detected.

[0017] Compared with the prior art, the present invention has the following beneficial effects:

[0018] 1) This paper proposes a Transformer-based feature aggregation module, leveraging a multi-head attention mechanism to enable information exchange between aggregated features from different layers of the network. This effectively analyzes the correlation between feature maps of different scales, fully integrating the spatial and semantic information of feature maps of different scales, and enhancing the model's representational capabilities. Furthermore, it proposes the first Layer Feature Aggregation (LFA) module, which aggregates features at each layer to generate an aggregated feature vector for cross-layer information exchange.

[0019] 2) This paper proposes a very small target detection module, expands the neck network, and adds a very small target detection head, which can enhance the very small target perception ability of the network model;

[0020] 3) This paper proposes a backbone network and neck network for downsampling based on SPD convolution. It uses SPD convolution with smaller parameters and computational complexity to achieve feature map downsampling without information loss by converting the spatial dimension to the channel dimension, thereby achieving lightweight model. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 This is a structural diagram of a network model for small target detection in lightweight UAV aerial images based on TFA-YOLO11 provided by the present invention;

[0022] Figure 2 This is a schematic diagram of the structure of the small target detection device for UAV aerial images of the present invention;

[0023] Figure 3 This is a structural diagram of the TFA module of the present invention;

[0024] Figure 4 for Figure 1 Schematic diagram of the SPD convolution structure;

[0025] Figure 5 The VisDrone2021 drone aerial image dataset used in this example;

[0026] Figure 6 This is a comparison chart of the detection effects of the VisDrone2021 drone aerial image dataset using an embodiment of the present invention and the YOLO11s model. DETAILED DESCRIPTION

[0027] The present invention will be further described and illustrated below in conjunction with specific embodiments. The embodiments are merely illustrative of the present disclosure and do not limit its scope. The technical features of the various embodiments of the present invention may be combined accordingly, provided that there is no conflict between them.

[0028] Example 1

[0029] like Figure 1 As shown in the figure, it is a structural diagram of the network model for small target detection in lightweight UAV aerial images based on TFA-YOLO11 (Transformer-based Feature Aggregation-YOLO11) of the present invention, which mainly includes:

[0030] Step 1: Get the dataset and perform data augmentation.

[0031] First, we obtain a dataset of drone aerial images. For each of these images, we randomly transform their hue, saturation, and brightness to create color perturbations. We then randomly translate, scale, and flip the images with a certain probability. This helps the model detect partially visible and multi-scale objects, and increases the diversity of the dataset. Furthermore, we use a mosaic data augmentation method to further enhance the dataset's diversity and prevent model overfitting.

[0032] The mosaic data enhancement method specifically includes:

[0033] (1) Randomly select four images from the dataset for mosaic data augmentation;

[0034] (2) Randomly generate the coordinates of the intersection points of the four spliced ​​images;

[0035] (3) Each image is randomly scaled while maintaining its aspect ratio, and then stitched together to form a new image, with the portion exceeding the size boundary being cropped and removed;

[0036] (4) The annotations of each image are also processed accordingly to obtain new training samples with annotations.

[0037] In this embodiment, the maximum ratios of random transformation of hue, saturation and lightness are 0.015, 0.7 and 0.4 respectively; the maximum ratios of random translation, scaling and horizontal flipping are 0.1, 0.5 and 0.5 respectively.

[0038] Step 2: Build the TFA-YOLO11 network model.

[0039] The TFA-YOLO11 network model uses the YOLO11 network as the basic network. The traditional YOLO11 network is divided into three parts: the backbone network, the neck network, and the detection head. Among them, the backbone network extracts multi-level features from the input image through a series of convolution, pooling and other operations to obtain multi-layer feature maps from large to small. Subsequently, the neck network adopts the Feature Pyramid Network (FPN) structure to transfer feature information from bottom to top and top to bottom on feature maps of different scales, transfer the semantic information of the deep feature map to the shallow feature map, and transfer the spatial position information of the shallow feature map to the deep feature map, thereby improving the representation ability of the model. The detection head predicts the target position and category of the feature map processed by the neck network.

[0040] The TFA-YOLO11 network model of the present invention is an improvement based on the traditional YOLO11 network. Figure 1 As shown, it includes a backbone network and a neck network for downsampling based on SPD convolution, a feature aggregation module based on Transformer, a very small target detection module and a standard detection head. The present invention adopts SPD convolution to replace the standard convolution in the backbone network and neck network of YOLO11. The backbone network and neck network for downsampling based on SPD convolution achieve downsampling with fewer parameters and calculations without information loss by converting the spatial dimension to the channel dimension, thereby achieving lightweight model. The present invention adds a feature aggregation module based on Transformer between the backbone network and the neck network of YOLO11. The feature aggregation module based on Transformer is used to perform feature aggregation on the adjacent three-layer feature maps extracted by the backbone network, effectively analyzes the correlation between feature maps of different scales, fully integrates the spatial and semantic information of feature maps of different scales, and enhances the representation ability of the model. The present invention also adds a very small target detection module after one of the feature aggregation modules. The very small target detection module adds a feature map with higher resolution to the feature aggregation and feature fusion of the neck network, thereby increasing the model's perception of very small targets and effectively improving the model's small target detection performance. A very small target detection head (Tiny Head) is introduced into the very small target detection module, which performs prediction based on a feature map with higher spatial resolution. Compared with the standard detection head, the very small target detection head has a stronger perception ability for small targets, thereby effectively improving the detection accuracy of the network model for small targets.

[0041] Furthermore, the backbone network of YOLO11 includes multiple standard convolutional layers, multiple C3k2 layers, an SPPF layer and a C2PSA layer. The structural layers are connected in series and numbered from layer 0 to layer 10 in sequence according to the process, of which layers 0, 1, 3, 5, and 7 are standard convolutional layers, layers 2, 4, 6, and 8 are C3k2 layers, and layer 9 is an SPPF layer. The backbone network of the present invention replaces layers 1, 3, 5, and 7 with SPD convolutional layers on this basis. The SPD convolutional layer can downsample the feature map to reduce the resolution of the feature map, and at the same time transfer the spatial information reduced by downsampling to the channel dimension, thereby increasing the depth of the feature map. The output of the 0th convolutional layer, each C3k2 layer, and the C2PSA layer is the output of the backbone network, that is, a multi-layer feature map, specifically, a 6-layer feature map P = {P1,...,P i ,...,P6}.

[0042] Further, such as Figure 3 The specific working process of the Transformer-based feature aggregation module is as follows:

[0043] According to the order of feature maps in the multi-layer feature map output by the backbone network, the two feature maps before and after the feature map are its adjacent feature maps. The multi-layer feature map is input into the feature aggregation module based on Transformer, and the feature aggregation module performs the same feature aggregation operation on the feature maps output by the four C3k2 layers of the backbone network and their adjacent feature maps. Specifically, for each feature map output by the C3k2 layer, the feature aggregation module based on Transformer first aligns the spatial resolution of the feature map and its adjacent feature maps, and makes the resolution of the three feature maps the same by upsampling the smaller feature map and downsampling the larger feature map, and keeps the number of channels of the three feature maps consistent. i Taking the feature aggregation module as an example, the scale alignment process can be expressed as:

[0044] X i-1 =DownSample(P i-1 )

[0045] X i+1 =UpSample(P i+1 )

[0046] Among them, X i-1 、X i+1 They represent the feature maps after scale alignment, DownSample and UpSample represent the downsampling and upsampling functions respectively.

[0047] After the spatial resolution adjustment is completed, the feature maps after each three-scale alignment are sent to the layer feature aggregation module for feature aggregation of the feature maps of that layer. The layer feature aggregation module first extracts significant features from each scale-aligned feature map through global maximum pooling and global average pooling, and obtains two feature vectors for each feature map. The two feature vectors are then processed using a shared multi-layer perceptron (Shared MLP) to exchange information and obtain higher-level features. The two feature vectors output by the shared multi-layer perceptron are added together and given nonlinear characteristics through the SiLU activation function to obtain the aggregated feature vector of the feature map of that layer. This process can be expressed as:

[0048] M i =SiLU(MLP(X i,max )+MLP(X i,avg ))

[0049] Among them, X i,max 、X i,avg Represents X i The global maximum pooling and global average pooling features, M i Represents X i The operation on adjacent feature maps is consistent with the above formula.

[0050] After the three adjacent layers of feature maps obtain the aggregated feature vector in the same way, the three aggregated feature vectors are sent to the Transformer encoder to realize cross-layer information interaction. The Transformer encoder weights the effective information according to the correlation between the features of different layers, thereby giving the aggregated features of each layer the ability to perceive the semantic information contained in the aggregated features of the adjacent layers, solving the deficiency that only the features of the previous level can be perceived in the bottom-up path of the feature pyramid. The three aggregated feature vectors after cross-layer information interaction are used as channel attention weights and acted on the feature maps after scale alignment of the corresponding layers, and the feature maps after cross-layer information interaction are obtained by combining residual connections. Finally, the three feature maps after cross-layer information interaction are connected according to the channel dimension, and the channel dimension feature fusion is realized through 1×1 convolution and the dimension is reduced to the original number of channels to obtain the high-quality aggregated feature map finally output by the Transformer-based feature aggregation module. The above operations can be expressed as:

[0051] [M i ' -1 ,M i ′,M i ' +1 ]=Transformer([M i-1 ,M i ,M i+1 ])

[0052] X i′=M i ′⊙X i +X i

[0053] X fuse =Conv(Concat(X i ' -1 ,X i ′,X i ' +1 ))

[0054] Among them, M i ' -1 、M i ′、M i ' +1 Represents the aggregated feature vectors after cross-layer interaction of feature maps at all levels; ⊙ is the Hadamard product, which realizes channel-by-channel weighting; X i ' -1 、X i ′、X i ' +1 Represent the feature maps after information interaction; X fuse A high-quality aggregated feature map representing the model output.

[0055] Furthermore, the neck network includes two upsampling modules and two downsampling modules, forming a feature pyramid structure. The feature maps in the two upsampling modules are transferred from bottom to top, and the feature maps in the two downsampling modules are transferred from top to bottom. The upsampling module includes an upsampling layer, a concatenation layer, and a C3k2 layer. The downsampling module includes an SPD convolution layer, a concatenation layer, and a C3k2 layer.

[0056] The Transformer-based feature aggregation module outputs four aggregated feature maps with different spatial resolutions. The aggregated feature map with the lowest resolution first enters the first upsampling module. After passing through the first upsampling layer, it is spliced ​​with the aggregated feature map with the second lowest resolution in the first splicing layer in the channel dimension. Then, it passes through the first C3k2 layer and the second upsampling layer in the second upsampling module. The feature map obtained is spliced ​​with the aggregated feature map with the second highest resolution in the channel dimension in the second splicing layer. The obtained feature map is the output of the bottom-to-top path.

[0057] In the top-to-bottom path, the feature map entering the top of the path for information transmission first passes through the SPD convolution layer of the second downsampling module, and is spliced ​​with the feature map output by the first C3k2 layer at the splicing layer. It then passes through the C3k2 layer of the second downsampling module, the SPD convolution layer of the first downsampling module, and is spliced ​​with the lowest resolution aggregated feature map at the splicing layer. After passing through the C3k2 layer of the first downsampling module, the resulting feature map is the output of the top-to-bottom path. The outputs of the C3k2 layers of the two downsampling modules are respectively input into a standard detection head. The standard detection head is the original detection head of the YOLO11 network.

[0058] The very small target detection module includes a third upsampling module, a third downsampling module and a very small target detection head. The structure is the same as the upsampling module and downsampling module in the neck network. The upsampling module and downsampling module of the very small target detection module are added to the feature pyramid of the neck network to form a three-level feature pyramid. At the same time, the feature map with the highest spatial resolution in the process of feature pyramid information transmission is passed through the very small target detection head for detection tasks. The third upsampling module receives the aggregated feature map with the highest spatial resolution as input 1, and receives the feature map transmitted from the bottom-to-top path in the feature pyramid of the neck network as input 2. After upsampling input 2, it is spliced ​​with input 1, and after feature fusion through the C3k2 layer of YOLO11, the P2 feature map with the highest resolution is output. P2 feature Figure 1 On the one hand, it enters the third downsampling module, and on the other hand, it is input into the Tiny Head. In the third downsampling module, the P2 feature map passes through the SPD convolution layer and is concatenated with the output of the neck network's bottom-up path. It is then processed by the C3k2 layer and enters the neck network's top-down path for information transfer. The output of the C3k2 layer of the third downsampling module is also input into the third standard detection head for detection.

[0059] The minimal target detection head is a decoupled head consisting of two branches. The first branch is responsible for the target position regression task, and further feature extraction is performed by two standard convolution blocks. The number of channels is then adjusted through a convolution with a kernel size of 1 to predict the target's bounding box position and calculate the positioning loss. The second branch is responsible for the target category prediction task. The category prediction task is simpler than the position regression task, so the second branch consists of two depth-wise separable convolutions with smaller parameters and computational complexity, and the number of channels is adjusted through a convolution with a kernel size of 1 to predict the target category and calculate the category loss. The other detection heads have the same structure as the minimal target detection head.

[0060] Furthermore, the structure of the SPD convolutional layer is as follows Figure 4 As shown, the specific working process is as follows:

[0061] Assume an input feature map X of size H×W×C, where H and W represent the height and width of the feature map, and C represents the number of channels. Given a downsampling factor s, SPD convolution divides X into s 2 subgraphs, and all subgraphs are connected along the channel dimension, and then a 1×1 convolution is used to adjust the number of channels to obtain the final output. It can be simplified to the following equation:

[0062] X i,j =X[i:H:s,j:W:s],

[0063] where X i,j Represents the subgraph with starting coordinates i,j, i,j∈{0,1,···,s-1}.

[0064] X′=Conv 1×1 [Concat(X i∈[0,s-1],j∈[0,s-1] )],

[0065] Among them, X′ represents the output feature map obtained by applying SPD convolution to the input feature map X.

[0066] As you can understand, SPD convolution first converts spatial information into depth information, halving the spatial resolution of the feature map without losing information. Then, a 1×1 convolution maintains the spatial resolution while reducing the number of channels, achieving feature extraction while reducing the number of parameters and computation. Replacing standard convolution with SPD convolution preserves key details that may be lost in standard convolution, improving the perception of small objects.

[0067] Step 3: Use the data-enhanced dataset to train the TFA-YOLO11 network model to obtain a trained network model.

[0068] Specifically, in this embodiment, the data augmented dataset is divided into a training set and a validation set in a ratio of 8:2; 8 pictures are input for each training; the input picture size is uniformly scaled to 800×800 pixels; this embodiment iterates for 500 rounds of training; CIoU loss and distributed focus loss (DFL) are used as positioning losses, binary cross entropy loss BCE is used as classification loss, and the network weights are updated using stochastic gradient descent, with an initial learning rate of 0.01 and a momentum of 0.937; after the training is completed, the trained network weights are saved.

[0069] Step 4: Use the trained network model to detect drone aerial images.

[0070] Specifically, the TFA-YOLO11 network model and trained network weights are used to detect objects in drone aerial images. The network performs forward inference, and the detection heads (including three standard detection heads and a tiny object detection head) output the bounding box coordinates and category of the detected objects.

[0071] Example 2

[0072] This embodiment also provides a lightweight UAV aerial image small target detection system based on TFA-YOLO11, which is used to implement the above-mentioned embodiments. The terms "module," "unit," etc. used below may refer to a combination of software and / or hardware that implements the predetermined functions. Although the systems described in the following embodiments are preferably implemented in software, implementation in hardware, or a combination of software and hardware, is also possible.

[0073] Figure 2 FIG. 1 is a block diagram of a lightweight UAV aerial image small target detection system based on TFA-YOLO11 according to an embodiment. Figure 2 As shown, the system includes:

[0074] The data enhancement module is used to enhance the data of drone aerial image datasets, increase data diversity, and improve network robustness;

[0075] Network model building module, which is used to construct the TFA-YOLO11 network model;

[0076] A neural network training module is used to train the TFA-YOLO11 network model using the data-enhanced dataset to obtain a trained TFA-YOLO11 network model.

[0077] The UAV aerial image detection module is used to detect the UAV aerial images to be detected using the trained TFA-YOLO11 network model to obtain the location and category of all targets in the image to be detected.

[0078] Regarding the apparatus in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0079] For the device embodiment, since it basically corresponds to the method embodiment, the relevant parts can be referred to the partial description of the method embodiment. The device embodiment described above is only schematic. For example, the data enhancement module can be divided into a logical function. There may be other division methods in actual implementation. For example, multiple modules can be combined or integrated into another unit. Another point is that the connection between the modules shown or discussed can be a communication connection through some interfaces, which can be electrical or other forms. Some or all of the modules can be selected according to actual needs to achieve the purpose of the present application. Ordinary technicians in this field can understand and implement it without paying creative labor. The following takes the target detection in real drone aerial images as an example to illustrate the specific implementation method to reflect the technical effect of the present invention. The specific steps in the embodiment will not be repeated.

[0080] Example 3

[0081] Next, the VisDrone2021 drone aerial image dataset is used as the research object to carry out the target detection method verification of the present invention. In order to comprehensively compare the target detection performance, the target detection results are displayed from the perspective of visualization and quantification, respectively, using the mean average precision (mAP), the average precision AP of small target detection, and the average precision AP of small target detection. S The proposed object detection method is evaluated. Here, mAP refers to taking 10 IoU thresholds from 0.5 to 0.95 with a step size of 0.05, and then calculating the average precision. This paper uses mAP 50-95 The calculation process of mAP is as follows:

[0082]

[0083]

[0084]

[0085]

[0086] Where P is the precision rate, R is the recall rate; TP represents true positive examples, FP represents false positive examples, and FN represents false negative examples; J(P,R) k represents the average precision function, and n represents the total number of categories.

[0087] According to the guidelines of the COCO dataset, the target pixel size less than 32×32 pixels is defined as a small target, and AP is used. S Represents the average precision of small targets, which is used to evaluate the detection performance of small targets. S The calculation method is the same as mAP, but different from mAP is that AP is calculated SOnly small targets with size less than 32x32 pixels are considered.

[0088] In addition, the model parameter quantity Params (Parameters) and the calculation quantity GFLOPs (Giga-Floating-Point Operations Per seconds) are used as the model lightweight evaluation indexes.

[0089] The VisDrone2021 dataset is composed of unmanned aerial vehicle (UAV) aerial images taken by a UAV at different locations and different altitudes, and has 10 target categories (pedestrian, car, bicycle, etc.). Figure 5 Part of typical images in the VisDrone2021 UAV aerial image dataset. Figure 6 The target detection results obtained by the embodiment of the present application and the YOLO11s algorithm are compared, wherein (a) is the detection result of the YOLO11s algorithm, and (b) is the detection result of the embodiment of the present application. Figure 6 It can be seen that the embodiment of the present application has stronger small target perception ability and better small target detection performance.

[0090] Table 1: Evaluation indexes of target detection results of the VisDrone2021 UAV aerial image dataset

[0091]

[0092] The comparative method Faster R-CNN comes from Ren S, He K, Girshick R, et al. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 1137-1149.

[0093] The comparison method RT-DETR-L comes from Zhao Y, Lv WY, Xu SL, et al. DETRs Beat YOLOs on Real-time Object Detection [C]. 2024 IEEE / CVF Conference on ComputerVision andPattern Recognition (CVPR), Seattle, WA, USA, 2024, pp.16965-16974.

[0094] The comparison method YOLOv10s is from Wang A, Chen H, Liu L, et al. YOLOv10: Real-Time End-to-End Object Detection: arXiv:2405.14458[Z / OL].arXiv, 2024.

[0095] Comparison methods YOLO11s and YOLO11m are from Jocher G, Qiu J. Ultralytics YOLO11[Z] 2024. https: / / github.com / ultralytics / ultralytics.

[0096] The target detection visualization results of the VisDrone2021 dataset are as follows Figure 6 As shown in Table 1, the quantification results are shown in Table 1. In the quantification results, the mAP 50-95 The present invention achieved the best performance in AP S The present invention achieves suboptimal performance, only slightly lower than YOLO11m, but with far fewer parameters and computational complexity than YOLO11m. This shows that the present invention achieves stronger small target perception capabilities based on the small target detection module, improving small target detection accuracy. At the same time, the Transformer-based feature aggregation module enhances the model's representation capabilities, leading to an improvement in overall detection accuracy. Furthermore, SPD convolution replaces standard convolution for downsampling, reducing the model's parameter count and computational complexity, thus achieving a lightweight model.

[0097] The above-described embodiments merely illustrate several implementations of the present invention, and while the descriptions are relatively specific and detailed, they are not to be construed as limiting the scope of the present invention. Persons skilled in the art will readily appreciate that variations and modifications may be made without departing from the spirit of the present invention, and all such variations and modifications fall within the scope of protection of the present invention.

Claims

1. A lightweight UAV aerial image small target detection method based on TFA-YOLO11, characterized by: The following steps are involved: Step 1): Obtain a UAV aerial image dataset and perform data enhancement; Step 2): Build the TFA-YOLO11 network model; The TFA-YOLO11 network model includes a backbone network, a neck network, a Transformer-based feature aggregation module, a very small target detection module and a detection head module; the backbone network is used to perform multi-level downsampling and feature extraction on the input image to obtain a multi-layer feature map; the feature aggregation module is used to perform feature aggregation of adjacent layer feature maps on the multi-layer feature map to obtain high-quality aggregated feature maps of multiple scales; the neck network and the very small target detection module together constitute a feature pyramid structure, through which high-quality aggregated feature maps of different scales are subjected to feature fusion, so that the final feature map obtained has feature information of aggregated feature maps of other scales; in addition to the parts that constitute the feature pyramid structure, the very small target detection module also includes a very small target detection head for detecting small targets on the final feature map with the highest spatial resolution; Except for the final feature map with the highest spatial resolution, the other final feature maps are used for target detection through the detection head module; Step 3): Use the data augmented dataset to train the TFA-YOLO11 network model to obtain a trained TFA-YOLO11 network model; Step 4): Use the trained TFA-YOLO11 network model to detect the drone aerial image to be detected, and obtain the location and category of all targets in the image to be detected; Specifically, the step 2) of performing feature aggregation of adjacent layer feature maps on the multi-layer feature maps specifically includes: For every three adjacent feature maps P in the multi-layer feature map i-1 、P i and P i+1 Perform steps a)-d): a) Through upsampling and downsampling, the feature map P with smaller resolution among the three adjacent feature maps is converted to i+1 and feature maps P with larger resolution i-1 The scale of the feature map P in the middle i Align to obtain three feature maps X after scale alignment i-1 、X i and X i+1 ; b) Three feature maps X after scale alignment i-1 、X i and X i+1 The feature aggregation is performed through the layer feature aggregation module respectively to obtain the respective aggregated feature vectors M i-1 、M i 、M i+1 , the layer feature aggregation module is a channel attention module; c) The three aggregated feature vectors are passed through the Transformer encoder to realize cross-layer information interaction, and the aggregated feature vector M′ after cross-layer interaction is obtained i-1 , M′ i , M′ i+1 ; The aggregated feature vector after cross-layer information interaction is used as the channel attention weight, which is applied to the feature map after scale alignment at this level, and the feature map X′ after cross-layer information interaction is obtained by combining the residual connection i-1 , X′ i , X′ i+1 , the expression is as follows: X′ i-1 =M′ i-1 ⊙X i-1 +X i-1 X′ i =M′ i ⊙X i +X i X′ i+1 =M′ i+1 ⊙X i+1 +X i+1 d) The feature maps after the three cross-layer information interactions are connected according to the channel dimension, and the channel dimension feature fusion is realized by 1×1 convolution and the dimension is reduced to the original number of channels to obtain the high-quality aggregated feature map X that is finally output by the Transformer-based feature aggregation module. fuse ; After performing steps a)-d) on all feature maps in the multi-layer feature map, multiple high-quality aggregated feature maps with different resolutions are obtained; The neck network includes a first upsampling module, a second upsampling module, a first downsampling module, and a second downsampling module. The very small target detection module includes a third upsampling module, a third downsampling module, and a very small target detection head. The three upsampling modules and the three downsampling modules together constitute a feature pyramid structure. The three upsampling modules constitute a bottom-to-top path of the feature pyramid structure, which is used to transfer the semantic information of the deep feature map to the shallow feature map. The three downsampling modules constitute a top-to-bottom path of the feature pyramid structure, which is used to transfer the spatial position information of the shallow feature map to the deep feature map. The upsampling modules all include an upsampling layer, a splicing layer, and a C3k2 layer, which are used to upsample, splice, and fuse the high-quality aggregated feature maps output by the feature aggregation module; the downsampling modules all include an SPD convolution layer, a splicing layer, and a C3k2 layer, which are used to downsample, splice, and fuse the feature maps output by the upsampling module; The feature aggregation module outputs four high-quality aggregated feature maps. The feature fusion of high-quality aggregated feature maps of different scales through the feature pyramid structure is specifically as follows: The high-quality aggregated feature map with the lowest resolution among the multiple high-quality aggregated feature maps output by the feature aggregation module first enters the first upsampling module, is upsampled in the first upsampling module, and is spliced ​​with the high-quality aggregated feature map with the second lowest resolution, and then features are fused through the C3k2 layer; The feature map output by the first upsampling module enters the second upsampling module, is upsampled in the second upsampling module, and is spliced ​​with the high-quality aggregated feature map with the second highest resolution and then fused through the C3k2 layer; The feature map output by the second upsampling module enters the third upsampling module, where it is upsampled and concatenated with the highest-resolution high-quality aggregated feature map before being fused through the C3k2 layer. The final feature map output by the third upsampling module is fed into the very small object detection head on the one hand, and into the third downsampling module on the other hand. In the third downsampling module, the feature map output by the third upsampling module is downsampled and concatenated with the feature map output by the second upsampling module, and then features are fused through the C3k2 layer. The feature map output by the third downsampling module enters the second downsampling module, is downsampled in the second downsampling module, and is spliced ​​with the feature map output by the first upsampling module and then fused through the C3k2 layer; The feature map output by the second downsampling module enters the first downsampling module, is downsampled in the first downsampling module, and is spliced ​​with the high-quality aggregated feature map with the lowest resolution and then fused through the C3k2 layer; the feature maps output by the three downsampling modules are all sent to the detection head module.

2. The method for detecting small targets in lightweight UAV aerial images based on TFA-YOLO11 according to claim 1 is characterized in that: The data enhancement described in step 1) specifically includes: The hue, saturation, and brightness of the images in the drone aerial image dataset are randomly transformed within a certain range of proportions; the images in the dataset are randomly translated, scaled, and horizontally flipped; and the mosaic data augmentation method is used to combine four images in the dataset into a new image.

3. The method for detecting small targets in lightweight UAV aerial images based on TFA-YOLO11 according to claim 1, characterized in that: The backbone network uses SPD convolution for downsampling.

4. The method for detecting small targets in lightweight UAV aerial images based on TFA-YOLO11 according to claim 1, characterized in that: The specific method for the layer feature aggregation module to perform feature aggregation is as follows: the feature map sent to the layer feature aggregation module is jointly extracted with global maximum pooling and global average pooling to obtain global maximum pooling features and global average pooling features; then the global maximum pooling features and global average pooling features are processed separately by a shared multi-layer perceptron, the two feature vectors output by the shared multi-layer perceptron are added together and given nonlinear characteristics through a SiLU activation function to obtain an aggregated feature vector of the feature map.

5. The method for detecting small targets in lightweight UAV aerial images based on TFA-YOLO11 according to claim 1, characterized in that: The detection head module includes three detection heads. The detection heads in the detection head module and the extremely small target detection head are both decoupling heads, which are used to perform target detection on the input feature map.

6. The method for detecting small targets in lightweight UAV aerial images based on TFA-YOLO11 according to claim 1, characterized in that: In the step 3), the training of the TFA-YOLO11 network model uses CIoU loss and distributed focus loss as positioning loss, binary cross entropy loss as classification loss, and uses stochastic gradient descent to update the weights of the network model.

7. A lightweight UAV aerial image small target detection system based on TFA-YOLO11 that implements the method described in claim 1, characterized in that: include: Data enhancement module, which is used to perform data enhancement on drone aerial image datasets; Network model building module, which is used to construct the TFA-YOLO11 network model; A neural network training module is used to train the TFA-YOLO11 network model using the data-enhanced dataset to obtain a trained TFA-YOLO11 network model. The UAV aerial image detection module is used to detect the UAV aerial images to be detected using the trained TFA-YOLO11 network model to obtain the location and category of all targets in the image to be detected.

Images