An unmanned aerial vehicle aerial small target detection method and system based on multi-scale feature enhancement

Abstract

The application provides a kind of unmanned aerial vehicle aerial small target detection method and system based on multi-scale feature enhancement, belongs to the cross field of computer vision and unmanned aerial vehicle technology.Aiming at the problem of insufficient detection accuracy and difficult edge deployment caused by low pixel ratio, large scale difference, dense distribution and complex background of small targets in unmanned aerial vehicle aerial scene, the application takes YOLOv8s as the basic model, adds P2 high resolution detection layer with 160x160 resolution, and integrates double-branch attention enhancement mechanism, multi-form adaptive fusion strategy and multi-scale feature fusion small target enhancement mechanism, through channel and spatial double-dimensional feature reconstruction, adaptive weight distribution and multi-scale feature alignment aggregation, to strengthen small target feature expression and suppress background noise;At the same time, the model is lightened by pruning, quantization and multi-frame optimization.The application realizes the lightweight of the algorithm model while ensuring the detection accuracy, effectively improves the robustness and engineering applicability of small target detection in complex aerial scene.

Description

Technical Field

[0001] This invention relates to the interdisciplinary fields of computer vision, unmanned aerial vehicle (UAV) technology, and deep learning. Specifically, it relates to a method and system for detecting small targets in UAV aerial photography based on multi-scale feature enhancement, which is applicable to real-time detection of small targets in scenarios such as UAV inspection and monitoring, emergency rescue, traffic monitoring, and agricultural plant protection. Background Technology

[0002] With the rapid development of artificial intelligence and drone technology, drone aerial photography plays a vital role in fields such as inspection and monitoring, emergency rescue, and traffic monitoring due to its high mobility and wide coverage. Small target detection in aerial images is a key task of drone visual perception. However, due to the unique perspective of drones, targets typically exhibit characteristics such as low pixel ratio, large scale differences, and dense distribution. Furthermore, they are often affected by complex background interference, making it difficult for traditional detection algorithms to effectively extract discriminative features, resulting in insufficient detection accuracy. In addition, drone edge computing platforms have limited computing power and storage resources, while mainstream deep learning detection algorithms typically have high computational complexity, making it difficult to meet the demands of real-time detection. Therefore, how to improve the accuracy of small target detection while ensuring detection efficiency has become a pressing technical challenge.

[0003] In existing technologies, single-stage detection algorithms based on the YOLO series are widely used in target detection tasks due to their advantages in balancing speed and accuracy. Among them, YOLOv8s, as a relatively advanced algorithm, has strong feature extraction capabilities, but it still has significant limitations in small target detection scenarios in UAV aerial photography. First, its ability to perceive ultra-small targets is insufficient, resulting in a high false negative rate; second, its fixed feature fusion strategy is difficult to adapt to drastic changes in target scale, leading to uneven detection accuracy across multiple scales; furthermore, under complex background interference, the algorithm's ability to suppress noise is weak, further affecting detection performance. Therefore, there is an urgent need for a detection method that can effectively enhance multi-scale feature representation while maintaining model lightweightness, in order to solve the key technical bottlenecks in small target detection in UAV aerial photography. Summary of the Invention

[0004] To address the technical challenges posed by the low pixel ratio, large scale differences, dense distribution, and complex backgrounds of small targets in UAV aerial photography scenarios, which lead to insufficient accuracy of traditional detection algorithms and difficulty in adapting to edge computing platforms, this invention provides a method and system for small target detection in UAV aerial photography based on multi-scale feature enhancement. This invention uses YOLOv8s as the base model and adds a 160×160 resolution P2 high-resolution detection layer, integrates a dual-branch attention enhancement mechanism, a multi-form adaptive fusion mechanism, and a multi-scale feature fusion small target enhancement mechanism, supplemented by a lightweight model deployment strategy. This achieves both high detection accuracy and a lightweight algorithm model, effectively improving the robustness and engineering applicability of small target detection in complex aerial photography scenarios, and adapting to the real-time inference requirements of UAV edge computing platforms.

[0005] The technical means employed in this invention are as follows:

[0006] A method for small target detection in UAV aerial photography based on multi-scale feature enhancement includes: S1. Collect raw images from drone aerial photography scenes, label small target bounding boxes and category labels, and construct a drone small target raw dataset; S2. Perform size standardization and composite data augmentation on the original dataset, and complete the division of the training set, validation set, and test set to construct a standardized UAV small target detection dataset; S3. Construct a multi-scale feature enhancement network. Based on the YOLOv8s architecture, add a 160×160 resolution P2 detection layer on the basis of the original detection layer, and integrate a dual-branch attention enhancement mechanism, a multi-form adaptive fusion mechanism, and a multi-scale feature fusion small target enhancement mechanism. Input the standardized UAV small target detection dataset into the network for training to obtain the trained detection model. S4. Prune, quantize, and convert the format of the trained detection model to generate a lightweight inference model and deploy it to the UAV edge computing platform; S5. Input the aerial images collected in real time by the UAV into the lightweight inference model deployed on the edge computing platform. After small target detection and post-processing, output the detection results including target bounding boxes, categories and confidence levels.

[0007] Furthermore, in step S1, the resolution of the original aerial images covers 1080P and 2K, and the target categories include pedestrians, vehicles, bicycles and motorcycles; the bounding boxes and category labels of small targets are manually labeled using annotation tools to construct an original dataset of no less than 10,000 drone small targets.

[0008] Further, step S2 includes: S21. Scale all original images to 640×640 pixels to complete size standardization; S22. A composite data augmentation strategy, including Mosaic stitching, MixUp interpolation, HSV color space perturbation, and random flipping, scaling, and translation, is used to expand the dataset distribution boundary. S23. Construct a standardized UAV small target detection dataset with annotations by dividing the dataset according to the ratio of training set: validation set: test set = 8:1:1.

[0009] Further, step S3 includes: S31. Based on the original 80×80 resolution P3 detection layer, 40×40 resolution P4 detection layer, and 20×20 resolution P5 detection layer of YOLOv8s, the FPN and PAN feature fusion structure is extended vertically, and a new 160×160 resolution P2 detection layer is added to capture ultra-small targets with a very small pixel ratio. S32. Construct a dual-branch attention enhancement mechanism, dividing the input features according to the number of channels. Each sub-level feature group optimizes feature representation through channel feature reconstruction branch and spatial feature reconstruction branch respectively, enhancing fine-grained features of small targets and suppressing background noise; S33. A multi-form adaptive fusion mechanism is adopted, and a channel gating mechanism is introduced to divide the feature maps of each level of FPN into channels and adaptive weight allocation, so as to dynamically balance the shallow spatial details and deep semantic information. S34. Introduce a multi-scale feature fusion small target enhancement mechanism, which progressively reduces the dimensionality, upsamples and aligns the multi-scale features output by the backbone network, and performs channel shuffling. Combined with the channel prior convolutional attention mechanism, it realizes the gradual alignment and aggregation of multi-level features, and restores the diluted small target detail information.

[0010] Further, step S32 includes: S321. Input feature map Classified by the number of channels Sub-level features, satisfying ,in , ; S322. In the channel feature reconstruction branch, average pooling and max pooling operations are performed on each sub-feature to generate two sets of vectors. and ,through After convolution and ReLU activation, element-wise addition is performed to obtain the first... Channel reconstruction weights for sub-level features The formula is as follows:

[0011] Among them, among them, Indicates the first Sub-level features, and These represent max pooling and average pooling, respectively. express Operation and operate; S323. Reconstruct the channel weights of each sub-level feature. Reconstruct and splice into overall channel weight The formula is as follows:

[0012] in, Indicates the first Channel reconstruction weights for sub-level features This indicates a reshaping and splicing operation; S324. Adjusting the overall channel weights using the Sigmoid function. Activation is performed, channel reconstruction weights are generated, and the input feature map is compared. Multiplication yields the channel reconstruction result. The calculation formula is:

[0013] in, Represents the original input features. Represents the Sigmoid function; S325. In the spatial feature reconstruction branch, the features after channel reconstruction are... Grouping along the channel dimension again yields sub-features. For each sub-feature, perform max pooling and average pooling along the channel axis and concatenate them to obtain the first sub-feature. Spatial reconstruction weights of sub-level features The formula is as follows:

[0014] in, Indicates a splicing operation; S326. Spatial reconstruction weights of each sub-level feature Reshaping and splicing into overall spatial weight The formula is as follows:

[0015] S327, Through Convolution and Sigmoid activation affect the overall spatial weights The process is performed to generate spatial reconstruction weights, which are then compared with the channel reconstruction results. Multiplication yields the final enhanced feature. The formula is as follows:

[0016] in, express convolution.

[0017] Further, step S33 includes: S331, Output via backbone network To process the object, the number of channels and spatial dimensions of each feature map are adjusted to match those of the intermediate layer feature maps. Maintain consistency; S332, Large-scale feature maps of shallow layers ,pass Convolution reduces the number of channels from Compress to A dual-branch structure combining max pooling and average pooling is used for downsampling to generate the output feature map. The formula is as follows:

[0018] in, Indicates global max pooling. Indicates global average pooling. This indicates element-wise summation. express operate, Operation and operate; S333, Deep Small-Scale Feature Map The number of channels is reduced from 1×1 convolution. Reduced to Upsampling is performed using nearest neighbor interpolation to generate the output feature map. The formula is as follows:

[0019] in, This indicates the nearest neighbor interpolation method; S334, Output feature map , With intermediate layer feature maps The feature map is uniformly divided into three sub-feature maps with an equal number of channels. For each sub-feature map, the Sigmoid function is applied to generate adaptive fusion weights, dynamically balancing the contribution weights of shallow and deep features to complete adaptive feature selection and fusion. The formula is as follows:

[0020] in, This represents the activation function. This indicates element-wise multiplication. ; S335. Adaptively fused feature maps , and Stitching along the channel dimension, through Convolution enhances the feature interactions between channels, ultimately generating the output feature map. The calculation formula is:

[0021] in, This indicates a channel splicing operation.

[0022] Further, step S34 includes: S341. Feature map output by the backbone network , , , Progressive dimensionality reduction is performed by gradually compressing the channel dimension through cascaded 1×1 convolutions and combining... The activation function is followed by upsampling using nearest neighbor interpolation to align all feature maps with the 160×160 resolution P2 detection layer, as shown in the following formula:

[0023]

[0024]

[0025] S342, Feature map and , , The features are concatenated along the channel dimension to form a unified multi-level feature representation. A channel shuffling operation is then introduced to redistribute the channel order, promoting information interaction between features of different scales. The multi-level features after channel shuffling are then processed... Convolutional integration, through Convolution reduces the channel dimension from Compress to Generate feature maps The formula is as follows:

[0026] in, This indicates a channel mixing operation. This indicates a splicing operation. This represents a combined operation of convolution, batch normalization, and SiLU activation; S343, Feature Map After thinning by 3×3 convolution, the input channel prior convolutional attention mechanism jointly calculates channel attention and spatial attention and performs dynamic weight allocation to enhance feature representation capabilities and generate enhanced feature maps. The formula is as follows:

[0027] in, This indicates the channel prior convolutional attention operation; S344, Enhance feature map The feature map before the P2 detection layer is added element-wise to improve the detection performance of small targets.

[0028] Further, step S4 includes: S41. Prune the trained detection model and perform INT8 quantization to remove redundant parameters and convert floating-point parameters to integers to reduce the number of model parameters and computational overhead. S42. Convert the pruned and quantized model into the ONNX general format, and then perform targeted optimization using TensorRT, OpenVINO or MNN frameworks to generate inference models for drone edge computing platforms that are adapted to GPU, CPU and ARM architectures respectively. S43. Deploy the optimized inference model to the UAV embedded computing platform to achieve real-time detection of small targets in aerial images.

[0029] Further, step S5 includes: S51. Input the aerial images collected in real time by the UAV into the inference model deployed on the edge computing platform, and generate the initial detection results through forward inference; S52. Filter the initial detection results using a confidence threshold, with the confidence threshold set to 0.5; S53. Perform NMS non-maximum suppression processing on the filtered detection results, and set the IoU threshold to 0.45; S54, Output format is The list of test results, among which , The coordinates of the top left corner of the bounding box. , The coordinates of the bottom right corner To test the confidence level, Index the target category.

[0030] This invention also provides a UAV aerial small target detection system based on the above-mentioned UAV aerial small target detection method based on multi-scale feature enhancement, comprising: a data acquisition and annotation module, a data preprocessing module, a multi-scale feature enhancement network module, a model deployment module, and a real-time detection module, wherein: The data acquisition and annotation module is used to acquire raw images from drone aerial photography scenes and annotate small target bounding boxes and category labels to construct a drone small target raw dataset; The data preprocessing module is used to perform size standardization, composite data augmentation, and dataset partitioning on the original dataset to construct a standardized UAV small target detection dataset. The multi-scale feature enhancement network module is used to construct a multi-scale feature enhancement network. Based on the YOLOv8s architecture, a 160×160 resolution P2 detection layer is added on the basis of the original detection layer. It also integrates a dual-branch attention enhancement mechanism, a multi-form adaptive fusion mechanism, and a multi-scale feature fusion small target enhancement mechanism. The standardized UAV small target detection dataset is input into the network for training to obtain a trained detection model. The model deployment module is used to prune, quantize, and convert the format of the trained detection model to generate a lightweight inference model and deploy it to the UAV edge computing platform. The real-time detection module inputs aerial images collected by the UAV in real time into a lightweight inference model deployed on an edge computing platform. After small target detection and post-processing, it outputs detection results including target bounding boxes, categories, and confidence levels.

[0031] Compared with the prior art, the present invention has the following advantages: 1. The UAV aerial photography small target detection method based on multi-scale feature enhancement provided by this invention achieves effective capture of ultra-small targets with extremely small pixel ratios (minimum detectable 4×4 pixel targets) by adding a 160×160 resolution P2 high-resolution detection layer and cooperating with the vertically extended FPN-PAN feature fusion structure, which significantly makes up for the problem of insufficient perception capability of the original YOLOv8s model for ultra-small targets.

[0032] 2. The UAV aerial photography small target detection method based on multi-scale feature enhancement provided by this invention designs a dual-branch attention enhancement mechanism. Through the synergistic effect of the channel feature reconstruction branch and the spatial feature reconstruction branch, it realizes the enhanced expression of fine-grained features of small targets and the effective suppression of background noise, thereby improving the saliency of small target features in complex aerial photography scenarios.

[0033] 3. The UAV aerial photography small target detection method based on multi-scale feature enhancement provided by this invention designs a multi-form adaptive fusion strategy. By introducing a channel gating mechanism to perform channel division and adaptive weight allocation on feature maps at each level, a dynamic balance between shallow spatial details and deep semantic information is achieved, which effectively adapts to the problem of drastic changes in target scale caused by changes in UAV flight altitude.

[0034] 4. The UAV aerial photography small target detection method based on multi-scale feature enhancement provided by this invention designs a multi-scale feature fusion small target enhancement mechanism. By performing progressive dimensionality reduction, upsampling alignment, channel shuffling, and channel prior convolutional attention enhancement on multi-scale features, it realizes the gradual alignment and aggregation of multi-level features, restores the diluted small target detail information, and improves the detection capability of dense small targets.

[0035] 5. The UAV aerial photography small target detection method based on multi-scale feature enhancement provided by this invention designs a lightweight model deployment scheme. By pruning, INT8 quantization and ONNX format conversion of the trained model, and combining targeted optimization of TensorRT, OpenVINO and MNN frameworks, the detection model can achieve efficient real-time inference on GPU, CPU and ARM architecture UAV edge computing platforms, taking into account both detection accuracy and model lightweight requirements.

[0036] 6. The present invention provides a method for detecting small targets in UAV aerial photography based on multi-scale feature enhancement, which designs a... In summary, the technical solution of this invention solves the problems of insufficient accuracy in small target detection by UAV aerial photography, difficulty in adapting models to edge computing platforms, and uneven accuracy in multi-scale target detection in the prior art.

[0037] Based on the above reasons, this invention can be widely applied in the field of small target detection in computer vision drone aerial photography. Attached Figure Description

[0038] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0039] Figure 1 This is a flowchart of the method of the present invention.

[0040] Figure 2 This is a schematic diagram of the overall framework of the Multi-Scale Feature Enhancement Network (MSFENet) of this invention.

[0041] Figure 3 This is a schematic diagram of the dual-branch attention enhancement module (DBAEM) of the present invention.

[0042] Figure 4 This is a schematic diagram of the Multi-Form Adaptive Fusion (MFAF) module structure of the present invention.

[0043] Figure 5This is a schematic diagram of the structure of the Multi-Scale Feature Fusion Small Target Enhancement Module (MFFSOE) of the present invention. Detailed Implementation

[0044] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0045] It should be noted that the terms "comprising" and "having" and any variations thereof in the specification, claims and accompanying drawings of this invention are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such process, method, product or device.

[0046] like Figure 1 As shown, this invention provides a method for detecting small targets in UAV aerial photography based on multi-scale feature enhancement, including: S1. Collect raw images from drone aerial photography scenes, label small target bounding boxes and category labels, and construct a drone small target raw dataset; S2. Perform size standardization and composite data augmentation on the original dataset, and complete the division of the training set, validation set, and test set to construct a standardized UAV small target detection dataset; S3. Construct a multi-scale feature enhancement network based on YOLOv8s architecture. Add a 160×160 resolution P2 detection layer to the existing detection layer, and integrate a dual-branch attention enhancement mechanism, a multi-form adaptive fusion mechanism, and a multi-scale feature fusion small target enhancement mechanism. Input the standardized UAV small target detection dataset into the network for training to obtain the trained detection model. In this embodiment, model training is based on Ubuntu 20.04 system, using a single NVIDIA RTX 3080Ti GPU (12GB VRAM), and the deep learning frameworks are Python 3.8.18 and PyTorch 1.11. Stochastic gradient descent (SGD) is used as the optimizer, with an initial learning rate of 0.01, a batch size of 4, and 300 training epochs. The loss function uses classification loss (BCE) and CIoU. The joint loss of bounding box regression balances classification accuracy and localization accuracy. During training, the model performance is monitored in real time using a validation set, and an early stopping strategy (training is stopped if there is no performance improvement after 10 consecutive epochs) is adopted to prevent overfitting, resulting in a fully trained detection model.

[0047] S4. Prune, quantize, and convert the format of the trained detection model to generate a lightweight inference model and deploy it to the UAV edge computing platform; S5. Input the aerial images collected in real time by the UAV into the lightweight inference model deployed on the edge computing platform. After small target detection and post-processing, output the detection results including target bounding boxes, categories and confidence levels.

[0048] In a specific implementation, as a preferred embodiment of the present invention, in step S1, the resolution of the original aerial images covers 1080P and 2K, and the target categories include pedestrians, vehicles, bicycles and motorcycles; the bounding boxes and category labels of small targets are manually labeled using annotation tools to construct an original dataset of no less than 10,000 drone small targets.

[0049] In a preferred embodiment of the present invention, step S2 involves standardizing the original drone aerial images to eliminate interference from factors such as small sample sizes, inconsistent backgrounds, and inconsistent dimensions, thereby providing a high-quality dataset for model training. The specific steps are as follows: S21. Scale all raw drone aerial images to 640×640 pixels to standardize the size, ensure consistency of model input dimensions, and reduce memory access overhead for parallel computing; S22. The dataset distribution boundary is expanded by using a composite data augmentation strategy of Mosaic stitching (randomly stitching 4 images into 1 image), MixUp interpolation (linearly fusing two images), HSV color space perturbation (adjusting brightness, contrast, and saturation), and random flipping, scaling, and translation, thereby improving the robustness of the model to interference factors such as changes in lighting, occlusion, and deformation. S23. Divide the enhanced dataset into training set: validation set: test set = 8:1:1 to construct a standardized UAV small target detection dataset with annotations. The supplementary dataset categories are labeled in VOC format.

[0050] In a specific implementation, as a preferred embodiment of the present invention, in step S3, based on the YOLOv8s architecture, optimizations are made to address the problem of drastic changes in target scale during UAV aerial photography. A new P2 small target detection layer is added, and a dual-branch attention enhancement mechanism, a multi-form adaptive fusion mechanism, and a multi-scale feature fusion small target enhancement mechanism are integrated. The overall framework is as follows: Figure 2 As shown, it specifically includes: S31. Based on the original 80×80 resolution P3 detection layer, 40×40 resolution P4 detection layer, and 20×20 resolution P5 detection layer of YOLOv8s, the FPN and PAN feature fusion structure is extended vertically, and a new 160×160 resolution P2 detection layer is added to capture ultra-small targets with extremely small pixel ratios (the smallest detectable target is 4×4 pixels), making up for the original model's lack of perception of ultra-small targets. S32. Construct a dual-branch attention enhancement mechanism, such as Figure 3 As shown, the input features are divided according to the number of channels. Each sub-level feature group optimizes feature representation through channel feature reconstruction branch and spatial feature reconstruction branch respectively, enhancing fine-grained features of small targets and suppressing background noise; S33. Employ a multi-form adaptive fusion mechanism, such as Figure 4 As shown, a channel gating mechanism is introduced to divide the feature maps of each level of FPN into channels and adaptively allocate weights, dynamically balancing shallow spatial details and deep semantic information, and adapting to the drastic changes in target scale caused by changes in the flight altitude of the UAV. S34. Introduce a multi-scale feature fusion mechanism to enhance small targets, such as... Figure 5 As shown, progressive dimensionality reduction, upsampling alignment, and channel shuffling are performed on the multi-scale features output by the backbone network. Combined with the channel prior convolutional attention (CPCA) mechanism, the multi-level features are gradually aligned and aggregated to restore the diluted details of small targets.

[0051] In a specific implementation, as a preferred embodiment of the present invention, step S32 includes: S321. Input feature map Classified by the number of channels Sub-level features, satisfying ,in , ; S322. In the channel feature reconstruction branch, average pooling and max pooling operations are performed on each sub-feature to generate two sets of vectors. and ,through After convolution and ReLU activation, element-wise addition is performed to obtain the first... Channel reconstruction weights for sub-level features The formula is as follows:

[0052] Among them, among them, Indicates the first Sub-level features, and These represent max pooling and average pooling, respectively. express Operation and operate; S323. Reconstruct the channel weights of each sub-level feature. Reconstruct and splice into overall channel weight The formula is as follows:

[0053] in, Indicates the first Channel reconstruction weights for sub-level features This indicates a reshaping and splicing operation; S324. Adjusting the overall channel weights using the Sigmoid function. Activation is performed, channel reconstruction weights are generated, and the input feature map is compared. Multiplication yields the channel reconstruction result. The calculation formula is:

[0054] in, Represents the original input features. Represents the Sigmoid function; S325. In the spatial feature reconstruction branch, the features after channel reconstruction are... Grouping along the channel dimension again yields sub-features. For each sub-feature, perform max pooling and average pooling along the channel axis and concatenate them to obtain the first sub-feature. Spatial reconstruction weights of sub-level features The formula is as follows:

[0055] in, Indicates a splicing operation; S326. Spatial reconstruction weights of each sub-level feature Reshaping and splicing into overall spatial weight The formula is as follows:

[0056] S327, Through Convolution and Sigmoid activation affect the overall spatial weights The process is performed to generate spatial reconstruction weights, which are then compared with the channel reconstruction results. Multiplication yields the final enhanced feature. The formula is as follows:

[0057] in, express convolution.

[0058] In a specific implementation, as a preferred embodiment of the present invention, step S33 includes: S331, Output via backbone network To process the object, the number of channels and spatial dimensions of each feature map are adjusted to match those of the intermediate layer feature maps. Maintain consistency; S332, Large-scale feature maps of shallow layers ,pass Convolution reduces the number of channels from Compress to A dual-branch structure combining max pooling and average pooling is used for downsampling to generate the output feature map. The formula is as follows:

[0059] in, Indicates global max pooling. Indicates global average pooling. This indicates element-wise summation. express operate, Operation and operate; S333, Deep Small-Scale Feature Map The number of channels is reduced from 1×1 convolution. Reduced to Upsampling is performed using nearest neighbor interpolation to generate the output feature map. The formula is as follows:

[0060] in, This indicates the nearest neighbor interpolation method; S334, Output feature map , With intermediate layer feature maps The feature map is uniformly divided into three sub-feature maps with an equal number of channels. For each sub-feature map, the Sigmoid function is applied to generate adaptive fusion weights, dynamically balancing the contribution weights of shallow and deep features to complete adaptive feature selection and fusion. The formula is as follows:

[0061] in, This represents the activation function. This indicates element-wise multiplication. ; S335. Adaptively fused feature maps , and Stitching along the channel dimension, through Convolution enhances the feature interactions between channels, ultimately generating the output feature map. The calculation formula is:

[0062] in, This indicates a channel splicing operation.

[0063] In a specific implementation, as a preferred embodiment of the present invention, step S34 includes: S341. Feature map output by the backbone network , , , Progressive dimensionality reduction is performed by gradually compressing the channel dimension through cascaded 1×1 convolutions and combining... The activation function is followed by upsampling using nearest neighbor interpolation to align all feature maps with the 160×160 resolution P2 detection layer, as shown in the following formula:

[0064]

[0065]

[0066] S342, Feature map and , , The features are concatenated along the channel dimension to form a unified multi-level feature representation. A channel shuffling operation is then introduced to redistribute the channel order, promoting information interaction between features of different scales. The multi-level features after channel shuffling are then processed... Convolutional integration, through Convolution reduces the channel dimension from Compress to Generate feature maps The formula is as follows:

[0067] in, This indicates a channel mixing operation. This indicates a splicing operation. This represents a combined operation of convolution, batch normalization, and SiLU activation; S343, Feature Map After thinning by 3×3 convolution, the input channel prior convolutional attention mechanism jointly calculates channel attention and spatial attention and performs dynamic weight allocation to enhance feature representation capabilities and generate enhanced feature maps. The formula is as follows:

[0068] in, This indicates the channel prior convolutional attention operation; S344, Enhance feature map The feature map before the P2 detection layer is added element-wise to improve the detection performance of small targets.

[0069] In a specific implementation, as a preferred embodiment of the present invention, step S4 includes: S41. Prune the trained detection model with a pruning rate of 0.2, and perform INT8 quantization to remove redundant parameters and convert floating-point parameters to integers to reduce the number of model parameters and computational overhead. S42. Convert the pruned and quantized model into the ONNX general format, and then perform targeted optimization using TensorRT, OpenVINO or MNN frameworks to generate inference models for drone edge computing platforms that are adapted to GPU, CPU and ARM architectures respectively. S43. Deploy the optimized inference model to the UAV embedded computing platform to achieve real-time detection of small targets in aerial images.

[0070] In a specific implementation, as a preferred embodiment of the present invention, step S5 includes: S51. Input the aerial images collected in real time by the UAV into the inference model deployed on the edge computing platform, and generate the initial detection results through forward inference; S52. Filter the initial detection results using a confidence threshold, with the confidence threshold set to 0.5; S53. Perform NMS non-maximum suppression processing on the filtered detection results, and set the IoU threshold to 0.45; S54, Output format is The list of test results, among which , The coordinates of the top left corner of the bounding box. , The coordinates of the bottom right corner To test the confidence level, Index the target category.

[0071] This invention also provides a UAV aerial small target detection system based on multi-scale feature enhancement, implemented by the above-mentioned UAV aerial small target detection method, comprising: a data acquisition and annotation module, a data preprocessing module, a multi-scale feature enhancement network module, a model deployment module, and a real-time detection module, wherein: The data acquisition and annotation module is used to acquire raw images from drone aerial photography scenes and annotate small target bounding boxes and category labels to construct a drone small target raw dataset; The data preprocessing module is used to perform size standardization, composite data augmentation, and dataset partitioning on the original dataset to construct a standardized UAV small target detection dataset. The multi-scale feature enhancement network module is used to construct a multi-scale feature enhancement network. Based on the YOLOv8s architecture, a 160×160 resolution P2 detection layer is added on the basis of the original detection layer. It also integrates a dual-branch attention enhancement mechanism, a multi-form adaptive fusion mechanism, and a multi-scale feature fusion small target enhancement mechanism. The standardized UAV small target detection dataset is input into the network for training to obtain a trained detection model. The model deployment module is used to prune, quantize, and convert the format of the trained detection model to generate a lightweight inference model and deploy it to the UAV edge computing platform. The real-time detection module inputs aerial images collected by the UAV in real time into a lightweight inference model deployed on an edge computing platform. After small target detection and post-processing, it outputs detection results including target bounding boxes, categories, and confidence levels.

[0072] The embodiments of the present invention are described simply because they correspond to those in the embodiments above. For any similarities, please refer to the descriptions in the embodiments above, which will not be elaborated here.

[0073] Example To verify the effectiveness of this invention, experiments and ablation studies were conducted on the VisDrone2019 UAV small target public dataset. The experimental results are shown in Table 1. As can be seen from Table 1, the proposed Multi-Scale Feature Enhancement Network (MSFENet) achieves a precision of 56.9%, a recall of 45.2%, a mAP@0.5 of 47.8%, and a mAP@0.5:0.95 of 29.5% on the VisDrone2019 dataset, with a parameter count of 20.92M. All core metrics are significantly better than the original YOLOv8s model, achieving an effective balance between detection accuracy and model lightweighting.

[0074] Ablation experiments were conducted on the VisDrone2019 dataset for each improved module, and the results are shown in Table 2. As can be seen from Table 2, the addition of the P2 detection layer improved mAP@0.5 by 4.9%; the addition of the dual-branch attention enhancement module (DBAEM) further improved mAP@0.5 by 1.0%; the addition of the multi-form adaptive fusion (MFAF) strategy improved mAP@0.5 by 0.8%; the addition of the multi-scale feature fusion small target enhancement module (MFFSOE) improved mAP@0.5 by 0.1%; after integrating all modules, the model's mAP@0.5 improved by 8.2% compared to the baseline YOLOv8s, verifying the effectiveness of each module and the synergistic gain effect.

[0075] Table 1. Experimental results of this invention on the VisDrone2019 dataset.

[0076] Table 2 Ablation experimental results of the present invention on the VisDrone2019 dataset

[0077] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting small targets in UAV aerial photography based on multi-scale feature enhancement, characterized in that, include: S1. Collect raw images from drone aerial photography scenes, label small target bounding boxes and category labels, and construct a drone small target raw dataset; S2. Perform size standardization and composite data augmentation on the original dataset, and complete the division of the training set, validation set, and test set to construct a standardized UAV small target detection dataset; S3. Construct a multi-scale feature enhancement network. Based on the YOLOv8s architecture, add a 160×160 resolution P2 detection layer on the basis of the original detection layer, and integrate a dual-branch attention enhancement mechanism, a multi-form adaptive fusion mechanism, and a multi-scale feature fusion small target enhancement mechanism. Input the standardized UAV small target detection dataset into the network for training to obtain the trained detection model. S4. Prune, quantize, and convert the format of the trained detection model to generate a lightweight inference model and deploy it to the UAV edge computing platform; S5. Input the aerial images collected in real time by the UAV into the lightweight inference model deployed on the edge computing platform. After small target detection and post-processing, output the detection results including target bounding boxes, categories and confidence levels.

2. The method for detecting small targets in UAV aerial photography based on multi-scale feature enhancement according to claim 1, characterized in that, In step S1, the resolution of the original aerial images covers 1080P and 2K, and the target categories include pedestrians, vehicles, bicycles and motorcycles; We manually labeled the bounding boxes and category labels of small targets using annotation tools to construct an original dataset of no less than 10,000 drone small targets.

3. The method for detecting small targets in UAV aerial photography based on multi-scale feature enhancement according to claim 1, characterized in that, Step S2 includes: S21. Scale all original images to 640×640 pixels to complete size standardization; S22. A composite data augmentation strategy, including Mosaic stitching, MixUp interpolation, HSV color space perturbation, and random flipping, scaling, and translation, is used to expand the dataset distribution boundary. S23. Construct a standardized UAV small target detection dataset with annotations by dividing the dataset according to the ratio of training set: validation set: test set = 8:1:

1.

4. The method for detecting small targets in UAV aerial photography based on multi-scale feature enhancement according to claim 1, characterized in that, Step S3 includes: S31. Based on the original 80×80 resolution P3 detection layer, 40×40 resolution P4 detection layer, and 20×20 resolution P5 detection layer of YOLOv8s, the FPN and PAN feature fusion structure is extended vertically, and a new 160×160 resolution P2 detection layer is added to capture ultra-small targets with a very small pixel ratio. S32. Construct a dual-branch attention enhancement mechanism, dividing the input features according to the number of channels. Each sub-level feature group optimizes feature representation through channel feature reconstruction branch and spatial feature reconstruction branch respectively, enhancing fine-grained features of small targets and suppressing background noise; S33. A multi-form adaptive fusion mechanism is adopted, and a channel gating mechanism is introduced to divide the feature maps of each level of FPN into channels and adaptive weight allocation, so as to dynamically balance the shallow spatial details and deep semantic information. S34. Introduce a multi-scale feature fusion small target enhancement mechanism, which progressively reduces the dimensionality, upsamples and aligns the multi-scale features output by the backbone network, and performs channel shuffling. Combined with the channel prior convolutional attention mechanism, it realizes the gradual alignment and aggregation of multi-level features, and restores the diluted small target detail information.

5. The method for detecting small targets in UAV aerial photography based on multi-scale feature enhancement according to claim 4, characterized in that, Step S32 includes: S321. Input feature map Classified by the number of channels Sub-level features, satisfying ,in , ; S322. In the channel feature reconstruction branch, average pooling and max pooling operations are performed on each sub-feature to generate two sets of vectors. and ,through After convolution and ReLU activation, element-wise addition is performed to obtain the first... Channel reconstruction weights for sub-level features The formula is as follows: Among them, among them, Indicates the first Sub-level features, and These represent max pooling and average pooling, respectively. express Operation and operate; S323. Reconstruct the channel weights of each sub-level feature. Reconstruct and splice into overall channel weight The formula is as follows: in, Indicates the first Channel reconstruction weights for sub-level features This indicates a reshaping and splicing operation; S324. Adjusting the overall channel weights using the Sigmoid function. Activation is performed, channel reconstruction weights are generated, and the input feature map is compared. Multiplication yields the channel reconstruction result. The calculation formula is: in, Represents the original input features. Represents the Sigmoid function; S325. In the spatial feature reconstruction branch, the features after channel reconstruction are... Grouping along the channel dimension again yields sub-features. For each sub-feature, perform max pooling and average pooling along the channel axis and concatenate them to obtain the first sub-feature. Spatial reconstruction weights of sub-level features The formula is as follows: in, Indicates a splicing operation; S326. Spatial reconstruction weights of each sub-level feature Reshaping and splicing into overall spatial weight The formula is as follows: S327, Through Convolution and Sigmoid activation affect the overall spatial weights The process is performed to generate spatial reconstruction weights, which are then compared with the channel reconstruction results. Multiplication yields the final enhanced feature. The formula is as follows: in, express convolution.

6. The method for detecting small targets in UAV aerial photography based on multi-scale feature enhancement according to claim 4, characterized in that, Step S33 includes: S331, Output via backbone network To process the object, the number of channels and spatial dimensions of each feature map are adjusted to match those of the intermediate layer feature maps. Maintain consistency; S332, Large-scale feature maps of shallow layers ,pass Convolution reduces the number of channels from Compress to A dual-branch structure combining max pooling and average pooling is used for downsampling to generate the output feature map. The formula is as follows: in, Indicates global max pooling. Indicates global average pooling. This indicates element-wise summation. express operate, Operation and operate; S333, Deep Small-Scale Feature Map The number of channels is reduced from 1×1 convolution. Reduced to Upsampling is performed using nearest neighbor interpolation to generate the output feature map. The formula is as follows: in, This indicates the nearest neighbor interpolation method; S334, Output feature map , With intermediate layer feature maps The feature map is uniformly divided into three sub-feature maps with an equal number of channels. For each sub-feature map, the Sigmoid function is applied to generate adaptive fusion weights, dynamically balancing the contribution weights of shallow and deep features to complete adaptive feature selection and fusion. The formula is as follows: in, This represents the activation function. This indicates element-wise multiplication. ; S335. Adaptively fused feature maps , and Stitching along the channel dimension, through Convolution enhances the feature interactions between channels, ultimately generating the output feature map. The calculation formula is: in, This indicates a channel splicing operation.

7. The method for detecting small targets in UAV aerial photography based on multi-scale feature enhancement according to claim 4, characterized in that, Step S34 includes: S341. Feature map output by the backbone network , , , Progressive dimensionality reduction is performed by gradually compressing the channel dimension through cascaded 1×1 convolutions and combining... The activation function is followed by upsampling using nearest neighbor interpolation to align all feature maps with the 160×160 resolution P2 detection layer, as shown in the following formula: S342, Feature map and , , The features are concatenated along the channel dimension to form a unified multi-level feature representation. A channel shuffling operation is then introduced to redistribute the channel order, promoting information interaction between features of different scales. The multi-level features after channel shuffling are then processed... Convolutional integration, through Convolution reduces the channel dimension from Compress to Generate feature maps The formula is as follows: in, This indicates a channel mixing operation. This indicates a splicing operation. This represents a combined operation of convolution, batch normalization, and SiLU activation; S343, Feature Map After thinning by 3×3 convolution, the input channel prior convolutional attention mechanism jointly calculates channel attention and spatial attention and performs dynamic weight allocation to enhance feature representation capabilities and generate enhanced feature maps. The formula is as follows: in, This indicates the channel prior convolutional attention operation; S344, Enhance feature map The feature map before the P2 detection layer is added element-wise to improve the detection performance of small targets.

8. The method for detecting small targets in UAV aerial photography based on multi-scale feature enhancement according to claim 1, characterized in that, Step S4 includes: S41. Prune the trained detection model and perform INT8 quantization to remove redundant parameters and convert floating-point parameters to integers to reduce the number of model parameters and computational overhead. S42. Convert the pruned and quantized model into the ONNX general format, and then perform targeted optimization using TensorRT, OpenVINO or MNN frameworks to generate inference models for drone edge computing platforms that are adapted to GPU, CPU and ARM architectures respectively. S43. Deploy the optimized inference model to the UAV embedded computing platform to achieve real-time detection of small targets in aerial images.

9. The method for detecting small targets in UAV aerial photography based on multi-scale feature enhancement according to claim 1, characterized in that, Step S5 includes: S51. Input the aerial images collected in real time by the UAV into the inference model deployed on the edge computing platform, and generate the initial detection results through forward inference; S52. Filter the initial detection results using a confidence threshold, with the confidence threshold set to 0.5; S53. Perform NMS non-maximum suppression processing on the filtered detection results, and set the IoU threshold to 0.45; S54, Output format is The list of test results, among which , The coordinates of the top left corner of the bounding box. , The coordinates of the bottom right corner To test the confidence level, Index the target category.

10. A UAV aerial small target detection system based on the multi-scale feature enhancement method for UAV aerial photography as described in any one of claims 1-9, characterized in that, include: The system comprises a data acquisition and annotation module, a data preprocessing module, a multi-scale feature enhancement network module, a model deployment module, and a real-time detection module, among which: The data acquisition and annotation module is used to acquire raw images from drone aerial photography scenes and annotate small target bounding boxes and category labels to construct a drone small target raw dataset; The data preprocessing module is used to perform size standardization, composite data augmentation, and dataset partitioning on the original dataset to construct a standardized UAV small target detection dataset. The multi-scale feature enhancement network module is used to construct a multi-scale feature enhancement network. Based on the YOLOv8s architecture, a 160×160 resolution P2 detection layer is added on the basis of the original detection layer. It also integrates a dual-branch attention enhancement mechanism, a multi-form adaptive fusion mechanism, and a multi-scale feature fusion small target enhancement mechanism. The standardized UAV small target detection dataset is input into the network for training to obtain a trained detection model. The model deployment module is used to prune, quantize, and convert the format of the trained detection model to generate a lightweight inference model and deploy it to the UAV edge computing platform. The real-time detection module inputs aerial images collected by the UAV in real time into a lightweight inference model deployed on an edge computing platform. After small target detection and post-processing, it outputs detection results including target bounding boxes, categories, and confidence levels.

Images