A small target detection method for unmanned aerial vehicle images based on improved detr

Abstract

The application discloses a small target detection method for unmanned aerial vehicle images based on an improved DETR. The application is constructed based on an end-to-end attention target detection model DETR, and three key technical improvements are proposed: a dual-domain hybrid encoder module, an enhanced query selection mechanism and a knowledge distillation strategy. Among them, the dual-domain hybrid encoder module is used to integrate spatial domain and frequency domain features, and only high-level features are operated through a self-attention mechanism, while a dual-domain fusion module is used to combine low-level features; secondly, an enhanced query selection mechanism is introduced, and anchor boxes are generated on multi-scale feature maps, and an extended intersection over union is used to dynamically filter high-score anchor boxes as queries; in addition, through the knowledge distillation strategy, the knowledge of the teacher model is transferred to the lightweight student model, realizing accurate detection and efficient reasoning of small targets. The application has obvious advantages in small target detection accuracy and computational complexity, and can be used for various unmanned aerial vehicle image processing tasks.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision, in particular to a small target detection method for unmanned aerial vehicle images based on improved DETR. BACKGROUND

[0002] With the continuous development of unmanned aerial vehicle technology, unmanned aerial vehicles are increasingly widely used in fields such as surveying, inspection, security and agriculture. However, in high-altitude images obtained by unmanned aerial vehicles, small targets usually have characteristics such as large number, small size and variable target shape, which bring great challenges to target detection algorithms. Existing target detection methods, especially algorithms for general scenes, often have problems such as decreased detection accuracy and increased missed detection rate when facing small targets in unmanned aerial vehicle images.

[0003] DETR (Detection Transformer) is a target detection method based on attention mechanism that has emerged in recent years. By treating target detection as a set prediction problem, it simplifies the dependence on prior anchor boxes of many traditional detectors. However, DETR still has the following shortcomings when detecting small targets and in multi-target dense scenes:

[0004] 1. Loss of spatial resolution: direct processing in the feature pyramid or deeper network layers often leads to insufficient information for small targets in high-dimensional semantic feature maps;

[0005] 2. Single query mechanism: a fixed number of queries cannot dynamically adapt to the diversity of target distribution in images, especially in small target dense areas, and the allocation of query resources is prone to waste or insufficient allocation;

[0006] 3. Model complexity: in scenarios where the unmanned aerial vehicle platform or other computing resources are limited, deploying a larger model will result in higher inference latency and memory overhead. SUMMARY

[0007] The purpose of the present application is to overcome the problems of insufficient precision and efficiency in detecting small targets in unmanned aerial vehicle images in the prior art, and to provide a small target detection method for unmanned aerial vehicle images based on improved DETR. This method is implemented through the following technical solutions:

[0008] S100: receiving an unmanned aerial vehicle image to be subjected to target detection;

[0009] S200: using a backbone network to extract features from the input unmanned aerial vehicle image, and outputting the extracted features to a dual-domain hybrid encoder module;

[0010] S300: dual-domain hybrid encoding step:

[0011] S310: self-attention operation: in the dual-domain hybrid encoder module, the self-attention operation is performed on the highest layer features (s5 layer) from the backbone network;

[0012] S320: dual-domain fusion operation: the features of s2 and s3 layers are fused by the dual-domain fusion module, and the fused features are output to the enhanced query selection mechanism;

[0013] S400: enhanced query selection operation:

[0014] S410: anchor box generation: in the enhanced query selection mechanism, anchor boxes on the multi-scale feature map are generated;

[0015] S420: high-score anchor box selection: using the classification head and the regression head, the high-score anchor box is selected as the query by the extended intersection over union dynamic;

[0016] S500: in the decoder, an auxiliary prediction head is equipped to iteratively optimize the selected query to predict the category and anchor box of the target;

[0017] S600: knowledge distillation step:

[0018] S610: a teacher model based on RT-DETR-R50 and a student model derived from RT-DETR-R18 are constructed, and the student model uses EfficientFormerV2 to replace the traditional ResNet-18 backbone network;

[0019] S620: the distillation loss of classification and positioning is calculated by binary cross-entropy loss, L1 loss and intersection over union loss;

[0020] S630: the weight of the distillation loss is adjusted by linear decay strategy;

[0021] S700: output the detection result containing the target category and positioning information.

[0022] Preferably, the input includes the input unmanned aerial vehicle image to be subjected to target detection.

[0023] Preferably, the backbone network includes ResNet50 as the backbone network of the teacher model and EfficientFormerV2 as the backbone network of the student model.

[0024] Preferably, the specific operation of the dual-domain fusion module in the dual-domain hybrid encoder module is as follows: for a given feature map This module first performs convolution operation, and then performs channel segmentation:

[0025] [X1,X2]=Split(Conv(X) ,[C1,C2], dim = 0);

[0026] In the above formula, Conv(·) represents a convolution layer that adjusts the channel dimension while keeping the spatial dimension unchanged, and C1+C2=C, C1 is set to one quarter of the total number of channels; the first branch X1 is subjected to re-convolution and then passes through a Gaussian error linear unit to introduce nonlinearity:

[0027] X conv =GELU(Conv(X1));

[0028] In order to capture complex feature dependencies, the features X conv In the frequency domain, the following processing is performed:

[0029] X out = al·|IFFT(FFT(Conv(|X conv |)))·Conv(|X conv |)|+Conv(ReLU(X1+Conv(X conv )+b1·|X conv |));

[0030] In the above formula, al and b1 are learnable parameters for balancing the contributions from the frequency domain enhanced features and the spatial residual connection; the outputs of the two branches are spliced together and passed through a final convolution layer to integrate the refined features:

[0031] X final =Conv([X out ,X2]).

[0032] Preferably, the enhanced query selection mechanism includes first generating anchor boxes on multi-scale feature maps using a fixed grid, and then optimizing them through position transformation and selection in logarithmic space. During training, the regression head dynamically adjusts the anchor boxes to match the positions and scales of the targets, and in order to provide the performance of small target detection, the classification head dynamically selects the top k anchor boxes with the highest scores as queries in combination with the expanded IoU indicator.

[0033] Preferably, in the enhanced query selection mechanism, the expanded IoU is defined as follows: for the predicted anchor box B' p and the real anchor box B' gt , the expanded IoU is the intersection over union between the scaled predicted anchor box B' p and the scaled real anchor box B' gt , both of which are scaled by a scaling factor a2>1 while keeping their centers unchanged, and the scaled anchor boxes are:

[0034] B' p =(xp ,y p ,w p ×α2,h p ×α2);

[0035] B′ gt =(x gt ,y gt ,w gt ×α2,h gt ×α2);

[0036] where (x p ,y p ) and (x gt ,y gt ) are the center coordinates, w p ,h p ,w gt ,h gt represent the width and height of the predicted box and the real box respectively, and the extended IoU is defined as:

[0037]

[0038] Preferably, the knowledge distillation strategy comprises constructing a teacher model based on RT-DETR-R50 and a student model derived from RT-DETR-R18, wherein the derived model replaces the traditional ResNet-18 backbone network with EfficientFormerV2; the distillation loss of classification and positioning is calculated through binary cross entropy loss, L1 loss and intersection over union loss; and a linear decay strategy is adopted to adjust the weight of the distillation loss.

[0039] Preferably, the decoder comprises an auxiliary prediction head to iteratively optimize queries to predict target classes and anchor boxes.

[0040] Preferably, the binary cross entropy loss calculation formula in the knowledge distillation strategy is:

[0041]

[0042] In the above formula, s c and t c represent the classification scores output by the final decoder layer of the student model and the teacher model respectively; the L1 loss calculation formula is:

[0043]

[0044] where s b and t b represent the anchor box coordinates predicted by the student model and the teacher model respectively, t o represents the target confidence score of the teacher model; and the intersection over union loss calculation formula is:

[0045]

[0046] wherein the calculation formula of the expanded SIoU is:

[0047] Expanded_SIoU=SIoU-IoU+Expanded_IoU;

[0048] The calculation formula of the distillation loss is:

[0049]

[0050] In the above formula, alpha, beta and gamma are constant coefficients for balancing the contribution of each loss component.

[0051] Preferably, in the model inference stage, a multi-thread or asynchronous data acquisition strategy is adopted, and combined with software and hardware co-optimization, the detection delay is effectively reduced, the real-time performance of the system is significantly improved, and the demand for rapid detection in the unmanned aerial vehicle application scenario is met.

[0052] Preferably, by adopting compression technologies such as model pruning and quantization, the model is optimized, the parameter quantity and calculation quantity of the model are reduced, the limitations of the unmanned aerial vehicle system in terms of flight time and computing resources are overcome, and the method can efficiently run on the unmanned aerial vehicle device with limited resources.

[0053] Preferably, the linear decay strategy includes assigning a higher weight to the distillation loss at the beginning of training. As the training progresses, the weight of the distillation loss gradually decreases.

[0054] Preferably, the output includes outputting the detection result, including target category and positioning information.

[0055] Advantages

[0056] Compared with the prior art, the present application provides a small target detection method for unmanned aerial vehicle images based on improved DETR, which has the following advantages:

[0057] The small target detection method for unmanned aerial vehicle images based on the improved DETR plays a key role in improving detection accuracy, and the dual-domain hybrid encoder module innovatively integrates spatial domain and frequency domain features, and finely processes different levels of features output by the backbone network. When processing the highest layer feature (s5 layer), the self-attention operation enables it to focus on the key details in the high-level semantic information, and strengthens the deep understanding of small target features. The dual-domain fusion module fuses the s2 and s3 layer features, which ingeniously supplements the low-level spatial information. The synergistic effect of high and low layer information is excellent in enhancing high-resolution feature representation. For example, in complex urban mapping unmanned aerial vehicle images, for small targets such as tiny traffic signs on the street or small auxiliary facilities on buildings, this module can accurately capture their features, avoiding the disadvantages of traditional methods that directly process high-dimensional semantic feature maps, resulting in missing small target information. Through strict testing on the VisDrone-2019-DET and UAVVaste data sets, the accuracy (AP) and precision (AP50) of small target detection are significantly improved under different computational complexity models, which effectively proves the outstanding performance of this module in improving accuracy.

[0058] In terms of query resource allocation optimization, the enhanced query selection mechanism brings breakthrough progress. Traditional methods are limited by a fixed number of queries and are difficult to cope with the diversity of target distribution in images, especially in small target dense areas, which often fall into the dilemma of query resource allocation. This mechanism first generates anchor boxes on multi-scale feature maps using a fixed grid, then optimizes the selection in the logarithmic space through position transformation, and in the training process, the regression head dynamically adjusts the anchor box to accurately match the position and scale of the target, and the classification head selects according to the expanded IoU index. In the scene of agricultural plant protection unmanned aerial vehicle monitoring crop diseases and pests, when facing densely distributed small target diseases and pests in large farmland, this mechanism can flexibly allocate query resources according to the actual situation of the target, accurately select high-score anchor boxes as queries, greatly improve the detection efficiency, effectively reduce the occurrence of missed detection and false detection, and ensure the reliability and stability of the detection result.

[0059] In the optimization and adaptation of model performance for unmanned aerial vehicle platforms, the knowledge distillation strategy has shown significant results. By carefully constructing a teacher model based on RT-DETR-R50 and replacing the traditional backbone network of RT-DETR-R18 with EfficientFormerV2 to derive a student model, the binary cross-entropy loss is used to accurately measure the deviation of the student model and the teacher model in classification prediction, the L1 loss is used to calculate the anchor box coordinate difference according to the target confidence score of the teacher model, the intersection over union loss is introduced to help the student model align the small target anchor box, and the linear decay strategy is used to reasonably adjust the distillation loss weight, realizing the efficient migration of knowledge from the teacher model to the student model. In the application of unmanned aerial vehicle infrastructure inspection, the knowledge distillation strategy not only maintains high-precision detection capability while significantly reducing model complexity, effectively reducing model inference delay and memory overhead, but also further reduces detection delay by using multi-thread or asynchronous data acquisition strategies in model inference stage combined with software and hardware co-optimization, and using model pruning, quantization and other compression techniques, successfully overcoming the limitations of unmanned aerial vehicle system in flight time and computing resources, ensuring that the method can run efficiently and stably on resource-limited unmanned aerial vehicle devices, greatly expanding its feasibility and applicability in practical applications, providing strong technical support for intelligent detection of unmanned aerial vehicles in security monitoring, infrastructure inspection, agricultural plant protection, environmental monitoring and other fields, and having immeasurable economic and social value. BRIEF DESCRIPTION OF DRAWINGS

[0060] Figure 1 A model architecture diagram for improving DETR is provided for the present application.

[0061] Figure 2 A dual-domain fusion module architecture diagram is provided for the present application.

[0062] Figure 3 A data diagram of the verification results of the method provided by the present application on the VisDrone-2019-DET dataset is provided.

[0063] Figure 4 A data diagram of the verification results of the method provided by the present application on the UAVVaste dataset is provided.

[0064] Figure 5 A qualitative comparison diagram of the detection results and attention heat maps of the method provided by the present application is provided. DETAILED DESCRIPTION

[0065] With reference to the drawings and the embodiments of the present application, the technical solutions in the embodiments of the present application will be described clearly and completely. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of the present application.

[0066] The purpose of the present application is to provide a small target detection method for unmanned aerial vehicle images based on improved DETR. In order to make the above-mentioned purposes, characteristics and advantages of the present application more obvious and easy to understand, the following will combine the drawings of the embodiments of the present application to make a detailed description. Figures 1-5 The present application will be further described in detail. The specific implementation is as follows:

[0067] Referring to Figure 1 It can be seen that the overall architecture diagram of the model, a small target detection method for unmanned aerial vehicle images based on improved DETR, at least includes the following steps:

[0068] S100: receiving an unmanned aerial vehicle image to be subjected to target detection;

[0069] S200: using a backbone network to extract features of the input unmanned aerial vehicle image, outputting the extracted features to a dual-domain hybrid encoder module, using ResNet50 as the teacher model and using EfficientFormerV2 as the student model, and outputting the extracted features to the dual-domain hybrid encoder module;

[0070] S300: dual-domain hybrid encoding step:

[0071] S310: self-attention operation: in the dual-domain hybrid encoder module, performing self-attention operation on the highest layer features (s5 layer) from the backbone network;

[0072] S320: dual-domain fusion operation: fusing the features of s2 and s3 layers through a dual-domain fusion module, and outputting the fused features to an enhanced query selection mechanism;

[0073] Referring to Figure 2 In the dual-domain fusion module, for a given feature map The module

[0074] The block first performs convolution operation, and then performs channel segmentation:

[0075] [X1,X2]=Split(Conv(X),[C1,C2],dim=0);

[0076] In the above formula, Conv(·) represents a convolution layer that adjusts the channel dimension while keeping the spatial dimension unchanged. The above segmentation operation divides the features after convolution into two different branches: X1 has C1 channels, and X2 has C2 channels, where C1+C2=C. Specifically, C1 is set to one quarter of the total number of channels.

[0077] The first branch X1 is subjected to re-convolution and then passes through a Gaussian error linear unit to introduce nonlinearity:

[0078] X conv =GELU(Conv(X1));

[0079] To capture complex feature dependencies, the features X conv The following processing is performed in the frequency domain:

[0080] X out =α1·|IFFT(FFT(Conv(|X conv |)))·Conv(|X conv |)|+Conv(ReLU(X1+Conv(X conv )+β1·|X conv |));

[0081] In the above formula, α1 and β1 are learnable parameters used to balance the contributions from the frequency domain enhanced features and the spatial residual connection. The outputs of the two branches are concatenated and passed through a final convolution layer to integrate the refined features:

[0082] X final =Conv([X out ,X2]);

[0083] The generated features are then input into the fusion module, and the output features are output to the enhanced query selection mechanism.

[0084] S400: Enhanced query selection operation:

[0085] S410: Anchor box generation: In the enhanced query selection mechanism, anchor boxes on multi-scale feature maps are generated;

[0086] S420: High-score anchor box selection: Using the classification head and the regression head, high-score anchor boxes are dynamically selected as queries through the extended intersection over union;

[0087] For the multi-scale feature maps output by the dual-domain hybrid encoder module, anchor boxes are generated on the multi-scale feature maps using a fixed grid. Then, they are optimized by position transformation and selection in log space. During training, the regression head dynamically adjusts the anchor boxes to match the position and scale of the target. To provide the performance of small target detection, the classification head dynamically selects the top k anchor boxes with the highest scores as queries in combination with the expanded IoU metric.

[0088] For the predicted anchor box B p and the real anchor box B gt , the expanded IoU is defined as the intersection over union between the scaled predicted anchor box B' p and the scaled real anchor box B' gt . Both anchor boxes are scaled by a scaling factor α2>1 while keeping their centers unchanged. The scaled anchor boxes are:

[0089] B' p =(x p ,y p ,w p ×α2,h p ×α2);

[0090] B' gt =(x gt ,y gt ,w gt ×α2,h gt ×α2);

[0091] where (x p ,y p ) and (x gt ,y gt ) are the center coordinates, and w p ,h p ,w gt ,h gt represent the width and height of the predicted and real boxes, respectively.

[0092] The expanded IoU is defined as:

[0093]

[0094] Similarly, the expanded SIoU is defined as:

[0095] Expanded_SIoU=SIoU-IoU+Expanded_IoU.

[0096] S500: In the decoder, an auxiliary prediction head is equipped to iteratively optimize the selected queries to predict the class and anchor box of the target; the decoder, for the queries generated by the above enhanced query selection mechanism, iteratively optimizes the queries to predict the target class and anchor box by equipping an auxiliary prediction head.

[0097] S600: knowledge distillation step:

[0098] S610: a teacher model based on RT-DETR-R50 and a student model derived from RT-DETR-R18 are constructed, the student model uses EfficientFormerV2 to replace the traditional ResNet-18 backbone network; both the teacher model and the student model are embedded with the above-mentioned dual-domain hybrid encoder module and enhanced query selection mechanism;

[0099] S620: the distillation loss of classification and positioning is calculated by binary cross-entropy loss, L1 loss and intersection over union loss;

[0100] S630: the weight of distillation loss is adjusted by linear decay strategy;

[0101] The binary cross-entropy loss is used to calculate the deviation between the classification prediction of the student model and the teacher model:

[0102] In the above formula, s c and t c represent the classification scores output by the final decoder layer of the student model and the teacher model, respectively.

[0103] The L1 loss is used to calculate the difference of anchor box coordinates and is weighted according to the target confidence score of the teacher model:

[0104] In the above formula, s b and t b represent the anchor box coordinates predicted by the student model and the teacher model, respectively, and t o represents the target confidence score of the teacher model.

[0105] The intersection over union loss is used to help the student model align the anchor box of small targets, thereby improving its performance in small target detection, and its calculation formula is:

[0106] The calculation formula of the distillation loss is:

[0107] In the above formula, α, β and γ are constant coefficients that balance the contribution of each loss component.

[0108] The linear decay strategy includes assigning a higher weight to the distillation loss at the beginning of training, and gradually reducing the weight of the distillation loss as the training progresses.

[0109] S700: output the detection results containing target categories and positioning information.

[0110] Referring to Figure 3The method provided in the scheme is verified on the VisDrone-2019-DET dataset, and the model is divided into three groups according to the calculation complexity: a low-computation-complexity (Low Computation) model with a calculation amount less than 50 GFLOPs, a medium-computation-complexity (Medium Computation) model with a calculation amount between 50-100 GFLOPs, and a high-computation-complexity (High Computation) model with a calculation amount greater than 100 GFLOPs. The verification results show that on the VisDrone-2019-DET dataset, the method (SO-DETR) provided in the scheme achieves the highest AP and AP 50 scores in low, medium, and high calculation complexity categories.

[0111] Referring to Figure 4 The method provided in the scheme is verified on the UAVVaste dataset. The verification results show that on the UAVVaste dataset, the method (SO-DETR) provided in the scheme can effectively enhance the detection capability of small targets while maintaining competitive calculation efficiency. In addition, the verification results on the above two datasets show that the method provided in the scheme has shown more excellent performance and robustness on different datasets.

[0112] Referring to Figure 5 The figure shows the heat map of small target detection results in the VisDrone-2019-DET dataset. As shown in the figure, compared with RT-DETR-EV2, the method (SO-DETR-EV2) provided in the scheme significantly reduces the error in small target detection, and the method provided in the scheme has higher accuracy in small target detection, highlighting its excellent performance in complex scenes.

[0113] The embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be mutually referred to.

[0114] The principles and implementation modes of the present application are described by applying specific examples in this paper. The above embodiment description is only used to help understand the method and core idea of the present application; at the same time, for those skilled in the art, according to the idea of the present application, the specific implementation mode and application range will be changed. In summary, the content of the specification should not be understood as a limitation of the present application.

[0115] Technical advantages and application prospects

[0116] The small target detection method for unmanned aerial vehicle images based on the improved DETR provided in the application effectively fuses the spatial domain and frequency domain features through a dual-domain hybrid encoder, significantly improves the detection accuracy of small targets, and optimizes the query resource allocation through an enhanced query selection mechanism to ensure efficient detection in small target dense scenes. Combined with the knowledge distillation strategy, the model realizes lightweight while maintaining high accuracy, which is suitable for real-time deployment on the unmanned aerial vehicle platform with limited computing resources. The method has high adaptability and universality, and is widely applicable to many fields such as security monitoring, infrastructure inspection, agricultural plant protection, environmental monitoring, logistics distribution, disaster emergency response and military reconnaissance, and can significantly improve the intelligent detection capability of unmanned aerial vehicles in complex environments, and has broad application prospects and significant economic and social value.

[0117] Implementation effect

[0118] Through experiments on the VisDrone-2019-DET and UAVVaste data sets, the small target detection method for unmanned aerial vehicle images based on the improved DETR provided in the application shows significant superiority. In addition, the dual-domain hybrid encoder effectively fuses the spatial domain and frequency domain features, enhances the model's ability to recognize small targets in complex backgrounds, and optimizes the query resource allocation through an enhanced query selection mechanism to ensure efficient detection in small target dense scenes. The knowledge distillation strategy makes the model realize lightweight while maintaining high accuracy, which performs well in various application scenarios such as security monitoring, infrastructure inspection and agricultural plant protection. In summary, the method of the application not only significantly improves the performance of small target detection, but also optimizes the computing efficiency and resource utilization, meets the needs of unmanned aerial vehicle image processing for efficiency and accuracy, and shows broad application prospects and significant economic and social benefits.

[0119] It should be noted that in this paper, relational terms such as first and second are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "contain" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or device. Without more limitations, the element defined by the statement "including one" does not exclude the presence of another identical element in the process, method, article or device including the element.

[0120] While embodiments of the application have been shown and described, it is to be understood that the embodiments described are merely exemplary of the principles and application of the present application. Numerous modifications and adaptions can be effected without departing from the spirit and scope of the present application, which is not limited to the exact construction and arrangement described. It is intended, therefore, to cover all modifications and adaptions that fall within the scope of the claims and their equivalents.

Claims

1. An improved DETR-based small target detection method for unmanned aerial vehicle images, characterized by: Comprise the following steps: S100: receiving a UAV image to be subjected to target detection; S200: using a backbone network to perform feature extraction on the input UAV image, and outputting the extracted features to a dual-domain hybrid encoder module; S300: a dual-domain hybrid encoding step: S310: self-attention operation: in the dual-domain hybrid encoder module, a self-attention operation is performed on the highest layer features (L layer) from the backbone network; S320: dual-domain fusion operation: fuse the features of the layers by a dual-domain fusion module and the fused features to an enhanced query selection mechanism; S400: enhanced query selection operation: S410: anchor box generation: in the enhanced query selection mechanism, anchor boxes on a multi-scale feature map are generated; S420: high-score anchor box selection: using a classification head and a regression head, a high-score anchor box is dynamically selected as a query through an extended intersection over union; S500: an auxiliary prediction head is provided in the decoder to iteratively optimize the selected query to predict the category and anchor box of the target; S600: a knowledge distillation step: S610: a teacher model based on RT-DETR-R50 and a student model derived from RT-DETR-R18 are constructed, and the student model uses EfficientFormerV2 to replace the traditional ResNet-18 backbone network; S620: calculate the distillation loss of classification and positioning through binary cross-entropy loss, L1 loss and intersection over union loss; S630: adopt a linear decay strategy to adjust the weight of the distillation loss; S700: output the detection result containing the target category and positioning information.

2. The method of claim 1, wherein the method is based on an improved DETR for small target detection of images of unmanned aerial vehicles. The specific operation of the dual-domain fusion module in the dual-domain hybrid encoder module is as follows: for a given feature map The module first performs convolution operation, and then performs channel segmentation: ； In the above formula, denotes a convolutional layer that adjusts the channel dimension while keeping the spatial dimension unchanged, and , is set to one quarter of the total number of channels; the first branch is convolved again, and then passes through a Gaussian error linear unit to introduce nonlinearity: ； To capture complex feature dependencies, the convolved features In the frequency domain, the following is done: ； In the above formula, and are learnable parameters used to balance the contributions from the frequency domain enhancement features and the spatial residual connections, respectively; the outputs of the two branches are concatenated together and passed through a final convolutional layer to integrate the refined features: 。 3. The method of claim 1, wherein the method is based on an improved DETR for small target detection of images of unmanned aerial vehicles. The extended IoU in the enhanced query selection mechanism is defined as: for the model's predicted anchor box and real anchor frame The extended IoU is the scaled predicted anchor box. And the scaled-down real anchor frame The crossover ratio between the two anchor frames is proportional to the scaling factor. The anchor frames are then scaled while keeping their centers unchanged. The scaled anchor frames are as follows: ； ； where and are the center coordinates, respectively represent the width and height of the predicted and ground truth boxes, and the extended IoU is defined as: 。 4. The method of claim 1, wherein the method is based on an improved DETR for small target detection of images of unmanned aerial vehicles. The binary cross-entropy loss calculation formula in the knowledge distillation strategy is: ； In the above formula, and respectively represent the classification scores of the student model and the teacher model final decoder layer outputs; the L1 loss calculation formula is: ； wherein, and respectively represent the anchor box coordinates predicted by the student model and the teacher model, represents the target confidence score of the teacher model; the intersection over union loss calculation formula is: ； The calculation formula of the extended SIoU is: ; The calculation formula of the distillation loss is: ； In the above formula, , and are constant factors that balance the contributions of the individual loss component terms.

5. The method of claim 1, wherein the method is based on an improved DETR for small target detection of images of unmanned aerial vehicles. In the model inference stage, a multi-thread or asynchronous data acquisition strategy is adopted, combined with software and hardware co-optimization, which effectively reduces the detection delay and significantly improves the real-time performance of the system, meeting the demand for fast detection in the unmanned aerial vehicle application scenario.

6. The method of claim 1, wherein the method is based on an improved DETR for small target detection of images of unmanned aerial vehicles. By adopting compression technologies such as model pruning and quantization, the model is optimized to reduce the parameter quantity and computational complexity of the model, overcoming the limitations of the unmanned aerial vehicle system in terms of flight time and computing resources, so that the method can efficiently run on the unmanned aerial vehicle device with limited resources.

7. The method of claim 1, wherein, The method can be applied to various unmanned aerial vehicle use scenarios, including security monitoring, inspection and agricultural plant protection, etc., and can accurately detect different types of small targets such as vehicles, personnel, animals and plants, etc., showing the wide applicability and good detection performance of the method in practical applications.

8. An improved DE TR-based small object detection system for UAV images based on the method of any one of claims 1-7. Comprise: S100: a detection input module for inputting an image to be detected; S200: a backbone network for preliminary feature extraction; S300: a dual-domain hybrid encoder for feature extraction and fusion; S400: an enhanced query selection module for query resource allocation; S500: a decoder for predicting the target category and anchor box; S600: a knowledge distillation module for model optimization; S700: a detection output module for outputting the detection result.

9. A computer readable storage medium storing a computer program, characterized in that: The computer program is executed by the processor to realize the small target detection method for unmanned aerial vehicle images based on the improved DETR as claimed in any one of claims 1 to 4, providing a software storage and execution carrier for the implementation of the method.

Images