Unmanned aerial vehicle target real-time detection method based on MDLT-YOLO

Abstract

The invention discloses an unmanned aerial vehicle target real-time detection method based on MDLT-YOLO, and belongs to the technical field of anti-unmanned aerial vehicles. Firstly, a plurality of unmanned aerial vehicle infrared images are collected and marked, and a data set is obtained; then, a C2f module of the YOLOv8 backbone network is replaced with a multi-expansion-rate residual error module to improve the YOLOv8 network, and an MDLT-YOLO model is obtained; an input feature map x1 of the multi-expansion-rate residual error module is subjected to 3 * 3 convolution and then is subjected to three expansion convolution with different expansion rates to obtain feature maps x2, x3 and x4; splicing the feature maps x2, x3 and x4 on the channel dimension, and performing normalization and convolution with the size of 1 * 1 to obtain a feature map x5; the feature map x5 and the input feature map x1 are connected through a residual error to obtain an output feature map of the multi-expansion-rate residual error module; and finally, training and optimizing the MDLT-YOLO model, and using the optimized MDLT-YOLO model as a detection model for unmanned aerial vehicle target detection. The method effectively solves the problems of complex background, low contrast, multi-scale target, real-time detection and the like in a small target detection task of the unmanned aerial vehicle.

Description

Technical Field

[0001] The present invention belongs to the field of anti-UAV technology, and in particular relates to a real-time detection method for UAV targets based on MDLT (Infrared Lightweighting)-YOLO. Background Art

[0002] The continuous development of artificial intelligence (AI) technology has led to the widespread use of drones in both military and civilian fields. However, this widespread use also presents new challenges and threats. For example, the misuse of drones can lead to collisions during low-altitude flights, which can in turn cause mid-air accidents. Consequently, anti-drone technologies have emerged to remotely monitor drones, provide early warning, and intercept them.

[0003] The primary task of anti-drone technology is to detect drones. Existing detection methods mostly use visible light and infrared images. Visible light imaging is heavily dependent on weather conditions and suffers from shortcomings such as poor penetration and difficulty distinguishing at night, severely impacting recognition efficiency. Infrared thermal imaging, on the other hand, is independent of weather conditions and can achieve all-weather detection. Traditional infrared target detection algorithms are primarily based on filtering and data structures. While filter-based detection methods are computationally simple and time-efficient, they can only suppress uniform backgrounds to a certain extent and cannot effectively address target detection in complex backgrounds. The models also suffer from poor detection performance and insufficient robustness. Data structure-based detection methods distinguish between targets and backgrounds by leveraging different structural features, such as the low rank of the background and the sparsity of the target. While this improves detection accuracy, the detection speed is relatively slow, making it unsuitable for real-time detection.

[0004] Deep learning-based methods are widely used in various target detection tasks. However, since drones occupy a relatively small proportion of the total image pixels, they are considered small targets, leading to high missed detection rates. Furthermore, infrared imaging results in high background noise and low contrast, making it difficult for the model to extract target information, further increasing the detection challenge. To address the challenges of drone target detection, such as high background noise, weak target features, high missed detection rates, and poor real-time performance, this paper proposes a real-time drone target detection method based on MDLT-YOLO. Summary of the Invention

[0005] In view of the shortcomings of the existing technology, the technical problem to be solved by the present invention is to provide a real-time detection method for UAV targets based on MDLT-YOLO.

[0006] The present invention solves the technical problem by adopting the following technical solutions:

[0007] A real-time detection method for unmanned aerial vehicle targets based on MDLT-YOLO is characterized by comprising the following steps:

[0008] Step 1: Collect several drone infrared images and annotate them to obtain training and test sets;

[0009] Step 2: Improve the YOLOv8 network by replacing the C2f module of the YOLOv8 backbone network with a multi-dilution rate residual module to obtain the MDLT-YOLO model.

[0010] The input feature map x1 of the multi-expansion rate residual module undergoes a 3×3 convolution, and then undergoes three dilated convolutions with different dilation rates to obtain feature maps x2, x3, and x4. Feature maps x2, x3, and x4 are concatenated in the channel dimension, normalized, and convolved with a size of 1×1 to obtain feature map x5. Feature map x5 is residually connected with the input feature map x1 to obtain the output feature map of the multi-expansion rate residual module.

[0011] Step 3: Use the training set to train the MDLT-YOLO model, and use the test set to optimize the MDLT-YOLO model parameters. The optimized MDLT-YOLO model is used as the detection model for drone target detection.

[0012] NWD is introduced to measure the similarity between the predicted box and the real box. The calculation formula of NWD is:

[0013]

[0014] Where, Represents the second-order Wasserstein distance between the predicted box P and the real box G, (cx p ,cy p ) is the center point of the prediction box, w p 、h p is the width and height of the prediction box, (cx g ,cy g ) is the center point of the ground truth box, w g 、h g is the width and height of the real frame;

[0015] Normalize formula (1) to get the normalized Wasserstein distance:

[0016]

[0017] The loss function based on NWD is:

[0018] Loss = 1-N WD (P,G) (3)

[0019] Where C is a constant.

[0020] Furthermore, when the last three C2f modules of the YOLOv8 backbone network are replaced by three multi-dilation rate residual convolutions, the MDLT-YOLO model achieves the best balance between detection accuracy, computational complexity and real-time detection.

[0021] Furthermore, the detection head of the MDLT-YOLO model shares some convolutions, that is, the input feature map of the detection head undergoes two convolutions in sequence to obtain the feature map u; the size, stride and filling rate of the first convolution are 1×1, 1 and 0 respectively, and the size, stride and filling rate of the second convolution are 3×3, 1 and 1 respectively; the feature map u is subjected to a two-dimensional convolution with a size, stride and filling rate of 1×1, 1, 1 respectively to realize bounding box regression, and the feature map u is subjected to a convolution with a size, stride and filling rate of 3×3, 1 and 1 respectively to realize classification regression.

[0022] Furthermore, the drone includes multiple types including quad-rotor, hexacopter and fixed-wing.

[0023] Compared with the prior art, the present invention has the following beneficial effects:

[0024] For the task of detecting small targets in drones, the MDRB module is proposed. The receptive field of the dilated convolution is larger, which fully captures the subtle features and contextual information of the target, can better distinguish foreground and background information, improve the detection accuracy of the model, and reduce the missed detection rate. The MDRB module improves the feature expression ability, robustness and stability of the model through residual connection. Compared with the C2f module of YOLOv8, the MDRB module has fewer parameters. At the same time, the detection head uses the weights of some shared convolution layers and uses smaller convolution kernels to further reduce the number of parameters and computational complexity of the model, achieving the purpose of real-time detection through efficient calculation. Smaller convolution kernels can also make the model more focused on capturing the detailed information of local targets. The normalized weighted distance (NWD) is introduced to replace the traditional IoU as the loss function, optimize the position of the bounding box, reduce the model's sensitivity to the position of small targets, further reduce the missed detection rate, and improve the detection performance of the model. In summary, the present invention effectively solves the problems of complex background, low contrast, multi-scale targets and real-time detection in the task of detecting small targets in drones. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 Schematic diagram of the structure of the MDLT-YOLO model;

[0026] Figure 2 It is a structural diagram of the MDRB module;

[0027] Figure 3 Schematic diagram of the structure of the detection head with shared convolutional layer;

[0028] Figure 4 The following is a visual comparison of the detection results in different scenarios. DETAILED DESCRIPTION

[0029] The technical solution of the present invention is described in detail below with reference to the accompanying drawings and specific embodiments, but this does not limit the scope of protection of the present application.

[0030] The present invention provides a real-time detection method for drone targets based on MDLT-YOLO (referred to as method, see Figures 1 to 4 ), including the following steps:

[0031] The first step is to collect several drone infrared images and annotate them to obtain a data set;

[0032] In this example, a multispectral detector was used to capture multiple videos of drones, including quadrotors, hexacoptors, and fixed-wing drones, from which a total of 600 drone infrared images were extracted. The position and size information of the drones in the images were annotated using Lableimg software, and all the annotated images were then divided into training, test, and validation sets in a ratio of 6:2:2.

[0033] Step 2: Build the MDLT-YOLO model;

[0034] like Figure 1 As shown in the figure, YOLOv8 is selected as the base network and improved using the Multi-Dilation Residual Block (MDRB). This involves replacing the C2f module in the YOLOv8 backbone network with the MDRB module. Meanwhile, the YOLOv8 detection head shares some convolutions, resulting in the MDLT-YOLO model. The MDRB module focuses on extracting features of small objects, and through residual connections, it improves the model's feature representation and stability, thereby enhancing its detection capabilities for small objects. Sharing some convolutions enables a lightweight model design and reduces redundant feature computation.

[0035] like Figure 2 As shown in the figure, in the MDRB module, the input feature map x1 undergoes a 3×3 convolution, followed by dilated convolutions with dilation rates of 1, 3, and 5, respectively, to obtain feature maps x2, x3, and x4. Feature maps x2, x3, and x4 are concatenated along the channel dimension, normalized, and convolved with a 1×1 convolution to obtain feature map x5. Feature map x5 is residually connected to the input feature map x1 to obtain the output feature map of the MDRB module. When the MDRB modules are used to replace the last three C2f modules of the YOLOv8 backbone network, the model achieves optimal detection performance.

[0036] like Figure 3 As shown in the figure, the input feature map of the detection head undergoes two convolutions in sequence to obtain the feature map u; the size, stride and filling rate of the first convolution are 1×1, 1 and 0 respectively, and the size, stride and filling rate of the second convolution are 3×3, 1 and 1 respectively; the feature map u undergoes a two-dimensional convolution with a size, stride and filling rate of 1×1, 1, 1 respectively to achieve bounding box regression; the feature map u undergoes a convolution with a size, stride and filling rate of 3×3, 1 and 1 respectively to achieve classification regression.

[0037] Step 3: Train and optimize the MDLT-YOLO model, and use the optimized MDLT-YOLO model as the detection model for drone target detection;

[0038] The MDLT-YOLO model is trained using the training set, and the Normalized Weighted Distance (NWD) is introduced to measure the similarity between the predicted box and the true box to replace the traditional IoU metric. First, the predicted box and the true box are modeled as Gaussian distributions respectively, and then the Wasserstein distance is used to measure the similarity between the two Gaussian distributions. The calculation of NWD is shown in formula (1):

[0039]

[0040] Where, Represents the second-order Wasserstein distance between the predicted box P and the real box G, (cx p ,cy p ) is the center position of the prediction box, w p 、h p is the width and height of the prediction box, (cx g ,cy g ) is the center position of the real frame, w g 、h g is the width and height of the real frame;

[0041] Formula (1) is the distance measure between the predicted box and the real box. It cannot directly measure the similarity, so it is normalized to obtain the normalized Wasserstein distance:

[0042]

[0043] Where C is a constant, which is generally the average absolute size of the objects in the dataset;

[0044] The loss function based on NWD is:

[0045] Loss = 1-N WD (P,G) (3)

[0046] When the loss function converges, the trained MDLT-YOLO model is obtained; the trained MDLT_YOLO model is tested using the test set, and the model parameters are optimized using the gradient descent method to ensure the detection accuracy and robustness of the model in different scenarios. The optimized MDLT-YOLO model is obtained and used as the detection model for drone target detection.

[0047] Example

[0048] This example is based on a server platform running Ubuntu 22.04, equipped with an Intel(R) Core(TM) i9-9820X CPU @ 3.3GHz and two NVIDIA GeForce RTX 2080Ti GPUs. The MDLT-YOLO model was built using the Python 3.9 programming language and the PyTorch 1.9.1 framework. The model was trained with 300 training rounds, a batch size of 16, and an image size of 640×640 pixels. The Stochastic Gradient Descent (SGD) optimizer was used, with an initial learning rate of 0.01, a momentum of 0.937, and a decay coefficient of 0.0005.

[0049] In order to verify the effectiveness of the MDRB module, an experiment was designed to compare it with the original C2f module of YOLOv8. The results of the comparative experiment are shown in Table 1. Among them, YOLOv8-MDRB1 represents the use of one MDRB module to replace the last C2f module of the YOLOv8 backbone network, YOLOv8-MDRB2 represents the use of two MDRB modules to replace the last two C2f modules of the YOLOv8 backbone network, YOLOv8-MDRB3 represents the use of three MDRB modules to replace the last three C2f modules of the YOLOv8 backbone network, and YOLOv8-MDRB4 represents the use of four MDRB modules to replace the four C2f modules of the YOLOv8 backbone network.

[0050] Table 1 Comparative experimental results of MDRB module

[0051]

[0052] The experimental results in Table 1 show that as the number of MDRB modules increases from 1 to 3, detection accuracy (mAP@0.5) gradually improves. In particular, when the number of MDRB modules reaches 3, the highest detection accuracy reaches 87.4%, significantly outperforming the original YOLOv8 (85.3%). The computational complexity decreases slightly (from 8.7 GB to 8.6 GB), and the number of parameters decreases slightly (from 3,085,824 to 3,151,904 in the original YOLOv8). This indicates that the MDRB module provides higher detection accuracy while reducing computational complexity. As the number of MDRB modules increases, the FPS (frames per second) also improves. In particular, YOLOv8-MDRB3 achieves an FPS of 141.6 while maintaining high accuracy. This demonstrates that the model is not only more accurate but also significantly improves real-time processing, achieving an optimal balance between detection accuracy, computational complexity, and real-time detection performance. The MDRB module uses convolutions with multiple dilation rates to better capture features of different scales and details. This multi-scale feature extraction capability helps improve the model's detection accuracy in complex backgrounds and for small objects. The MDRB module provides stronger feature extraction capabilities, enabling the design of replacing three modules to fully utilize the advantages of MDRB while maintaining high real-time detection performance.

[0053] In order to verify the detection performance of the MDLT-YOLO model, the MDLT-YOLO model is compared with common target detection models. The comparative experimental results are shown in Table 2.

[0054] Table 2 Comparative experimental results of different models

[0055]

[0056] As shown in Table 2, two-stage object detection algorithms, such as Faster-RCNN, suffer from slow inference speed and low detection accuracy, making them ineffective for drone target detection and difficult to apply in anti-drone systems. While SSD improves inference speed compared to Faster-RCNN, its predefined anchor box approach performs relatively poorly for small object detection. While the MDLT-YOLO model's FPS is slightly lower than YOLOv5, the detection accuracy of YOLOv5 and YOLOv7 falls far short of the MDLT-YOLO model, resulting in relatively high missed detection rates and failing to meet detection requirements. The detection performance of the MDLT-YOLO model has been further improved. While keeping the accuracy basically unchanged, the recall rate has been increased from 77.3% of YOLOv8-n to 84.7%, significantly improving the problem of frequently missed detection of small targets. The mAP@0.5 has increased from 85.3% of YOLOv8-n to 89.7%, an increase of 4.4%. The model's inference speed has increased from 133.1FPS to 144.7FPS, an increase of 11.6FPS, and the model's computational complexity has decreased by 1.5G, fully demonstrating the superiority of the MDLT-YOLO model in drone target detection tasks. It has achieved a dual improvement in computing accuracy and speed, while maintaining efficient performance and better meeting the needs of real-time detection. The MDLT-YOLO model can identify more targets in less time and effectively reduce missed detections, which is of great significance for drone target detection in complex scenarios.

[0057] Figure 4 It is a visualization of the detection results in different scenes. Each row in the image represents a scene, including the infrared images of drones under different occlusion backgrounds. Figure 4 It can be seen that the MDLT_YOLO model can accurately detect targets in scenarios such as the target is integrated with the surrounding environment, the target is at the edge of the image, the edge is unclear due to target motion, and the information difference of small targets is not obvious under cloud background. Therefore, the present invention can be used for UAV target detection in complex backgrounds, effectively reducing the missed detection rate, and has stronger robustness and higher detection accuracy.

[0058] Any matters not described in the present invention are applicable to the prior art.

Claims

1. A real-time detection method for UAV targets based on MDLT-YOLO, characterized in that: The steps include: Step 1: Collect several drone infrared images and annotate them to obtain training and test sets; Step 2: Improve the YOLOv8 network by replacing the C2f module of the YOLOv8 backbone network with a multi-dilution rate residual module to obtain the MDLT-YOLO model. The input feature map x1 of the multi-expansion rate residual module undergoes a 3×3 convolution, and then undergoes three dilated convolutions with different dilation rates to obtain feature maps x2, x3, and x4. Feature maps x2, x3, and x4 are concatenated in the channel dimension, normalized, and convolved with a size of 1×1 to obtain feature map x5. Feature map x5 is residually connected with the input feature map x1 to obtain the output feature map of the multi-expansion rate residual module. Step 3: Use the training set to train the MDLT-YOLO model, and use the test set to optimize the MDLT-YOLO model parameters. The optimized MDLT-YOLO model is used as the detection model for drone target detection. NWD is introduced to measure the similarity between the predicted box and the real box. The calculation formula of NWD is: Where, Represents the second-order Wasserstein distance between the predicted box P and the real box G, (cx p ,cy p ) is the center point of the prediction box, w p 、h p is the width and height of the prediction box, (cx g ,cy g ) is the center point of the ground truth box, w g 、h g is the width and height of the real frame; Normalize formula (1) to get the normalized Wasserstein distance: The loss function based on NWD is: Loss=1-N WD (P,G) (3) Where C is a constant.

2. The real-time detection method for drone targets based on MDLT-YOLO according to claim 1 is characterized in that: When the last three C2f modules of the YOLOv8 backbone network are replaced by three multi-dilation rate residual convolutions, the MDLT-YOLO model achieves the best balance between detection accuracy, computational complexity, and real-time detection.

3. The real-time detection method for unmanned aerial vehicle targets based on MDLT-YOLO according to claim 1 or 2, characterized in that: The detection head of the MDLT-YOLO model shares some convolutions, that is, the input feature map of the detection head is sequentially subjected to two convolutions to obtain the feature map u; the size, stride, and filling rate of the first convolution are 1×1, 1, and 0, respectively, and the size, stride, and filling rate of the second convolution are 3×3, 1, and 1, respectively; the feature map u is subjected to a two-dimensional convolution with a size, stride, and filling rate of 1×1, 1, and 1 to achieve bounding box regression, and the feature map u is subjected to a convolution with a size, stride, and filling rate of 3×3, 1, and 1 to achieve classification regression.

4. The real-time detection method for drone targets based on MDLT-YOLO according to claim 1, characterized in that: The drones include four-rotor, six-rotor and fixed-wing types.

Images