A method and apparatus for small target detection of unmanned aerial vehicles based on YOLOv12 and infrared radiation characteristics.

Abstract

This application provides a method and apparatus for small target detection on unmanned aerial vehicles (UAVs) based on YOLOv12 and infrared radiation features, relating to the field of target detection technology. The UAV small target detection method includes: acquiring infrared and visible light images and inverting them to obtain infrared radiation features; fusing the two-light images and inputting them, along with the infrared radiation features, into a two-stream network of a target detection model for processing. The image stream is based on the YOLOv12 architecture; the radiation feature stream uses an optimized Bottleneck structure to reduce computational complexity. By fusing the two feature streams, multimodal fusion features are generated, and the final detection result is output. The method in this application effectively solves the problem of weak and easily missed detection features of small UAV targets in complex backgrounds, significantly improving detection accuracy and robustness.

Description

Technical Field

[0001] This application relates to the field of target detection technology, and more specifically, to a method and apparatus for detecting small targets on unmanned aerial vehicles based on YOLOv12 and infrared radiation characteristics. Background Technology

[0002] With the rapid development of the low-altitude economy in recent years, while small drones have brought convenience to various industries, unauthorized flight activities have also posed many hidden dangers to low-altitude safety. Therefore, rapid and accurate detection of small drones is particularly important. Infrared imagery, due to its advantages under different lighting conditions, is widely used in target detection, especially in complex environments such as low light and nighttime. Currently, using ground-based infrared detectors to detect small aerial drone targets is the most common detection method. However, using ground-based detectors faces many challenges, including low contrast of small targets, interference from complex backgrounds, and blurring caused by rapid movement. These factors make infrared image detection of drones very complex, requiring the combination of multi-dimensional detection information and advanced image processing technology to improve detection accuracy and efficiency. Visible light images have higher resolution than infrared images, and the technology is more mature and contains richer details. Fusing infrared and visible light images can enhance image visibility, obtain multi-modal features of targets, and further improve the accuracy of target detection.

[0003] With the rapid development of deep learning technology, convolutional neural networks (CNNs) have become one of the important technologies in image processing and object detection. In recent years, models based on the YOLO (You Only Look Once) series have achieved remarkable results in object detection tasks, especially in terms of real-time performance and accuracy. YOLOv12, as an advanced model in the YOLO series, has powerful real-time object detection capabilities. However, it mainly relies on image pixel information for feature extraction and cannot achieve fast and accurate detection of small targets such as drones at long distances.

[0004] Therefore, there is an urgent need for a method for detecting small targets on UAVs based on YOLOv12 and infrared radiation characteristics to solve one of the aforementioned technical problems. Summary of the Invention

[0005] The purpose of this application is to provide a method and apparatus for detecting small targets on unmanned aerial vehicles (UAVs) based on YOLOv12 and infrared radiation characteristics, which can solve at least one of the aforementioned technical problems. The specific solution is as follows:

[0006] According to a specific embodiment of this application, this application provides a method for detecting small targets on a UAV based on YOLOv12 and infrared radiation characteristics, including:

[0007] Acquire infrared and visible light images of the area to be detected, and invert the corresponding infrared radiation features;

[0008] After preprocessing the infrared image and the visible light image, a fused image is obtained by fusing them.

[0009] The fused image is input into the image stream network of the target detection model to extract visual features; simultaneously...

[0010] The infrared radiation features are input into the radiation feature flow network of the target detection model to extract the radiation features;

[0011] By fusing the visual features and the radiation features, multimodal fusion features are obtained;

[0012] Based on the multimodal fusion features, the detection results of small targets on the UAV are output;

[0013] in,

[0014] The image streaming network is based on the YOLOv12 architecture;

[0015] The radiation characteristic flow network is based on a simplified network structure optimized from the Botteleneck module.

[0016] Furthermore, the image streaming network is based on the YOLOv12 architecture and includes:

[0017] Introduce a residual shortcut connection from input to output into the backbone of the YOLOv12 neural network;

[0018] The original convolutional modules are replaced with an adjustable number of C3k modules, which construct a depth visual transformer based on the input features.

[0019] Furthermore, the simplified network structure based on the optimized Bottleneck module includes:

[0020] A simplified network structure consisting of multiple optimized Bottleneck modules is used; by separating the computation of channel and spatial dimensions, the computational complexity of feature extraction from radiative feature sequences is reduced.

[0021] Furthermore, the target detection model is trained using a multi-task learning mechanism.

[0022] Furthermore, the multi-task learning mechanism includes:

[0023] The target detection model is configured to simultaneously output the detection results of small targets from the UAV and the predicted radiation intensity of the target area;

[0024] The overall loss function of the target detection model consists of the target detection loss and the radiation feature regression loss.

[0025] The target detection loss is calculated based on the detection result and the true label, and the radiation feature regression loss is calculated based on the radiation intensity prediction result and the true radiation intensity label.

[0026] Furthermore, the training process of the target detection model is as follows:

[0027] Use the Adam optimizer to optimize parameters;

[0028] After training, precision, recall, and mean precision were used as performance evaluation metrics for the object detection model.

[0029] Furthermore, the step of outputting the detection result of small targets of the UAV based on the multimodal fusion features includes:

[0030] The multimodal fusion features are input into the neck network of the YOLOv12 architecture;

[0031] The multimodal fusion features are fused and enhanced using the A2C2f module in the neck network.

[0032] The enhanced features are input into the detection head, and the detection results of the small targets of the UAV are output.

[0033] Further, the preprocessing of the infrared image and the visible light image includes:

[0034] The infrared image is subjected to denoising processing based on nonlocal means;

[0035] The visible light image is subjected to super-resolution reconstruction based on bicubic interpolation.

[0036] Furthermore, the inversion yields the corresponding infrared radiation features, including:

[0037] The infrared camera was calibrated to establish a mapping relationship between the target temperature and the image grayscale value;

[0038] By using the system response coefficient, atmospheric transmittance, and target parameters, combined with the original infrared image, the corresponding infrared radiation characteristics are obtained through infrared radiation inversion calculation.

[0039] According to a specific embodiment of this application, this application also provides a small target detection device for unmanned aerial vehicles based on YOLOv12 and infrared radiation characteristics, comprising:

[0040] The data acquisition module is used to acquire infrared and visible light images of the area to be detected.

[0041] The radiation feature inversion module is used to calculate infrared radiation features based on the infrared image.

[0042] An image fusion module is used to fuse the infrared image and the visible light image into a fused image.

[0043] The dual-stream feature extraction module is used to input the fused image into an image stream network based on the YOLOv12 architecture to extract visual features, and simultaneously input the infrared radiation features into a radiation feature stream network based on an optimized Bottleneck module to extract radiation features.

[0044] The feature fusion and detection module is used to fuse the visual features and the radiation features to obtain multimodal fusion features, and output the detection results of small targets of the UAV based on these features.

[0045] Compared with the prior art, the above-described solutions of this application have at least the following beneficial effects:

[0046] 1. This application discloses a method and apparatus for detecting small targets on UAVs based on YOLOv12 and infrared radiation features. By constructing a dual-stream processing architecture that includes a YOLOv12 image stream and an optimized Bottleneck radiation feature stream, visual features and physical radiation features are deeply fused. This solves the technical problem that a single visual model is insufficient in extracting small target features in complex backgrounds and is susceptible to environmental interference, leading to missed detections and false detections. It achieves high-precision and robust detection of small targets on UAVs under harsh conditions such as low light and haze.

[0047] 2. This application discloses a method and apparatus for detecting small targets on unmanned aerial vehicles (UAVs) based on YOLOv12 and infrared radiation features. By introducing infrared radiation features to construct a multimodal dataset, and using a YOLOv12 network with residual shortcut connections and an adjustable C3k module to process fused images, the detection and radiation regression losses are optimized simultaneously by combining a multi-task learning mechanism. This solves the problems of insufficient feature information and weak model generalization ability in the detection of extremely small targets by traditional methods, and significantly improves the detection recall and accuracy.

[0048] 3. This application discloses a method and apparatus for detecting small targets on UAVs based on YOLOv12 and infrared radiation features. By designing a dedicated dual-stream network structure, the image stream is based on an improved YOLOv12 architecture to ensure feature extraction capability, while the radiation feature stream adopts a simplified Bottleneck structure to control computational complexity. This solves the problem of large model computation and difficulty in real-time deployment in small target detection, and achieves real-time and efficient detection of small targets on UAVs while maintaining high accuracy. Attached Figure Description

[0049] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort. In the drawings:

[0050] Figure 1 This is a flowchart illustrating a method for detecting small targets on a drone based on YOLOv12 and infrared radiation characteristics, according to an embodiment of this application.

[0051] Figure 2 This is a schematic diagram of the structure of an image stream network for extracting visual features, as shown in an embodiment of this application.

[0052] Figure 3 This is a schematic diagram of the structure of the radiation feature flow network for extracting radiation features, as shown in an embodiment of this application.

[0053] Figure 4 This is a schematic diagram of the structure of a target detection model based on the YOLOv12 architecture, as shown in an embodiment of this application.

[0054] Figure 5 The diagram shows the structure of the C3k module, the Bottleneck module, and the A2C2f module as illustrated in the embodiments of this application. Detailed Implementation

[0055] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0056] The terminology used in the embodiments of this application is for the purpose of describing particular embodiments only and is not intended to limit the application. The singular forms “a,” “said,” and “the” used in the embodiments of this application and the appended claims are also intended to include the plural forms, and “multiple” generally includes at least two unless the context clearly indicates otherwise.

[0057] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0058] It should be understood that although the terms first, second, third, etc., may be used in the embodiments of this application, these descriptions should not be limited to these terms. These terms are only used to distinguish the descriptions. For example, first may also be referred to as second without departing from the scope of the embodiments of this application, and similarly, second may also be referred to as first.

[0059] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that an article or device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such an article or device. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the article or device that includes said element.

[0060] The optional embodiments of this application are described in detail below with reference to the accompanying drawings.

[0061] like Figure 1 As shown, a method for detecting small targets on a UAV based on YOLOv12 and infrared radiation features includes:

[0062] S1. Acquire infrared and visible light images of the area to be detected, and invert the corresponding infrared radiation features.

[0063] Data was collected from various types of small unmanned aerial vehicles (UAVs) using both infrared and visible light dual-light detectors. During the acquisition process, image data was obtained under different ambient lighting conditions, capturing dual-light images of UAVs at different distances and angles. Infrared images were acquired using existing infrared sensors, providing clear infrared information, while visible light images provided high-resolution visual features of conventional targets.

[0064] This application provides a preferred implementation method for inverting the corresponding infrared radiation characteristics, including: calibrating the infrared camera and establishing a mapping relationship between the target temperature and the image grayscale value; using the system response coefficient, atmospheric transmittance, and target parameters, combined with the original infrared image, to obtain the corresponding infrared radiation characteristics through infrared radiation inversion calculation.

[0065] The infrared camera is calibrated to obtain the relationship between target temperature and grayscale values, preparing for the next step of infrared radiometric inversion calculation. Secondly, calibration provides essential information for infrared radiometric inversion, including system response coefficients such as responsivity and bias at different integration times; atmospheric transmittance calculated using atmospheric software based on parameters such as temperature, humidity, air pressure, and visibility; and target parameters such as absolute time, azimuth, elevation, altitude, target distance, band, integration time, and focal length. The original image and necessary information are then input into the infrared characteristic inversion software, and based on Planck's radiation law, the infrared radiometric characteristics representing the radiation intensity of each pixel are calculated.

[0066] To calculate the infrared radiation characteristics representing the radiation intensity of each pixel, the radiation intensity needs to be calculated. The calculation process is as follows:

[0067] First, based on the relationship between the total radiant energy and total radiant power of blackbody radiation and its temperature, the expression is:

[0068]

[0069] in, Represents the radiant energy per unit area; denoted by Stefan-Boltzmann constant; T represents the absolute temperature of the object.

[0070] Next, calculate the spectral distribution of the radiation emitted by the object at a specific temperature, expressed as:

[0071]

[0072] in, Indicates wavelength as The radiation intensity; h represents Planck's constant; c represents the speed of light; K represents Boltzmann's constant; T represents absolute temperature; λ represents wavelength.

[0073] Finally, the infrared radiation characteristics are expressed as follows:

[0074]

[0075] Where P represents the radiation intensity of an object, i.e., the infrared radiation characteristic.

[0076] S2. After preprocessing the infrared and visible light images, a fused image is obtained by fusing them.

[0077] This application provides a preferred embodiment that preprocesses infrared and visible light images. The infrared image undergoes non-local mean denoising, using a neighborhood similarity weighting method to remove noise. The visible light image undergoes bicubic interpolation-based super-resolution reconstruction to restore edge texture. This preprocessing of the infrared and visible light images results in clearer image data.

[0078] The technical solution of this application embodiment adopts a denoising method based on non-local means to denoise infrared images. The principle of this method is to perform a weighted average of each pixel in the infrared image by means of its similarity to other pixels in the image, thereby achieving noise removal.

[0079] The expression for nonlocal means denoising is:

[0080]

[0081] in, This represents the pixel value after non-local mean denoising. Represents the pixel value of any point p; Represents the neighborhood of an image; Parameters that control the degree of similarity; This represents the pixel values ​​in the image that have not been denoised.

[0082] The technical solution of this application's embodiments, the basic idea of ​​visible light image super-resolution reconstruction, is to recover the details of a high-resolution image from a low-resolution image. In super-resolution reconstruction, given a low-resolution image (LR), it is converted into a higher-resolution image (HR) using a bicubic interpolation method. Specifically, bicubic interpolation uses a weighted average of the surrounding 16 pixels to estimate the new pixel value, effectively preserving the edge details and texture information of the image.

[0083] For any target pixel position (x, y), interpolation can be performed as follows: The expression for bicubic interpolation is:

[0084]

[0085] in, This represents the pixel value after bicubic interpolation; This represents the pixel value at coordinates (i,j) in a low-resolution image. and These represent the weighting weights of the bicubic interpolation function in the x-axis and y-axis directions, respectively.

[0086] The expression for the weighting function W(t) of bicubic interpolation is:

[0087]

[0088] Where t represents the offset from the target pixel.

[0089] In this embodiment of the application, firstly, a low-resolution image (LR) of size W is input. LR ×H LR Secondly, select the output size and determine the size of the target high-resolution image (HR), which is typically N times larger than the original image, with a size of W. HR =N×W LR and H HR =N×H LR Next, bicubic interpolation reconstruction is performed, estimating each pixel using bicubic interpolation to output a high-resolution image. Finally, the final high-resolution bicubic interpolation reconstruction result image is output, which is a super-resolution reconstruction process based on bicubic interpolation for visible light images.

[0090] The technical solution of this application embodiment, based on bicubic interpolation super-resolution reconstruction processing, can better preserve image details and reduce blurring. Furthermore, it has a fast computation speed, making it suitable for real-time processing.

[0091] Each preprocessed infrared image is fused with its corresponding visible light image to create a fused image, which is then used to create a multimodal dataset. Each fused image is associated with and labeled with extracted infrared radiation features. The final dataset contains information from both the fused image data and the infrared radiation feature data.

[0092] The technical solution of this application embodiment uses a weighted pixel-level fusion method to fuse visible light and infrared images. The fused image is obtained by weighted averaging of each pixel in the preprocessed visible light and infrared images. Other methods in the prior art can also be used for fusion, depending on the actual situation; this application does not limit this approach.

[0093] In this embodiment of the application, the expression for the weighted pixel-level fusion method is:

[0094]

[0095] in, This represents the pixel values ​​of the merged image; Represents the pixel values ​​of a visible light image; Represents the pixel values ​​of an infrared image; This represents the weighting coefficient, adjusted between 0 and 1. By adjusting α, the contribution ratio of the visible light image and the infrared image in the fused image can be controlled. When... As the value increases, the weight of the visible light image increases, and the contours and color details of the fused image become more apparent; when When the value of decreases, the weight of the infrared image increases, and the fusion result highlights temperature differences more clearly. This can be adjusted according to the actual situation. The numerical values ​​may be adjusted, but this application embodiment does not limit this.

[0096] S3, such as Figure 2 , Figure 3 and Figure 5 As shown, the fused image is input into the image flow network of the target detection model to extract visual features; at the same time, the infrared radiation features are input into the radiation feature flow network of the target detection model to extract radiation features.

[0097] In this embodiment, YOLOv12-based architecture is used to extract fused image features. A residual shortcut connection from input to output is introduced into the backbone network of the YOLOv12 neural network. An adjustable number of C3k modules are used to replace the original convolutional modules. The C3k modules construct a depth visual transformer based on the input features.

[0098] The simplified network structure based on the optimized Bottleneck module includes: a simplified network structure composed of multiple optimized Bottleneck modules; the optimized Bottleneck module sequentially includes: a channel compression layer, a spatial feature extraction layer, and a channel recovery layer. By separating the computation of channel and spatial dimensions, the computational complexity in the feature extraction process of the radiometric feature sequence is significantly reduced.

[0099] The technical solution of this application embodiment uses the YOLOv12 architecture to extract and fuse image features and radiation feature streams. The network is based on a simplified network structure optimized from the Bottleneck module. The extraction structures of the two are combined to form a dual-stream detection model, namely image stream and radiation feature stream, which fuses the multimodal features of visual features and radiation features.

[0100] This application provides a preferred implementation method for outputting detection results of small targets on a UAV based on multimodal fusion features, including: inputting the multimodal fusion features into the neck network of the YOLOv12 architecture; performing multi-scale feature fusion and enhancement on the multimodal fusion features through the neck network; inputting the enhanced features into the detection head, and outputting the detection results of small targets on the UAV.

[0101] To improve detection performance, an image flow network of the object detection model is used to extract features from the fused image, and a radiation feature flow network of the object detection model is used to process the radiation features, fusing visual and radiation features to obtain multimodal fusion features. The image flow network is responsible for processing visual information from the fused image. Based on the YOLOv12 architecture, the image flow network first performs convolution operations on the fused image, introduces a residual shortcut connection from input to output throughout the model, and adds an adjustable number of C3k modules. The number of C3k modules is adjusted according to the input requirements of different images to construct a depth visual transformer. Furthermore, the image flow network design uses transition layers to adjust channel dimensions and generate a single visual feature map, which is then processed by subsequent modules before being stitched together. The method of this embodiment not only retains the feature aggregation capability of the original image but also reduces computational cost and parameters.

[0102] The radiation feature flow network is responsible for processing radiation feature information from infrared images. Radiation feature sequence data has good extractability and feature representation capabilities, but overly cumbersome radiation feature extraction operations cause computational complexity to increase exponentially with the square of the input sequence length L. Specifically, for a radiation feature input sequence of length L and feature dimension d, feature extraction computation requires L²×d operations because each feature point interacts with other feature points, and excessive computation leads to a multiple increase in the number of parameters. Therefore, the radiation feature flow processing branch uses the simpler Bottleneck module, resulting in a simpler extraction model structure that reduces computational complexity while ensuring sequence feature extraction.

[0103] S4. Fuse visual features and radiation features to obtain multimodal fusion features.

[0104] In the technical solution of this application embodiment, the target detection model fuses visual features and radiation features through a dual-stream architecture to generate multimodal fusion features, as expressed below:

[0105]

[0106] in, This represents visual features extracted based on the YOLOv12 architecture; This represents the radiation features extracted based on the radiation feature flow network; This indicates the feature fusion operation, concat.

[0107] In the technical solutions of the embodiments of this application, such as Figure 4As shown, the target detection model consists of three parts: the backbone network, the neck network, and the head. Visual and radial features, after undergoing a concat operation, enter the backbone for further extraction of fused features. In the neck network, feature extraction and fusion are performed at three different stages. Finally, the head network calculates the loss function and detects the target. In this embodiment, the A2C2f module in the neck network utilizes its attention mechanism and cross-stage connection structure to perform multi-scale optimization of the multimodal fused features. The A2C2f module enhances the fused multimodal features, controlling computational complexity while maintaining feature extraction capabilities, making it particularly suitable for the multi-scale detection needs of small targets on UAVs.

[0108] This application provides a preferred implementation where the target detection model is trained using a multi-task learning mechanism. A regression branch is added next to the detection head of the target detection model to predict the radiation intensity of the target area. The target detection model is configured to simultaneously output the detection results of small UAV targets and the radiation intensity prediction results of the target area. The main task of the target detection model is to output the detection results of small UAV targets, and the auxiliary task is to predict the radiation intensity of the target area, i.e., radiation intensity regression.

[0109] The overall loss function of the target detection model consists of the target detection loss and the radiation feature regression loss, and is used to simultaneously output the detection results of small targets of the UAV and the radiation intensity prediction results of the target area.

[0110] The expression for the overall loss function is:

[0111]

[0112] in, Represents the overall loss function; Indicates the target detection loss. ; This represents the regression loss of radiation characteristics; Indicates positioning loss; Indicates confidence loss; Indicates category loss; This represents the weight hyperparameter, used to balance the effects of the two losses.

[0113] Object detection loss functions include: localization loss, confidence loss, and class loss. Localization loss is used to optimize the error in the predicted bounding box position; it calculates the difference between the ground truth bounding box and the predicted bounding box, and its expression is as follows:

[0114]

[0115] in, and These represent the coordinates of the center point of the prediction box; and These represent the width and height of the prediction box, respectively; , , and These represent the corresponding coordinates of the actual bounding box; This indicates the indicator function, which is 1 if an object exists, and 0 otherwise. The value represents the weight used to control the loss term; N represents the batch size, i.e., the total number of samples processed simultaneously in one training session; i represents the sample index, used to iterate through each sample in the batch for calculation.

[0116] Confidence loss is used to determine whether the predicted bounding box contains an object. Binary cross-entropy loss is typically used to calculate the presence of an object, as shown in the following expression:

[0117]

[0118] in, Indicates the confidence level of the predicted bounding box; Indicates the confidence level of the true bounding box; This indicates an indicator function; the value is 1 if there is an object in the current cell. This indicates an indicator function; the value is 1 when the current cell contains no objects. This represents the weight used to control the confidence loss when there is no object.

[0119] The category loss uses multi-class cross-entropy loss to optimize the probability prediction for each class, as shown in the following expression:

[0120]

[0121] in, This represents the predicted probability of the predicted bounding box for category c. The true probability is represented by C, which represents the number of categories of the true bounding box.

[0122] The radiation eigenregression loss can be calculated using the mean squared error (MSE), as shown in the following expression:

[0123]

[0124] in, Indicates the first i The predicted radiation intensity of each sample Indicates the first i The true radiation intensity of each sample.

[0125] This application provides a preferred implementation method in which the training process of the target detection model is as follows: the parameters are optimized using the Adam optimizer; after training, the precision, recall, and mean precision are used as the performance evaluation indicators of the target detection model.

[0126] The fused image and infrared radiation feature data are input into the target detection model for multiple rounds of iterative training. During training, the Adam optimizer is used to progressively improve the model's performance. After training, the target detection model is evaluated using metrics including precision, recall, mAP, and F1-score. During deployment, the model's operational efficiency is optimized to enable real-time detection of small targets on UAVs.

[0127] After image data is input, the model built in S4 undergoes multiple iterations, and the Adam optimizer is used during training to gradually improve the model's performance. After training, the model is evaluated. Evaluation metrics include precision, recall, mAP, and F1-score. During model deployment, the model's operational efficiency is optimized to enable real-time detection of small targets by UAVs.

[0128] S5. Based on multimodal fusion features, output the detection results of small targets on the UAV. Input the multimodal fusion features into the detection head, calculate and output the detection results of small targets on the UAV.

[0129] This application also provides a small target detection device for unmanned aerial vehicles based on YOLOv12 and infrared radiation characteristics, comprising:

[0130] The data acquisition module is used to acquire infrared and visible light images of the area to be detected.

[0131] The radiation feature inversion module is used to calculate infrared radiation features based on infrared images.

[0132] The image fusion module is used to fuse infrared images and visible light images into a fused image.

[0133] The dual-stream feature extraction module is used to input the fused image into an image stream network based on the YOLOv12 architecture to extract visual features, while simultaneously inputting infrared radiation features into a radiation feature stream network based on an optimized Bottleneck module to extract radiation features.

[0134] The feature fusion and detection module is used to fuse visual features and radiation features to obtain multimodal fusion features, and output the detection results of small targets on UAVs based on these features.

[0135] In this embodiment of the application, the effectiveness of the technical solution is verified through comparative experiments. YOLOv5-SC, YOLOv7-tiny, YOLOv8-GD, and YOLOv12, representative models from the prior art, are selected for comparative experiments. mAP50, mAP50-95, FPS (frames per second), and Parameters are used as evaluation metrics.

[0136] Two evaluation metrics based on mAP are used: mAP50 is the mAP at an IoU threshold of 0.5. IoU measures the overlap between the predicted bounding box and the ground truth bounding box, with values ​​ranging from [0,1]. A higher value indicates a higher degree of overlap. mAP50-95 is a more stringent metric, calculating the average accuracy of the model within an IoU threshold range of 0.5 to 0.95 (inclusive). This provides a more comprehensive evaluation of the model's performance at different IoU thresholds, reflecting the model's stability and robustness under varying detection difficulties.

[0137] Parameters can intuitively reflect the scale and complexity of the model, while FPS reflects the detection speed of the model, which is of great significance for evaluating the practicality and embedded feasibility of the model.

[0138] The specific data is shown in Table 1:

[0139] Table 1 Comparison of experimental results

[0140]

[0141] The specific results show that the technical solution presented in this application leads in detection accuracy across the board, has a significant advantage in inference speed, and achieves a balanced model efficiency. This application achieves an mAP50 of 0.939, a 2.7% improvement over the best baseline YOLOv12 (0.912) and a 9.9% improvement over YOLOv5-SC (0.854). The mAP50:95 score reaches 0.608, significantly outperforming all compared existing technologies and representing a 5.2% improvement over the best baseline YOLOv5-SC (0.578). With an FPS of 165.4 frames per second, this application maintains the highest inference speed despite the increased number of parameters, representing a 71.8% improvement over YOLOv12 (96.3 FPS), demonstrating significant engineering optimization. Regarding the number of parameters, this application has 28.1M, higher than YOLOv12 (16.6M) but slightly lower than YOLOv8-GD (25.7M); achieving a dual improvement in accuracy and speed with a moderate increase in the number of parameters. The method and apparatus for detecting small targets on UAVs based on YOLOv12 and infrared radiation features proposed in this application achieve the best balance in terms of detection accuracy, inference speed and model complexity, and effectively solve the technical problems of insufficient detection accuracy and poor environmental adaptability of existing technologies in complex environments.

[0142] This application provides a method and apparatus for small target detection on unmanned aerial vehicles (UAVs) based on YOLOv12 and infrared radiation features. By introducing infrared radiation features and visual images to form multimodal fusion features, it provides the model with strong discriminative basis directly related to the physical properties of the target, significantly enhancing the feature representation capability of small targets and effectively improving detection accuracy and recall in complex backgrounds. A dedicated dual-stream processing architecture is constructed, employing an image stream network based on the YOLOv12 architecture (including residual shortcut connections and an adjustable number of C3k modules) to ensure the quality of visual feature extraction. Simultaneously, a radiation feature stream network based on an optimized Bottleneck module effectively reduces computational complexity, achieving an optimized balance between detection accuracy and computational efficiency. The target detection model is trained through a multi-task learning mechanism, enabling the model to simultaneously learn target detection and radiation intensity prediction, enhancing feature representation capability and model generalization ability. Combined with multi-scale feature fusion and enhancement of the neck network, the robustness and reliability of detection in various harsh environments are significantly improved.

[0143] This application provides a method and apparatus for detecting small targets on UAVs based on YOLOv12 and infrared radiation features. It effectively solves the technical problems of weak feature information and difficulty in extracting effective features when the pixel ratio of small targets on UAVs is extremely small, the background is complex, and the lighting conditions are poor, resulting in low detection accuracy and high false negative rate.

[0144] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0145] The units described in the embodiments of this application can be implemented in software or hardware. The names of the units are not, in some cases, limiting the scope of the unit itself.

Claims

1. A UAV small target detection method based on YOLOv12 and infrared radiation features, characterized in that, The method comprises the following steps: Obtain an infrared image and a visible light image of a to-be-detected region, and obtain corresponding infrared radiation features through inversion; After pre-processing the infrared image and the visible light image, a fused image is obtained through fusion; The fused image is input into an image stream network of a target detection model to extract visual features; at the same time, The infrared radiation features are input into a radiation feature stream network of the target detection model to extract radiation features; The visual features and the radiation features are fused to obtain multi-modal fusion features; Based on the multi-modal fusion features, a detection result of a small target of a UAV is output; The image stream network is based on a YOLOv12 architecture; The radiation feature stream network is based on a simplified network structure of an optimized Bottleneck module; The image stream network based on the YOLOv12 architecture comprises the following steps: A residual shortcut connection from input to output is introduced in the backbone network of the YOLOv12 neural network; An adjustable number of C3k modules are used to replace the original convolution modules, and the C3k modules construct a deep visual transformer according to the input features; The simplified network structure based on the optimized Bottleneck module comprises the following steps: A simplified network structure composed of multiple optimized Bottleneck modules; The calculation complexity in the feature extraction process of the radiation feature sequence is reduced through separation of channel and spatial dimension calculation; Based on the multi-modal fusion features, the detection result of the small target of the UAV is output, which comprises the following steps: The multi-modal fusion features are input into the neck network of the YOLOv12 architecture; Through the A2C2f module in the neck network, multi-scale feature fusion and enhancement are performed on the multi-modal fusion features; The enhanced features are input into a detection head to output the detection result of the small target of the UAV. The target detection model is trained through a multi-task learning mechanism. 2.The UAV small target detection method of claim 1, wherein, The multi-task learning mechanism comprises the following steps: 3.The UAV small target detection method of claim 2, wherein, The target detection model is configured to simultaneously output the detection result of the small target of the UAV and a radiation intensity prediction result of the target region; The overall loss function of the target detection model is composed of a target detection loss and a radiation feature regression loss; The target detection loss is calculated according to the detection result and the true label, and the radiation feature regression loss is calculated according to the radiation intensity prediction result and the true radiation intensity label. The training process of the target detection model comprises the following steps:

4. The UAV small target detection method of claim 3, wherein, An Adam optimizer is used for parameter optimization; After training is completed, the precision, recall and average precision are used as evaluation indicators for the performance of the target detection model. The pre-processing of the infrared image and the visible light image comprises the following steps:

5. The UAV small target detection method of claim 1, wherein, Non-local mean-based denoising processing is performed on the infrared image; Super-resolution reconstruction processing based on bicubic interpolation is performed on the visible light image. The inversion to obtain the corresponding infrared radiation features comprises the following steps: 6.The UAV small target detection method of claim 1, wherein, An infrared camera is calibrated to establish a mapping relationship between the target temperature and the image gray value; ​ The infrared radiation characteristic is obtained through infrared radiation inversion calculation by using a system response coefficient, an atmospheric transmittance, a target parameter, and an original infrared image.

7. A UAV small target detection device for implementing the UAV small target detection method according to any one of claims 1-6. The method comprises the following steps: a data acquisition module is configured to acquire an infrared image and a visible light image of a detection area; a radiation characteristic inversion module is configured to calculate an infrared radiation characteristic according to the infrared image; an image fusion module is configured to fuse the infrared image and the visible light image into a fused image; a dual-flow feature extraction module is configured to input the fused image into a YOLOv12 architecture-based image flow network to extract visual features, and input the infrared radiation characteristic into an optimized Bottleneck module-based radiation feature flow network to extract radiation features; a feature fusion and detection module is configured to fuse the visual features and the radiation features to obtain multi-modal fusion features, and output a detection result of a small target of an unmanned aerial vehicle based on the multi-modal fusion features.

Images