An infrared target detection method and system based on a YOLO improved model

Abstract

The application discloses an infrared target detection method and system based on a YOLO improved model, and belongs to the technical field of computer vision, comprising the following steps: in the feature extraction module of the YOLOV8 model backbone network, a windmill convolution is used to replace the standard convolution in the bottleneck layer to obtain an improved backbone network; in the neck network of the YOLOV8 model, a frequency dynamic upsampling feature fusion module is used to replace the feature fusion module and the upsampling module to obtain an improved neck network; a convolution block attention module is arranged at the connection position of the improved backbone network and the improved neck network; in the head network of the YOLOV8 model, a wisdom intersection over union loss function is used as a bounding box regression loss function to obtain an improved head network; the improved YOLOV8 model obtained based on the improved backbone network, the improved neck network, the convolution block attention module and the improved head network is used as a detection model; and the detection model is used for infrared target detection.

Description

Technical Field

[0001] This application belongs to the field of computer vision technology, specifically relating to an infrared target detection method and system based on an improved YOLO model. Background Technology

[0002] With the continuous development of technology, target detection in good environments can no longer meet the needs of special industries. Infrared target detection has the advantages of all-weather operation, long range and strong anti-interference. Therefore, UAV-based infrared target detection plays an important role in military, accident search and rescue and traffic monitoring.

[0003] However, key challenges remain in the field of infrared small target detection:

[0004] 1. Because the target is far away and small in size, it occupies a very small proportion in the whole image. The target in the infrared image often only appears as a blurry spot, without any shape or texture features that can be extracted.

[0005] 2. Low contrast and complex background: Infrared images usually have low contrast, and the brightness difference between the target and the background is not significant, resulting in blurred edges and contours. For example, sea ripples, clouds, smoke or ground heat sources may have similar brightness characteristics to the target, leading to a high false detection rate. Summary of the Invention

[0006] Purpose of the invention: This application develops an infrared target detection method and system based on an improved YOLO model, aiming to solve the technical problems in the prior art.

[0007] Technical Solution: In a first aspect, embodiments of this application provide an infrared target detection method based on an improved YOLO model, comprising:

[0008] In the feature extraction module of the YOLOV8 model backbone network, the standard convolution in the bottleneck layer is replaced by windmill convolution to obtain an improved backbone network;

[0009] In the neck network of the YOLOV8 model, an improved neck network is obtained by replacing the feature fusion module and the upsampling module with a frequency dynamic upsampling feature fusion module. The steps for obtaining the frequency dynamic upsampling feature fusion module include:

[0010] The frequency-aware feature fusion module is used to extract high-frequency and low-frequency information from features through adaptive high-pass and adaptive low-pass filters, so that low-level features have high-frequency information and high-level features have low-frequency information.

[0011] Based on the frequency-aware feature fusion module, channel attention mechanism, and cross-fusion, the frequency dynamic upsampling feature fusion module is obtained. The channel attention mechanism is used to enhance the attention of features. The cross-fusion adopts two methods, feature addition and feature channel fusion, to fuse feature information at different levels and improve local semantic information. The representation formula of the frequency dynamic upsampling feature fusion module includes:

[0012] ;

[0013] ;

[0014] ;

[0015] ;

[0016] ;

[0017] ;

[0018] ;

[0019] ;

[0020] in, Original low-resolution input; For raw, high-resolution input; Number of channels; For height; Width; For pointwise convolution; This is a low-resolution input after unifying the number of input channels through pointwise convolution; For frequency-aware feature fusion module; These are the high-level features after feature fusion; The frequency-aware feature fusion module performs frequency feature fusion on the original low-resolution input and the original high-resolution input, resulting in a low-resolution output. For splicing operations; This is the concatenated input feature map; This is a global draw pooling operation; This is a pointwise convolution operation; For activation functions; Attention features; For feature splitting operations; and These are the two attention features after feature splitting; Multiply by channel; This is an upsampling operation; To Features after upsampling and attention features The result after multiplying by the channels; For raw high-resolution input and attention characteristics The result after multiplying by the channels; for and To each and Perform element-by-element addition and then concatenate the two outputs by adding them together.

[0021] A convolutional block attention module is provided at the connection between the improved backbone network and the improved neck network;

[0022] In the YOLOV8 model head network, the intelligent intersection-union ratio loss function is used as the bounding box regression loss function to obtain an improved head network.

[0023] The improved YOLOV8 model is obtained as the detection model based on the improved backbone network, the improved neck network, the convolutional block attention module, and the improved head network.

[0024] Infrared target detection is performed using the aforementioned detection model.

[0025] In some embodiments, the convolutional block attention module is connected to the feature extraction module in the improved backbone network, and the convolutional block attention module is connected to the frequency dynamic upsampling feature fusion module in the improved neck network.

[0026] In some embodiments, the intelligent intersection-union loss function, through a dynamic non-monotonic focusing mechanism, enables the detection model to pay more attention to high error rate samples and high-difficulty samples;

[0027] The intelligent intersection-union ratio loss function uses a gradient allocation method to distribute gradient gain to anchor boxes of different qualities, in order to avoid the model being misled by abnormal samples.

[0028] In some embodiments, the representation formula of the smart cross-union ratio loss function includes:

[0029] ;

[0030] ;

[0031] ;

[0032] ;

[0033] ;

[0034] ;

[0035] ;

[0036] ;

[0037] in, The ratio is used to measure the degree of overlap between the ground truth bounding box and the predicted bounding box; The bounding box loss function; (x,y) and ( , These are the location information of the predicted bounding box and the ground truth bounding box, respectively; , ) represents the width and height of the smallest bounding box that merges the predicted and ground truth boxes; * represents... and The separation is used to mitigate harmful gradients; is the normalized distance between the center points of the ground truth bounding box and the predicted bounding box; m is the momentum; t is the number of training epochs; n is the batch size; This represents the average bounding box loss. The actual bounding box loss; m is used for delay. near The time, and take the time after training to t rounds. ; This represents the actual bounding box loss value; The degree of outlier in the anchor frame; and These are hyperparameters used to control the gradient gain r and the outlierness of the anchor frame. The relationship between them; The third-generation intelligent cross-union ratio loss function; The intelligent intersection-union ratio loss function; It is the third generation; Gradient gain; This is the first generation.

[0038] In some embodiments, the feature extraction module of the improved neck network replaces the standard convolution in the bottleneck layer with a windmill convolution.

[0039] Secondly, embodiments of this application also provide an infrared target detection system based on an improved YOLO model, comprising:

[0040] The backbone network improvement module is used to obtain an improved backbone network by replacing the standard convolution in the bottleneck layer with windmill convolution in the feature extraction module of the YOLOV8 model backbone network.

[0041] A neck network improvement module is used to replace the feature fusion module and the upsampling module in the neck network of a YOLOV8 model with a frequency dynamic upsampling feature fusion module to obtain an improved neck network; the steps for obtaining the frequency dynamic upsampling feature fusion module include:

[0042] The frequency-aware feature fusion module is used to extract high-frequency and low-frequency information from features through adaptive high-pass and adaptive low-pass filters, so that low-level features have high-frequency information and high-level features have low-frequency information.

[0043] Based on the frequency-aware feature fusion module, channel attention mechanism, and cross-fusion, the frequency dynamic upsampling feature fusion module is obtained. The channel attention mechanism is used to enhance the attention of features. The cross-fusion adopts two methods, feature addition and feature channel fusion, to fuse feature information at different levels and improve local semantic information. The representation formula of the frequency dynamic upsampling feature fusion module includes:

[0044] ;

[0045] ;

[0046] ;

[0047] ;

[0048] ;

[0049] ;

[0050] ;

[0051] ;

[0052] in, Original low-resolution input; For raw, high-resolution input; Number of channels; For height; Width; For pointwise convolution; This is a low-resolution input after unifying the number of input channels through pointwise convolution; For frequency-aware feature fusion module; These are the high-level features after feature fusion; The frequency-aware feature fusion module performs frequency feature fusion on the original low-resolution input and the original high-resolution input, resulting in a low-resolution output. For splicing operations; This is the concatenated input feature map; This is a global equal pooling operation; This is a pointwise convolution operation; For activation functions; Attention features; For feature splitting operations; and These are the two attention features after feature splitting; Multiply by channel; This is an upsampling operation; To Features after upsampling and attention features The result after multiplying by the channels; For raw high-resolution input and attention characteristics The result after multiplying by the channels; for and To each and Perform element-by-element addition and then concatenate the two outputs by adding them together.

[0053] A connection improvement module is used to set a convolutional block attention module at the connection between the improved backbone network and the improved neck network;

[0054] The head network improvement module is used to obtain an improved head network in the YOLOV8 model head network by using the intelligent intersection-over-union loss function as the bounding box regression loss function.

[0055] The detection model acquisition module is used to acquire the improved YOLOV8 model as the detection model based on the improved backbone network, the improved neck network, the convolutional block attention module and the improved head network;

[0056] The target detection module is used to perform infrared target detection using the detection model.

[0057] Beneficial Effects: Compared with existing technologies, the infrared target detection method based on an improved YOLO model provided in this application includes: in the feature extraction module of the YOLOV8 model backbone network, replacing the standard convolution in the bottleneck layer with windmill convolution to reduce the use of parameters and enhance the attention of features in the receptive field center, thus obtaining an improved backbone network; in the YOLOV8 model neck network, replacing the feature fusion module and upsampling module with a frequency dynamic upsampling feature fusion module to improve the local and global semantic information of the receptive field without increasing parameters, effectively solving the problems of inconsistency within object classes, blurred boundaries, and boundary displacement, thus obtaining an improved neck network; setting a convolutional block attention module at the connection between the improved backbone network and the improved neck network to improve the detection efficiency of small targets; in the YOLOV8 model head network, using the intelligent intersection-union loss function as the bounding box regression loss function to establish a reasonable gradient gain distribution and increase the sensitivity to small targets, thus obtaining an improved head network; obtaining an improved YOLOV8 model based on the improved backbone network, improved neck network, convolutional block attention module, and improved head network as the detection model; and performing infrared target detection through the detection model. Attached Figure Description

[0058] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0059] Figure 1 A flowchart illustrating the steps of the infrared target detection method based on the improved YOLO model provided in this application embodiment;

[0060] Figure 2 A module connection diagram of an infrared target detection system based on an improved YOLO model provided in an embodiment of this application;

[0061] Figure 3 A model diagram of windmill convolution;

[0062] Figure 4 The structure diagrams for integrating windmill convolution into the bottleneck layer and the improved feature extraction module are shown.

[0063] Figure 5 This is a schematic diagram of the frequency-aware feature fusion module.

[0064] Figure 6 This is a schematic diagram of the structure of the frequency dynamic upsampling feature fusion module;

[0065] Figure 7 This is a structural diagram of the convolutional block attention module;

[0066] Figure 8 This is a structural diagram of the detection model in the infrared target detection method based on the improved YOLO model provided in the embodiments of this application;

[0067] Figure 9 A schematic diagram of the HIT-UAV dataset;

[0068] Figure 10 A schematic diagram of the FLIR dataset;

[0069] Figure 11 A schematic diagram for visualizing experimental results 1;

[0070] Figure 12 Schematic diagram for visualizing experimental results 2;

[0071] Figure 13 This is a structural diagram of the original YOLOv8 model;

[0072] Figure labels: 10, Backbone network improvement module; 20, Neck network improvement module; 30, Connection improvement module; 40, Head network improvement module; 50, Detection model acquisition module; 60, Target detection module. Detailed Implementation

[0073] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0074] With the continuous development of technology, target detection in good environments can no longer meet the needs of special industries. Infrared target detection has the advantages of all-weather operation, long range and strong anti-interference. Therefore, UAV-based infrared target detection plays an important role in military, accident search and rescue and traffic monitoring.

[0075] Please see Figure 13 , Figure 13 This is a structural diagram of the original YOLOv8 model. YOLO, as a single-stage target detection model, has the advantages of speed and accuracy in target detection, but it still faces key challenges in infrared small target detection:

[0076] 1. Because the target is far away and small in size, it occupies a very small proportion in the whole image. The target in the infrared image often only appears as a blurry spot, without any shape or texture features that can be extracted.

[0077] 2. Low contrast and complex background: Infrared images usually have low contrast, and the brightness difference between the target and the background is not significant, resulting in blurred edges and contours. For example, sea ripples, clouds, smoke or ground heat sources may have similar brightness characteristics to the target, leading to a high false detection rate.

[0078] In view of this, embodiments of this application provide an infrared target detection method based on an improved YOLO model. This method includes: in the feature extraction module of the YOLOV8 model backbone network, replacing the standard convolution in the bottleneck layer with a windmill convolution to reduce parameter usage and enhance the attention to features in the receptive field center, thus obtaining an improved backbone network; in the YOLOV8 model neck network, replacing the feature fusion module and upsampling module with a frequency dynamic upsampling feature fusion module to improve local and global semantic information of the receptive field without increasing parameters, effectively solving problems of inconsistency within object classes, blurred boundaries, and boundary displacement, thus obtaining an improved neck network; setting a convolutional block attention module at the connection between the improved backbone network and the improved neck network to improve the detection efficiency of small targets; in the YOLOV8 model head network, using the intelligent intersection-union loss function as the bounding box regression loss function to establish a reasonable gradient gain distribution and increase sensitivity to small targets, thus obtaining an improved head network; obtaining an improved YOLOV8 model based on the improved backbone network, improved neck network, convolutional block attention module, and improved head network as the detection model; and performing infrared target detection using the detection model.

[0079] In some embodiments, please refer to Figure 1 , Figure 1 This is a flowchart illustrating the steps of the infrared target detection method based on the improved YOLO model provided in this application embodiment. The infrared target detection method based on the improved YOLO model provided in this application embodiment is specifically implemented through steps 100 to 600:

[0080] Step 100: In the feature extraction module of the YOLOV8 model backbone network, the standard convolution in the bottleneck layer is replaced by windmill convolution to obtain the improved backbone network.

[0081] Understandably, the YOLO series of algorithms are among the fastest, smallest, and most accurate single-stage object detection models. Common single-stage detection models include SSD and RetinaNet. YOLOv8 can simultaneously support multiple vision tasks, such as object detection, semantic segmentation, and image classification. It also employs an anchor-free design, simplifying the computation process and improving detection speed. Furthermore, YOLOv8 is easy to deploy, making it very suitable for industrial applications. Therefore, this application adopts YOLOv8 as the benchmark algorithm for infrared small target detection. The windmill convolution algorithm performs well in infrared small target detection. Integrating this convolution into the feature extraction module of the YOLOv8 backbone network improves the model's feature extraction capability for infrared small targets.

[0082] Specifically, please refer to Figure 3 and Figure 4 , Figure 3 This is a model diagram of windmill convolution. Figure 4 To integrate the windmill convolution into the structure diagram of the bottleneck layer and the improved feature extraction module, this application introduces the windmill convolution into the feature extraction module (C2f) of the backbone network, replacing the standard convolution in the bottleneck layer of the feature extraction module. The standard convolution has a 3×3 kernel, while the windmill convolution has a 3×1 or 1×3 kernel, thus reducing the number of parameters in the improved version. The workflow formula for the windmill convolution is as follows:

[0083] (1)

[0084] The windmill convolution sets asymmetric padding for the rectangular convolution kernel, such as P(1,0,0,3) in formula (1). These four numbers represent the padding amounts in the left, right, top, and bottom directions, respectively. (1,0,0,3) means that 1 row is filled on the left and 3 rows are filled on the bottom, requiring a downward 1×3 convolution. After the convolution, batch normalization (BN) is performed to map the feature values ​​to the range of 0-1. Then, the activation function (SiLU) is applied, and finally, the output is... To prevent gradient explosion or vanishing, the windmill shape requires four padding operations: P(1,0,0,3), P(0,3,0,1), P(0,1,3,0), and P(3,0,1,0), representing downward, rightward, upward, and leftward convolutions, respectively. The four outputs are then concatenated according to their channel dimensions to obtain a new output. Formula (1) is the most important part of the windmill convolution. In order to output the desired number of channels, a 2×2 standard convolution is added, as shown in the following formula:

[0085] (2)

[0086] Step 200: In the neck network of the YOLOV8 model, the feature fusion module and the upsampling module are replaced by the frequency dynamic upsampling feature fusion module to obtain the improved neck network.

[0087] Understandably, most improvements to feature fusion in the YOLO series models are achieved by upsampling through nearest neighbor interpolation or downsampling using progressive convolution to transform features of different scales into a unified scale before fusing them along the channel dimensions. For infrared targets, simply extracting features from the surface does not yield good results. Therefore, this application uses frequency as the main component of feature fusion for small infrared targets.

[0088] In some embodiments, the step of obtaining the frequency dynamic upsampling feature fusion module includes: obtaining a frequency-aware feature fusion module, used to extract high-frequency and low-frequency information from features through an adaptive high-pass filter and an adaptive low-pass filter, so that low-level features have high-frequency information and high-level features have low-frequency information; obtaining the frequency dynamic upsampling feature fusion module based on the frequency-aware feature fusion module, the channel attention mechanism, and cross fusion, wherein the channel attention mechanism is used to enhance the attention of features, and the cross fusion adopts two methods, feature addition and feature channel fusion, to fuse feature information at different levels and improve local semantic information.

[0089] In some embodiments, please refer to Figure 5 , Figure 5 This is a schematic diagram of the frequency-aware feature fusion module, including an adaptive high-pass filter, an offset generator, and an adaptive low-pass filter. The adaptive low-pass filter is mainly used to solve the problems of intra-class inconsistency and boundary displacement within objects. However, the low-pass filter cannot solve the problems of widespread intra-class inconsistency and boundary region refinement. The offset generator helps to obtain more consistent features and more accurate boundary delineation. The adaptive low-pass filter and the offset generator effectively recover the upsampled high-level features with high intra-class consistency and fine boundaries. However, the detailed boundary information existing in the low-level features lost during downsampling cannot be fully recovered in the high-level features. The high-pass filter enhances high-frequency power above the Nyquist frequency, highlighting its ability to capture and preserve complex details and boundaries, which is crucial for tasks requiring high-resolution and accurate feature representation. The combination of these three components effectively solves the problems of intra-class inconsistency, boundary ambiguity, and boundary displacement by extracting and combining high (low) frequency information from features at different levels.

[0090] In some embodiments, feature fusion is particularly important to fully utilize global context information. This application proposes an FDUFM frequency dynamic upsampling feature fusion module based on the FreqFusion frequency-aware feature fusion module. Please refer to [link to relevant documentation]. Figure 6 , Figure 6This is a schematic diagram of the frequency dynamic upsampling feature fusion module. This module fuses frequency features and backbone network features from different levels using various methods, such as addition and concatenation, to obtain a new fused feature map containing frequency information, shallow detail information, and deep semantic information. This fused feature map exhibits better performance in small object detection. The representation formula for the frequency dynamic upsampling feature fusion module includes:

[0091] (3)

[0092] (4)

[0093] (5)

[0094] (6)

[0095] (7)

[0096] (8)

[0097] (9)

[0098] (10)

[0099] in, Original low-resolution input; For raw, high-resolution input; Number of channels; For height; Width; For pointwise convolution; This is a low-resolution input after unifying the number of input channels through pointwise convolution; For frequency-aware feature fusion module; These are the high-level features after feature fusion; The frequency-aware feature fusion module performs frequency feature fusion on the original low-resolution input and the original high-resolution input, resulting in a low-resolution output. For splicing operations; This is the concatenated input feature map; This is a global draw pooling operation; This is a pointwise convolution operation; For activation functions; Attention features; For feature splitting operations; and These are the two attention features after feature splitting; Multiply by channel; This is an upsampling operation; To Features after upsampling and attention features The result after multiplying by the channels; For raw high-resolution input and attention characteristics The result after multiplying by the channels; for and To each and After performing element-by-element addition, the two outputs are then concatenated by adding them together.

[0100] Understandably. Formula (5) fuses two features along the channel dimension using Concat, Formula (6) employs global average pooling, one-dimensional convolution, and activation functions to construct an attention mechanism, and Formulas (7-9) combine features along the channel dimension. The inputs of FFM are divided equally and applied to the two inputs of FFM respectively. Here, × means multiplying by channel, dynamically amplifying the channels that are more critical to the current task and suppressing redundant or noisy channels. Formula (10) cross-adds the input of channel attention to the output of FFM element by element and then fuses them. Cross-fusion can fill in the edge features at different scales and solve the problem of the disappearance of edge features of small targets at different scales.

[0101] In some embodiments, in the feature extraction module of the improved neck network, the standard convolution in the bottleneck layer is replaced by a windmill convolution.

[0102] Step 300: Set up convolutional block attention modules at the connection between the improved backbone network and the improved neck network.

[0103] In some embodiments, the convolutional block attention module is connected to the feature extraction module in the improved backbone network, and the convolutional block attention module is connected to the frequency dynamic upsampling feature fusion module in the improved neck network.

[0104] In some embodiments, the convolutional block attention module is connected to the feature extraction module in the improved backbone network, and the convolutional block attention module is connected to the frequency dynamic upsampling feature fusion module in the improved neck network.

[0105] Understandingly, attention mechanisms can be categorized into channel attention mechanisms ("what to look at"), spatial attention mechanisms ("where to look"), hybrid attention mechanisms, and Transformer-like attention mechanisms. CBAM is a hybrid attention mechanism that simultaneously focuses on both "what to look at" and "where to look." It first selects key feature channels using channel attention (CAM) and then determines the location of key features using spatial attention (SAM), ensuring comprehensive information filtering. CBAM uses only global average pooling and max pooling, resulting in a very small number of parameters and lightweight computation. CBAM maintains consistent input and output sizes, eliminating the need to adjust subsequent network structures, making it easy to integrate, and significantly improving detection accuracy. By adding a Convolutional Block Attention (CBAM) module between the backbone and neck network, spatial and channel attention are applied to the extracted features, highlighting semantically rich features and providing further assurance for local semantic information extraction.

[0106] Specifically, please refer to Figure 7 , Figure 7 This is a structural diagram of the Convolutional Block Attention (CBAM), a method to improve the representation capabilities of CNN networks. This module uses two unique modules, channel and spatial, for attention-based feature optimization, achieving considerable performance gains while maintaining relatively low overhead. For channel attention, max-pooling and average-pooling features are used, resulting in more refined attention than SE. Placing CBAM behind the features of the backbone network used for neck feature fusion further enhances performance by leveraging spatial attention. CBAM learns what and where to emphasize or suppress intermediate features and effectively refine them.

[0107] Step 400: In the YOLOV8 model head network, use the intelligent intersection-union loss function as the bounding box regression loss function to obtain the improved head network.

[0108] Understandably, the bounding box loss function of the head network is replaced with the Smart Intersection over Union (WIoU) loss function. This loss function employs a dynamic non-monotonic focusing mechanism and gradient allocation strategy to better handle the balance between high-quality and low-quality samples, preventing the model from being misled by some anomalous samples. The bounding box regression loss function is crucial for object detection. Its well-defined loss function can significantly improve the model's performance. Most existing works assume that the samples in the training data are of high quality and focus on enhancing the fitting ability of the bounding box regression loss. If bounding box regression is strengthened on low-quality examples, localization performance will be compromised. The Smart Intersection over Union (WIoU) based on the dynamic non-monotonic focusing mechanism loss of IoU provides a smart gradient gain allocation strategy. This strategy reduces the competitiveness of high-quality anchor boxes while also reducing harmful gradients generated by low-quality examples, focusing on low-quality anchor boxes and improving the overall performance of the detector.

[0109] In some embodiments, the intelligent intersection-union ratio loss function uses a dynamic non-monotonic focusing mechanism to enable the detection model to pay more attention to high error rate samples and high difficulty samples; the intelligent intersection-union ratio loss function uses a gradient allocation method to allocate gradient gain to anchor boxes of different qualities to avoid the model being misled by abnormal samples.

[0110] In some embodiments, this application replaces the full intersection-union ratio (IoU) with the third-generation intelligent intersection-union ratio (WIoU v3). The WIoU function dynamically adjusts the penalty for different anchor box sizes using an IoU-based gradient allocation strategy, reducing sensitivity to positional errors and improving performance when faced with diverse geometric factors (such as distance and aspect ratio). This enhances the sensitivity to small targets and the model's generalization performance. The representation formula of the intelligent intersection-union loss function includes:

[0111] (11)

[0112] (12)

[0113] (13)

[0114] in, This represents a ratio that measures the degree of overlap between the ground truth bounding box and the predicted bounding box. Let (x,y) represent the bounding box loss function. , ) represent the location information of the predicted bounding box and the ground truth bounding box, respectively; in addition, ( , The ) represents the width and height of the smallest bounding box that merges the predicted and ground truth boxes; * indicates... and Separation to mitigate harmful gradients, This represents the normalized distance between the center points of the ground truth bounding box and the predicted bounding box. To establish gradient gain, a dynamic non-monotonic mechanism is allowed to be incorporated into the loss function. WIoU v3 was developed based on WIoU v1. The calculation formula is as follows:

[0115] (14)

[0116] (15)

[0117] (16)

[0118] (17)

[0119] (18)

[0120] Where m represents momentum; t is the number of training rounds of the model, and in this invention t=150; n is the batch size, and in this invention it is set to 16; The average bounding box loss is initialized to 1, allowing samples with IoU of 0 to obtain a larger gradient gain; This represents the actual bounding box loss; the role of m is to delay... near The time, and take the time after training to t rounds. ; This represents the actual bounding box loss value. The outlierness of the anchor boxes is expressed as... and ratio This indicates that the value is inversely proportional to the sample quality; the smaller the value, the better the quality. Hyperparameters and Set to 1.9 and 3; these hyperparameters and Controls the gradient gain r and outlier degree The relationship between them; Third-generation intelligent cross-union ratio loss function; The intelligent intersection-union ratio loss function; This is the third generation of the loss function; Gradient gain; This is the first generation of the loss function. and According to the degree of dispersion It can be seen that WIoU v3 penalizes high-quality prediction boxes less, while penalizing low-quality prediction boxes more.

[0121] Step 500: Obtain the improved YOLOV8 model as the detection model based on the improved backbone network, improved neck network, convolutional block attention module, and improved head network.

[0122] In some embodiments, please refer to Figure 8 , Figure 8 This diagram illustrates the structure of the detection model in the infrared target detection method based on the improved YOLO model provided in this application. The improved model enhances the YOLOv8 architecture through C2f-P, FDUFM, CBAM, and WIoU v3. The C2f-P module uses fewer parameters while increasing the focus on features at the center of the receptive field. FDUFM improves the local and global semantic information of the receptive field without increasing the number of parameters. CBAM improves the detection efficiency for small targets. WIoU v3 establishes a reasonable gradient gain distribution, increasing sensitivity to small targets.

[0123] Step 600: Perform infrared target detection using the detection model.

[0124] Understandably, this application incorporates windmill convolutions into the YOLOv8 backbone network. By extracting features from four different directions through elongated convolutions and then stacking them along the channel dimension, the central features receive strong attention. Integrating the windmill convolutions into the bottleneck of the C2f module allows the C2f module, originally used for feature extraction, to effectively extract features of small infrared targets, addressing the challenge of weak features in small infrared targets. Improvements to backbone feature extraction lay the foundation for subsequent feature fusion. A frequency dynamic upsampling feature fusion module is added to the neck network. Frequency extraction from the image comes from the frequency-aware feature fusion module within this module. The frequency-aware feature fusion module extracts frequencies of different levels of features using adaptive high-pass and adaptive low-pass filters to address the issues of blurred boundaries and intra-class inconsistencies in small infrared targets. Based on this, the module is further improved by adding a channel attention mechanism and cross-feature fusion to enhance the model's extraction of local semantic information and address the low contrast problem in infrared images. A convolutional block attention mechanism was added between the backbone network and the neck network. Channel attention and spatial attention were used to enhance the features extracted by the C2f module in the backbone network and then fed into the feature fusion module in the neck network, further ensuring the extraction of local semantic information. A third-generation intelligent intersection-union (IUU) target box loss function was added to the head. Compared to the model's standard IUU, intelligent IUU has the advantage of a dynamic non-monotonic focusing mechanism, which increases the penalty for erroneous samples, making the model pay more attention to samples with higher error rates and greater difficulty. Simultaneously, a gradient allocation strategy was adopted to reasonably distribute gradient gains to anchor boxes of different qualities, preventing the model from being misled by some abnormal samples.

[0125] Understandably, the infrared target detection method based on the improved YOLO model provided in the embodiments of this application includes: in the feature extraction module of the YOLOV8 model backbone network, replacing the standard convolution in the bottleneck layer with windmill convolution to reduce the use of parameters and enhance the attention of features in the receptive field center, thus obtaining an improved backbone network; in the YOLOV8 model neck network, replacing the feature fusion module and upsampling module with a frequency dynamic upsampling feature fusion module to improve the local and global semantic information of the receptive field without increasing parameters, effectively solving the problems of inconsistency within object classes, blurred boundaries, and boundary displacement, thus obtaining an improved neck network; setting a convolutional block attention module at the connection between the improved backbone network and the improved neck network to improve the detection efficiency of small targets; in the YOLOV8 model head network, using the intelligent intersection-union loss function as the bounding box regression loss function to establish a reasonable gradient gain distribution and increase the sensitivity to small targets, thus obtaining an improved head network; obtaining an improved YOLOV8 model based on the improved backbone network, improved neck network, convolutional block attention module, and improved head network as the detection model; and performing infrared target detection through the detection model.

[0126] For example, this application uses two infrared datasets: the HIT-UAV dataset and the FLIR dataset. The HIT-UAV dataset contains 2886 infrared city images taken by drones. HIT-UAV includes five object categories: Person, Car, Bicycle, Other Vehicle, and Don'tCare. The dataset samples are shown below. Figure 9 As shown, Figure 9 This is a schematic diagram of the HIT-UAV dataset. Furthermore, FLIR is only used for generalization experiments, and the dataset samples are as follows... Figure 10 As shown, Figure 10 This is a schematic diagram of the FLIR dataset.

[0127] The visualization comparison results between the improved YOLOv8 and the YOLOv8 model are as follows: Figure 11 As shown, Figure 11 This is a visualization of Experiment 1. The figure shows the detection of objects on and around the playground; blue boxes represent people, and purple boxes represent DontCare. The first column is the true labels. As can be seen from the second column, YOLOv8 exhibited both false positives and false negatives, while the improved YOLOv8 accurately detected the targets. Figure 12 As shown, Figure 12 The diagram below illustrates the results of Experiment 2. The light blue target box represents the Car, and the pink target box represents the Bicycle. As can be seen from the second row, due to the noise in the infrared background, both YOLOv8 and the improved YOLOv8 exhibited false detections. However, the improved model reduced the number of false detections, demonstrating the effectiveness of the improved YOLOv8 model.

[0128] In some embodiments, please refer to Figure 2 , Figure 2 This is a module connection diagram of an infrared target detection system based on an improved YOLO model provided in an embodiment of this application. The infrared target detection system based on an improved YOLO model provided in an embodiment of this application includes:

[0129] Backbone network improvement module 10 is used to obtain an improved backbone network by replacing the standard convolution in the bottleneck layer with windmill convolution in the feature extraction module of the YOLOV8 model backbone network.

[0130] Neck network improvement module 20 is used to replace the feature fusion module and the upsampling module in the neck network of the YOLOV8 model with a frequency dynamic upsampling feature fusion module to obtain an improved neck network; the steps of obtaining the frequency dynamic upsampling feature fusion module include:

[0131] The frequency-aware feature fusion module is used to extract high-frequency and low-frequency information from features through adaptive high-pass and adaptive low-pass filters, so that low-level features have high-frequency information and high-level features have low-frequency information.

[0132] Based on the frequency-aware feature fusion module, channel attention mechanism, and cross-fusion, the frequency dynamic upsampling feature fusion module is obtained. The channel attention mechanism is used to enhance the attention of features. The cross-fusion adopts two methods, feature addition and feature channel fusion, to fuse feature information at different levels and improve local semantic information. The representation formula of the frequency dynamic upsampling feature fusion module includes:

[0133] ;

[0134] ;

[0135] ;

[0136] ;

[0137] ;

[0138] ;

[0139] ;

[0140] ;

[0141] in, Original low-resolution input; For raw, high-resolution input; Number of channels; For height; Width; For pointwise convolution; This is a low-resolution input after unifying the number of input channels through pointwise convolution; For frequency-aware feature fusion module; These are the high-level features after feature fusion; The frequency-aware feature fusion module performs frequency feature fusion on the original low-resolution input and the original high-resolution input, resulting in a low-resolution output. For splicing operations; This is the concatenated input feature map; This is a global draw pooling operation; This is a pointwise convolution operation; For activation functions; Attention features; For feature splitting operations; and These are the two attention features after feature splitting; Multiply by channel; This is an upsampling operation; To Features after upsampling and attention features The result after multiplying by the channels; For raw high-resolution input and attention characteristics The result after multiplying by the channels; for and To each and Perform element-by-element addition and then concatenate the two outputs by adding them together.

[0142] The connection improvement module 30 is used to set up convolutional block attention modules at the connection between the improved backbone network and the improved neck network;

[0143] The head network improvement module 40 is used to obtain an improved head network in the YOLOV8 model head network by using the intelligent intersection-over-union loss function as the bounding box regression loss function.

[0144] The detection model acquisition module 50 is used to acquire the improved YOLOV8 model as the detection model based on the improved backbone network, the improved neck network, the convolutional block attention module and the improved head network.

[0145] The target detection module 60 is used to perform infrared target detection using a detection model.

[0146] This application has provided a detailed description of an infrared target detection method and system based on an improved YOLO model, as provided in the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and its core ideas. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. An infrared target detection method based on an improved YOLO model, characterized in that, include: In the feature extraction module of the YOLOV8 model backbone network, the standard convolution in the bottleneck layer is replaced by windmill convolution to obtain an improved backbone network; In the neck network of the YOLOV8 model, the feature fusion module and the upsampling module are replaced by a frequency dynamic upsampling feature fusion module to obtain an improved neck network. The steps for obtaining the frequency dynamic upsampling feature fusion module include: The frequency-aware feature fusion module is used to extract high-frequency and low-frequency information from features through adaptive high-pass and adaptive low-pass filters, so that low-level features have high-frequency information and high-level features have low-frequency information. Based on the frequency-aware feature fusion module, channel attention mechanism, and cross-fusion, the frequency dynamic upsampling feature fusion module is obtained. The channel attention mechanism is used to enhance the attention of features. The cross-fusion adopts two methods, feature addition and feature channel fusion, to fuse feature information at different levels and improve local semantic information. The representation formula of the frequency dynamic upsampling feature fusion module includes: ； ； ； ； ； ； ； ； in, Original low-resolution input; For raw, high-resolution input; Number of channels; For height; Width; For pointwise convolution; This is a low-resolution input after unifying the number of input channels through pointwise convolution; For frequency-aware feature fusion module; These are the high-level features after feature fusion; The frequency-aware feature fusion module performs frequency feature fusion on the original low-resolution input and the original high-resolution input, resulting in a low-resolution output. For splicing operations; This is the concatenated input feature map; This is a global equal pooling operation; This is a pointwise convolution operation; For activation functions; Attention features; For feature splitting operations; and These are the two attention features after feature splitting; Multiply by channel; This is an upsampling operation; To Features after upsampling and attention features The result after multiplying by the channels; For raw high-resolution input and attention features The result after multiplying by the channels; for and To each and Perform element-by-element addition and then concatenate the two outputs by adding them together. A convolutional block attention module is provided at the connection between the improved backbone network and the improved neck network; In the YOLOV8 model head network, the intelligent intersection-union ratio loss function is used as the bounding box regression loss function to obtain an improved head network. The improved YOLOV8 model is obtained as the detection model based on the improved backbone network, the improved neck network, the convolutional block attention module, and the improved head network. Infrared target detection is performed using the aforementioned detection model.

2. The infrared target detection method based on the improved YOLO model according to claim 1, characterized in that, The convolutional block attention module is connected to the feature extraction module in the improved backbone network, and the convolutional block attention module is connected to the frequency dynamic upsampling feature fusion module in the improved neck network.

3. The infrared target detection method based on the improved YOLO model according to claim 1, characterized in that, The intelligent cross-union loss function, through a dynamic non-monotonic focusing mechanism, enables the detection model to pay more attention to high error rate samples and high-difficulty samples. The intelligent intersection-union ratio loss function uses a gradient allocation method to distribute gradient gain to anchor boxes of different qualities, in order to avoid the model being misled by abnormal samples.

4. The infrared target detection method based on the improved YOLO model according to claim 1, characterized in that, The characterization formula for the intelligent cross-union ratio loss function includes: ； ； ； ； ； ； ； ； in, The ratio is used to measure the degree of overlap between the ground truth bounding box and the predicted bounding box; The bounding box loss function; (x,y) and ( , These are the location information of the predicted bounding box and the ground truth bounding box, respectively; , ) represents the width and height of the smallest bounding box that merges the predicted and ground truth boxes; * represents... and The separation is used to mitigate harmful gradients; is the normalized distance between the center points of the ground truth bounding box and the predicted bounding box; m is the momentum used for delay. near The time period; t is the number of training epochs for the model; n is the batch size; This represents the average bounding box loss. This represents the actual bounding box loss. This represents the actual bounding box loss value; The degree of outlier in the anchor frame; and These are hyperparameters used to control the gradient gain r and the outlierness of the anchor frame. The relationship between them; The third-generation intelligent cross-union ratio loss function; The intelligent intersection-union ratio loss function; It is the third generation; Gradient gain; This is the first generation.

5. The infrared target detection method based on the improved YOLO model according to claim 1, characterized in that, In the feature extraction module of the improved neck network, the standard convolution in the bottleneck layer is replaced by windmill convolution.

6. An infrared target detection system based on an improved YOLO model, characterized in that, include: The backbone network improvement module is used to obtain an improved backbone network by replacing the standard convolution in the bottleneck layer with windmill convolution in the feature extraction module of the YOLOV8 model backbone network. The neck network improvement module is used to replace the feature fusion module and the upsampling module in the neck network of the YOLOV8 model by using the frequency dynamic upsampling feature fusion module to obtain an improved neck network. The steps for obtaining the frequency dynamic upsampling feature fusion module include: The frequency-aware feature fusion module is used to extract high-frequency and low-frequency information from features through adaptive high-pass and adaptive low-pass filters, so that low-level features have high-frequency information and high-level features have low-frequency information. Based on the frequency-aware feature fusion module, channel attention mechanism, and cross-fusion, the frequency dynamic upsampling feature fusion module is obtained. The channel attention mechanism is used to enhance the attention of features. The cross-fusion adopts two methods, feature addition and feature channel fusion, to fuse feature information at different levels and improve local semantic information. The representation formula of the frequency dynamic upsampling feature fusion module includes: ； ； ； ； ； ； ； ； in, Original low-resolution input; For raw, high-resolution input; Number of channels; For height; Width; For pointwise convolution; This is a low-resolution input after unifying the number of input channels through pointwise convolution; For frequency-aware feature fusion module; These are the high-level features after feature fusion; The frequency-aware feature fusion module performs frequency feature fusion on the original low-resolution input and the original high-resolution input, resulting in a low-resolution output. For splicing operations; This is the concatenated input feature map; This is a global equal pooling operation; This is a pointwise convolution operation; For activation functions; Attention features; For feature splitting operations; and These are the two attention features after feature splitting; Multiply by channel; This is an upsampling operation; To Features after upsampling and attention features The result after multiplying by the channels; For raw high-resolution input and attention features The result after multiplying by the channels; for and To each and Perform element-by-element addition and then concatenate the two outputs by adding them together. A connection improvement module is used to set a convolutional block attention module at the connection between the improved backbone network and the improved neck network; The head network improvement module is used to obtain an improved head network in the YOLOV8 model head network by using the intelligent intersection-over-union loss function as the bounding box regression loss function. The detection model acquisition module is used to acquire the improved YOLOV8 model as the detection model based on the improved backbone network, the improved neck network, the convolutional block attention module and the improved head network; The target detection module is used to perform infrared target detection using the detection model.

Images