Lightweight real-time detection method for infrared dim small target
By constructing a lightweight Mobilenetv3-Unet network and optimizing the convolution kernel and loss function, the problem of low detection efficiency of infrared weak target detection in complex scenes is solved, and efficient detection is achieved on resource-constrained devices.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- GUANGZHOU INSTITUTE OF TECHNOLOY XIDIAN UNIVERSITY
- Filing Date
- 2022-07-28
- Publication Date
- 2026-05-12
AI Technical Summary
Existing infrared weak target detection algorithms have low detection efficiency in complex scenarios and are difficult to deploy on resource-constrained edge devices. The large computational load of the network model leads to low detection accuracy.
A lightweight network based on Mobilenetv3-Unet is constructed. Through a feature extraction backbone network and a feature aggregation network, the convolution kernel size and downsampling times are optimized. The model is trained by combining BCE and DICE loss functions to achieve real-time detection of infrared weak targets.
提高了红外弱小目标的检测精度和适应性，降低了计算资源需求，使得检测算法能够在资源受限的边缘设备上运行。
Smart Images
Figure CN115223026B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a real-time target detection method, specifically a lightweight real-time infrared target detection method, belonging to the field of image recognition technology. Background Technology
[0002] Infrared weak target detection is a challenging and crucial research area in target detection, playing a vital role in national defense and military command. Traditional infrared weak target detection methods mainly include: searching for regions of interest based on target characteristics, threshold segmentation utilizing the physical properties of infrared imaging, and background prediction modeling. Deep learning-based infrared weak target detection algorithms mainly include the YOLO series based on target detection methods and semantic segmentation-based methods. However, these algorithms suffer from low detection efficiency, difficulty in handling complex and varied scenarios, or large network models that are difficult to deploy on resource-constrained edge devices.
[0003] In recent years, with the development of deep learning, deep learning-based object detection algorithms have made significant progress. Deep learning extracts abstract features from images through backpropagation algorithms via nonlinear changes in network layers, thereby accurately identifying targets. However, due to the low resolution and contrast of infrared images, as well as the lack of texture for small targets, object detection algorithms designed based on public datasets are not suitable for detecting small infrared targets. In the past few years, in order to improve the detection performance of networks, the depth and width of networks have been continuously increasing, resulting in increasingly large computational demands. Training a single network can sometimes require hundreds of servers for several days. Network training and deployment have become increasingly difficult, making it particularly important to design dedicated networks for specific application scenarios.
[0004] With the rapid development of deep learning methods, breakthroughs have been achieved in the field of target detection, bringing new methods and ideas to the research of infrared weak target detection. Patent application CN114549959A, entitled "Real-time Detection Method and System for Infrared Weak Targets Based on Target Detection Model," discloses an infrared small target detection method based on YOLOv4-tiny. This method solves the problems of low detection accuracy and difficulty in real-time detection in existing infrared weak target detection methods. However, the method still faces certain difficulties in deployment on resource-constrained edge devices.
[0005] Currently, most networks focus solely on network performance, continuously increasing network complexity to improve the detection of infrared weak targets. However, due to the small number of pixels and lack of texture features of infrared weak targets, overly deep and complex networks can actually cause the loss of these features. Therefore, it is of great significance to design a dedicated lightweight network specifically for the characteristics of infrared weak targets, enabling it to run in real time on resource-constrained devices.
[0006] In view of this, the present invention is proposed to help solve the above problems. Summary of the Invention
[0007] The purpose of this invention is to provide a lightweight real-time infrared weak target detection method to solve the above problems, which improves the detection capability of infrared weak targets in complex scenes and can use less computing resources.
[0008] This invention achieves the above objective through the following technical solution: a lightweight real-time detection method for weak infrared targets, the detection method comprising the following steps:
[0009] Step 1: Construct an infrared weak target dataset: After obtaining infrared weak targets, select and label them;
[0010] Step 2: Construct an infrared weak target detection network based on Mobilenetv3-Unet;
[0011] Step 3: Train the Mobilenetv3-Unet model using the infrared weak target dataset from Step 1 to obtain the trained Mobilenetv3-Unet model;
[0012] Step 4: Use the trained Mobilenetv3-Unet model to perform infrared weak target detection on the input image.
[0013] Furthermore, the Mobilenetv3-Unet model mentioned in step two includes a feature extraction backbone network and a feature aggregation network. The feature extraction backbone network is an optimized Mobilenetv3 model. The feature extraction backbone network extracts features from the input image and outputs them to the feature aggregation network for feature enhancement. The feature aggregation network outputs the detection results.
[0014] Furthermore, the optimized Mobilenetv3 model is the feature extraction part of Mobilenetv3-small, and the number of downsampling operations and the size of the convolution kernel are optimized to cope with the detection of small infrared targets.
[0015] Furthermore, the aforementioned feature extraction backbone network model is specifically as follows:
[0016] The input image is processed through a convolutional layer to obtain a first extracted feature. The first extracted feature is processed through block1 to obtain a second extracted feature. The second extracted feature is processed through block2 to obtain a third extracted feature. The third extracted feature is processed through block3 to obtain a fourth extracted feature. The fourth extracted feature is processed through block4 to obtain a fifth extracted feature.
[0017] Furthermore, the block1, block2, block3, and block4 modules are all composed of a series of stacked bneck modules. The bneck modules mainly implement channel-separable convolution, SE channel attention mechanism, and residual connections.
[0018] Furthermore, the adjustment of the convolution kernel size specifically involves:
[0019] The convHead layer consists of a convolutional layer with a kernel size of 3x3 and a stride of 1, a batch normalization (BN) layer, and an h-swish activation function. Block 1 consists of one BNeck module with a kernel size of 3x3 and a stride of 2; Block 2 consists of two BNeck modules with kernel sizes of 3x3 and 3x3 respectively and strides of 2 and 1 respectively; Block 3 consists of five BNeck modules with kernel sizes of 5x5, 5x5, 5x5, 5x5, and 5x5 respectively and strides of 2, 1, 1, 1, and 1 respectively; Block 4 consists of three BNeck modules with kernel sizes of 7x7, 7x7, and 7x7 respectively and strides of 2, 1, and 1.
[0020] Furthermore, the bneck module is a module with controllable parameters, including the convolution kernel size, the type of activation function, and whether to use an attention mechanism. Global average pooling is applied to the feature map to obtain a 1x1xC result. Each channel in the feature map describes a part of the features, and the result is global. To obtain the importance score of each feature map, two more fully connected layers are needed, and the final result is also 1x1xC.
[0021] Furthermore, the feature aggregation network consists of four feature aggregation modules, specifically:
[0022] The fourth and fifth extracted features of the feature extraction backbone network model are used as inputs to FA4, the third extracted features and the output of FA4 are used as inputs to FA3, the second extracted features and the output of FA3 are used as inputs to FA2, and the first extracted features and the output of FA2 are used as outputs to FA1.
[0023] Furthermore, the input to the feature aggregation module consists of shallow features and deep features. Shallow features refer to the outputs of each stage of the backbone network, while deep features refer to the output of the last feature aggregation module. We propose a simple yet effective aggregation module that uses, as shown in... Figure 2 The attention mechanism shown spontaneously adjusts the weights of the two inputs.
[0024] Furthermore, in step three, the BCE loss function and the DICE loss function are used for training. The BCE loss function is:
[0025]
[0026] Where y is the actual base value and ̂y is the predicted value.
[0027] The formula for calculating DICE is as follows:
[0028]
[0029] Where X represents the actual result and Y represents the predicted result.
[0030] The trained Mobilenetv3-Unet model was used to detect weak infrared targets, and the detection results were obtained:
[0031] Number of positive samples The number of incorrect positive samples and the number of incorrect negative samples ;
[0032] 1) Accuracy: Represents the proportion of positive and negative samples that are correctly classified.
[0033] ;
[0034] 2) Precision: Represents the proportion of samples that were originally classified as positive among all samples that were classified as positive.
[0035] ;
[0036] 3) Recall: The proportion of samples that were originally positive out of the original positive samples.
[0037] ;
[0038] 4) Overall evaluation metric (F-Measure): This is the harmonic mean of precision and recall.
[0039] .
[0040] The technical effects and advantages of this invention are as follows:
[0041] This invention utilizes the Mobilenetv3-Unet model for detecting weak infrared targets. It employs Mobilenetv3 as the framework for feature extraction and improves the feature extractor to address the characteristics of weak infrared targets. This significantly reduces network computational parameters and workload while enhancing the detection performance of weak infrared targets. The network is better suited to detecting weak targets, improving detection accuracy and reducing network complexity. Compared to existing technologies, this invention not only improves the detection capability of weak infrared targets in complex scenes but also uses fewer computational resources, allowing the weak target detection algorithm to be easily deployed on resource-constrained edge devices. Attached Figure Description
[0042] Figure 1 This is a flowchart of the present invention;
[0043] Figure 2 This is the basic network module designed for this invention;
[0044] Figure 3 This is a diagram of the network structure designed for this invention;
[0045] Figure 4 The images show the results of this invention in detecting small targets in different scenarios. Detailed Implementation
[0046] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0047] Please see Figure 1-4 As shown, a lightweight infrared real-time target detection method includes the following steps:
[0048] Step 1: Construct an infrared dataset of small, weak targets;
[0049] Step 2: Construct an infrared weak target detection network based on Mobilenetv3-Unet;
[0050] Step 3: Train the Mobilenetv3-Unet model using the infrared weak target dataset from Step 1 to obtain the trained Mobilenetv3-Unet model;
[0051] Step 4: Use the trained Mobilenetv3-Unet model to perform infrared weak target detection on the input image.
[0052] The Mobilenetv3-Unet model mentioned in step two includes a feature extraction backbone network and a feature aggregation network. The feature extraction backbone network is an optimized Mobilenetv3 model. The feature extraction backbone network extracts features from the input image and outputs them to the feature aggregation network for feature enhancement. The feature aggregation network outputs the detection results.
[0053] The optimized Mobilenetv3 model is the feature extraction part of Mobilenetv3-small, and the number of downsampling operations and the size of the convolution kernel are optimized to cope with the detection of small infrared targets.
[0054] The feature extraction backbone network model is specifically as follows:
[0055] The input image is processed through a convolutional layer to obtain a first extracted feature. The first extracted feature is processed through block1 to obtain a second extracted feature. The second extracted feature is processed through block2 to obtain a third extracted feature. The third extracted feature is processed through block3 to obtain a fourth extracted feature. The fourth extracted feature is processed through block4 to obtain a fifth extracted feature.
[0056] The block1, block2, block3 and block4 modules are all composed of a series of stacked bneck modules. The bneck modules mainly implement channel separable convolution, SE channel attention mechanism and residual connection.
[0057] The feature aggregation network consists of four feature aggregation modules, specifically:
[0058] The fourth and fifth extracted features of the feature extraction backbone network model are used as inputs to FA4, the third extracted features and the output of FA4 are used as inputs to FA3, the second extracted features and the output of FA3 are used as inputs to FA2, and the first extracted features and the output of FA2 are used as outputs to FA1.
[0059] Based on the above technical concept, the technical solution adopted to achieve the purpose of this invention includes the following steps:
[0060] Step 1: Construct an infrared dataset of small, weak targets;
[0061] Step 2: Construct an infrared weak target detection network based on Mobilenetv3-Unet;
[0062] Step 3: Train the Mobilenetv3-Unet model using the infrared weak target dataset from Step 1 to obtain the trained Mobilenetv3-Unet model;
[0063] Step 4: Use the trained Mobilenetv3-Unet model to perform infrared weak target detection on the input image.
[0064] The Mobilenetv3-Unet model described in step two includes a feature extraction backbone network and a feature aggregation network. The feature extraction backbone network is an optimized Mobilenetv3 model. The feature extraction backbone network extracts features from the input image and outputs them to the feature aggregation network for feature enhancement. The feature aggregation network outputs the detection results.
[0065] Preferably, the optimized Mobilenetv3 model is the feature extraction part of Mobilenetv3-small, and the number of downsampling operations and the size of the convolution kernel are optimized to cope with the detection of small infrared targets.
[0066] Preferably, the feature extraction backbone network model is as follows:
[0067] The input image is processed through a convolutional layer to obtain a first extracted feature. The first extracted feature is processed through block1 to obtain a second extracted feature. The second extracted feature is processed through block2 to obtain a third extracted feature. The third extracted feature is processed through block3 to obtain a fourth extracted feature. The fourth extracted feature is processed through block4 to obtain a fifth extracted feature.
[0068] Preferably, the block1, block2, block3 and block4 modules are all composed of a series of stacked bneck modules. The bneck modules mainly implement channel separable convolution, SE channel attention mechanism and residual connection.
[0069] like Figure 3As shown, the convHead passes through a convolutional layer with a kernel size of 3x3 and a stride of 1, a bneck layer, and an h-swish activation function. Block 1 consists of one bneck module with a kernel size of 3x3 and a stride of 2; Block 2 consists of two bneck modules with kernel sizes of 3x3 and 3x3, and strides of 2 and 1 respectively; Block 3 consists of five bneck modules with kernel sizes of 5x5, 5x5, 5x5, 5x5, and 5x5, and strides of 2, 1, 1, 1, and 1 respectively; Block 4 consists of three bneck modules with kernel sizes of 7x7, 7x7, and 7x7, and strides of 2, 1, and 1 respectively.
[0070] Specifically, the bneck module is a module with controllable parameters, including the kernel size, the type of activation function, and whether to use an attention mechanism.
[0071] The diagram of the bneck module is shown below. Figure 2 As shown, first, global average pooling is applied to the feature maps, resulting in a 1x1xC result. Each channel in the feature map describes a portion of the features, and the result is essentially global. Next, to obtain the importance score for each feature map, two more fully connected layers are needed, and the final result is also 1x1xC.
[0072]
[0073] The feature aggregation network consists of four feature aggregation modules, specifically:
[0074] The fourth and fifth extracted features of the feature extraction backbone network model are used as inputs to FA4, the third extracted features and the output of FA4 are used as inputs to FA3, the second extracted features and the output of FA3 are used as inputs to FA2, and the first extracted features and the output of FA2 are used as outputs to FA1.
[0075] Specifically, the input to the feature aggregation module consists of shallow features and deep features. Shallow features refer to the outputs of each stage of the backbone network, while deep features refer to the output of the last feature aggregation module. We propose a simple yet effective aggregation module that uses, for example... Figure 2 The attention mechanism shown spontaneously adjusts the weights of the two inputs.
[0076] Preferably, in step three, the BCE loss function and the DICE loss function are used for training. The BCE loss function is:
[0077]
[0078] Where y is the actual base value and ̂y is the predicted value.
[0079] The formula for calculating DICE is as follows:
[0080]
[0081] Where X represents the actual result and Y represents the predicted result.
[0082] The trained Mobilenetv3-Unet model was used to detect weak infrared targets, and the detection results were obtained:
[0083] Number of positive samples The number of incorrect positive samples and the number of incorrect negative samples ;
[0084] 1) Accuracy: Represents the proportion of positive and negative samples that are correctly classified.
[0085] ;
[0086] 2) Precision: Represents the proportion of samples that were originally classified as positive among all samples that were classified as positive.
[0087] ;
[0088] 3) Recall: The proportion of samples that were originally positive out of the original positive samples.
[0089] ;
[0090] 4) Overall evaluation metric (F-Measure): This is the harmonic mean of precision and recall.
[0091] .
[0092] The technical effects of the present invention will be further explained below with reference to simulation experiments:
[0093] 1. Simulation conditions and content:
[0094] The simulation experiment of this invention was implemented based on the PyTorch framework in a hardware environment with a GPU GeForce GTX 2080 Ti and 32G RAM and a software environment of Ubuntu 18.04.
[0095] Simulation Experiment: Following the infrared weak target dataset constructed according to this invention, the optimized network was trained on the training set for 50 iterations. The test set was then input into the trained infrared weak target detection data for detection, as shown in the figure.
[0096] 2. Simulation Result Analysis:
[0097] The infrared weak target detection results obtained by this invention have significant advantages compared with other infrared weak target detection algorithms. This invention uses the F1 parameter as the main evaluation index for infrared weak target detection, achieving a score of 0.6706, while existing technologies only achieve 0.6502. The detection results show that this invention can achieve excellent detection results for infrared weak targets in various complex scenes.
[0098] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.
[0099] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.
Claims
1. A lightweight real-time infrared target detection method, characterized in that: The detection method includes the following steps: Step 1: Construct an infrared weak target dataset: After obtaining infrared weak targets, select and label them; Step 2: Construct an infrared weak target detection network based on Mobilenetv3-Unet. The Mobilenetv3-Unet model in Step 2 includes a feature extraction backbone network and a feature aggregation network. The feature extraction backbone network is an optimized Mobilenetv3 model. The feature extraction backbone network extracts features from the input image and outputs them to the feature aggregation network for feature enhancement. The feature aggregation network outputs the detection results. The optimized Mobilenetv3 model is the feature extraction part of Mobilenetv3-small, and the downsampling times and convolution kernel sizes are optimized to handle the detection of infrared weak targets. Specifically, the convolution kernel size adjustment involves: the convHead passing through a convolutional layer with a kernel size of 3x3 and a stride of 1, a bn layer, and an h-swish activation function, and then... Block 1 consists of one bneck module with a 3x3 kernel size and a stride of 2; block 2 consists of two bneck modules with kernel sizes of 3x3 and 3x3 respectively and strides of 2 and 1 respectively; block 3 consists of five bneck modules with kernel sizes of 5x5, 5x5, 5x5, 5x5, and 5x5 respectively and strides of 2, 1, 1, 1, and 1 respectively; block 4 consists of three bneck modules with kernel sizes of 7x7, 7x7, and 7x7 respectively and strides of 2, 1, and 1. The feature aggregation network consists of four feature aggregation modules, specifically: the fourth and fifth extracted features of the feature extraction backbone network model are used as inputs to FA4, the third extracted feature and the output of FA4 are used as inputs to FA3, the second extracted feature and the output of FA3 are used as inputs to FA2, and the first extracted feature and the output of FA2 are used as outputs to FA1. Step 3: Train the Mobilenetv3-Unet model using the infrared weak target dataset from Step 1 to obtain the trained Mobilenetv3-Unet model; Step 4: Use the trained Mobilenetv3-Unet model to perform infrared weak target detection on the input image.
2. The lightweight infrared small target real-time detection method according to claim 1, characterized in that: The feature extraction backbone network model is specifically as follows: The input image is processed through a convolutional layer to obtain a first extracted feature. The first extracted feature is processed through block1 to obtain a second extracted feature. The second extracted feature is processed through block2 to obtain a third extracted feature. The third extracted feature is processed through block3 to obtain a fourth extracted feature. The fourth extracted feature is processed through block4 to obtain a fifth extracted feature.
3. The lightweight infrared small target real-time detection method according to claim 2, characterized in that: The block1, block2, block3 and block4 modules are all composed of a series of stacked bneck modules. The bneck modules implement channel-separable convolution, SE channel attention mechanism and residual connection.
4. The lightweight infrared small target real-time detection method according to claim 3, characterized in that: The bneck module is a module with controllable parameters, including the kernel size, activation function type, and whether to use attention mechanism. Global average pooling is applied to the feature map to obtain a 1x1xC result. Each channel in the feature map describes a part of the features, and after the operation, it is equivalent to a global result. Next, in order to obtain the importance score of each feature map, two more fully connected layers are needed, and the final result is also 1x1xC.
5. The lightweight infrared small target real-time detection method according to claim 1, characterized in that... The input to the feature aggregation module consists of shallow features and deep features. Shallow features refer to the outputs of each stage of the backbone network, while deep features refer to the output of the last feature aggregation module. This module uses an attention mechanism to spontaneously adjust the weights of the two inputs.
6. The lightweight infrared small target real-time detection method according to claim 1, characterized in that: In step three, the BCE loss function and the DICE loss function are used for training. The BCE loss function is: Where y is the actual base value and ̂y is the predicted value; The formula for calculating DICE is as follows: Where X represents the actual result and Y represents the predicted result; The trained Mobilenetv3-Unet model was used to detect weak infrared targets, and the detection results were obtained: Number of positive samples The number of incorrect positive samples and the number of incorrect negative samples ; 1) Accuracy: Represents the proportion of positive and negative samples that are correctly classified. ； 2) Precision: Represents the proportion of samples that were originally classified as positive among all samples that were classified as positive. ； 3) Recall: The proportion of samples that were originally positive out of the original positive samples. ； 4) Overall evaluation metric (F-Measure): This is the harmonic mean of precision and recall. 。