An image processing method for infrared image detail enhancement
By constructing an infrared image processing network that combines a backbone network and a neck network, and utilizing a detail enhancement module and an adaptive feature recalibration module, the problems of insufficient feature extraction of small targets and inadequate fusion of deep and shallow features in infrared images are solved, achieving high-precision identification and positioning of small infrared targets, which is suitable for monitoring traffic vehicles and crop pests and diseases.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- CHANGCHUN UNIV OF SCI & TECH
- Filing Date
- 2026-05-29
- Publication Date
- 2026-06-26
AI Technical Summary
Existing infrared image processing technologies face challenges in small target recognition and segmentation due to the extremely small target size, low pixel ratio, lack of image features, and difficulties in target-background fusion. Furthermore, traditional algorithms have poor adaptability in complex scenarios and a high false detection rate, while deep learning models are insufficient in feature extraction and fusion, making it difficult to achieve accurate positioning.
An image processing network is constructed by combining a backbone network and a neck network. Through a detail enhancement module and an adaptive feature recalibration module, the feature extraction of small targets and the fusion of deep and shallow features are enhanced. The network is trained using a multi-class dataset and integrates a spatial attention module and an adaptive feature recalibration module to optimize feature representation.
It significantly improves the recognition and positioning accuracy of small infrared targets in complex scenarios, reduces the false detection rate, enhances image details, and is more adaptable, making it suitable for scenarios such as traffic vehicle and crop pest and disease monitoring.
Smart Images
Figure CN122289042A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of infrared image processing technology, and specifically relates to an image processing method for enhancing the details of infrared images. Background Technology
[0002] The core of infrared image processing is to capture the difference in infrared radiation between the target and the background through image acquisition equipment, and then use specialized processing algorithms to identify and locate the target in the image. In specific application scenarios, infrared image processing is needed to identify and accurately locate the infrared images of small targets flying at long distances, providing core algorithm support for infrared search and tracking systems. Infrared image processing can be applied to scenarios such as traffic vehicle image classification and crop pest and disease image monitoring. By processing and analyzing infrared images containing small targets, scene-based intelligent recognition can be achieved.
[0003] However, existing infrared image processing technologies face numerous technical bottlenecks during processing, specifically: First, in infrared images containing small targets, the target size is extremely small, the pixel ratio is extremely low, and the image features such as texture and shape in the target area are extremely scarce, resulting in insignificant differences in grayscale values between the target image and the background image, making it easy for the target and background images to merge, increasing the difficulty of target segmentation and recognition in infrared image processing; Second, when multiple small targets are densely arranged in an infrared image, the image boundaries of each target are blurred, and the pixel areas of adjacent targets are prone to overlap, resulting in a high false detection rate and false negative rate in the infrared image processing algorithm during target recognition, making it difficult to achieve accurate target differentiation and localization; Third, traditional infrared image processing algorithms rely on manually designed image feature extraction rules, which have poor adaptability to infrared images in different scenarios and weak generalization ability, failing to meet the processing needs of complex scenarios.
[0004] Current mainstream deep learning-based infrared image processing models still suffer from technical defects such as insufficient extraction of detailed features and inadequate fusion of deep and shallow features. Traditional image feature extraction modules struggle to capture the fine pixel information of small targets in infrared images, and conventional feature pyramid networks have limited ability to calibrate and fuse deep and shallow features in infrared images. As a result, in infrared images with complex noise backgrounds, it is difficult to balance the accuracy of target recognition with the real-time processing, and thus cannot meet the needs of infrared image processing and target recognition in practical applications.
[0005] Therefore, in response to the technical pain points of infrared image processing, there is an urgent need for an infrared image processing method that can enhance the extraction of detailed features of small targets in infrared images and optimize the fusion effect of deep and shallow features in infrared images. Summary of the Invention
[0006] To address the technical problem of the urgent need for an infrared image processing method that can enhance the extraction of small target detail features in infrared images and optimize the fusion effect of deep and shallow features in infrared images, this invention provides an image processing method for enhancing infrared image details.
[0007] The method includes the following steps: S1. Construction of multi-category datasets: Select publicly available infrared datasets as the basic data source and divide the datasets; S2. Image Processing Network Construction: A combination of backbone and neck network is used to enhance the details of infrared images. A backbone network is used to extract infrared image features at four scales. The backbone network consists of five layers: a CBS module, two lightweight convolutional modules, and two detail enhancement modules. A spatial attention module is added after the second detail enhancement module. A neck network is used to fuse the extracted infrared image features at four scales to obtain four infrared images with enhanced details at different scales. The neck network adopts an FPN+PAN structure, which includes an upsampling part and a downsampling part. The upsampling part contains three adaptive feature recalibration modules, and the downsampling part contains three adaptive feature recalibration modules. The infrared images with enhanced details at four different scales are output by the three adaptive feature recalibration modules of the downsampling part and the last adaptive feature recalibration module of the upsampling part, respectively. S3. Image Processing Network Training: Configure the training environment based on the PyTorch framework, train the image processing network using the dataset divided in step S1, and use the validated network to perform detail enhancement processing on the infrared images.
[0008] Furthermore, when partitioning the dataset, it is divided into a training set, a validation set, and a test set.
[0009] Furthermore, the CBS module sequentially includes a convolutional layer, a batch normalization layer, and a SiLU activation layer. The CBS module is used to perform initial feature extraction and nonlinear transformation on the input infrared image.
[0010] Furthermore, a lightweight convolution module is used to improve the efficiency of basic feature extraction, which is implemented using existing depthwise separable convolution.
[0011] Furthermore, the detail enhancement module is a self-designed module used to enhance the detail features of small targets. Its structure is as follows: it adopts a dual-branch structure. The first branch adopts a structure in parallel with a multi-layer convolutional feature extraction module and a standard convolution, and then adds three convolutional layers. The second branch adopts a multi-layer convolutional feature extraction module. The outputs of the first branch and the second branch are superimposed, and then the output features are obtained through a convolutional layer.
[0012] Furthermore, the multi-layer convolutional feature extraction module includes parallel-connected central difference convolution, angular difference convolution, horizontal difference convolution, and vertical difference convolution.
[0013] Furthermore, the adaptive feature recalibration module is independently designed to achieve multi-scale feature fusion and calibration, and suppress background interference. Its specific structure consists of three recalibration attention units connected in series. The first and second recalibration attention units receive the input deep and shallow features, and the third recalibration attention unit receives the features input from the first and second recalibration attention units.
[0014] Furthermore, the recalibration attention unit is a self-designed component used for adaptive extraction and mutual representation of input features. The specific data processing flow involves using two feature vectors... and As input, , , ,in, and Represents a linear mapping. To recalibrate the output of the attention unit, This indicates element-wise multiplication.
[0015] The beneficial effects of the method described in this invention are as follows: To address the issues of small infrared targets being small in size, lacking features, and easily blending with the background, as well as the insufficient detail extraction and feature fusion of existing feature map construction methods, this invention constructs a novel image processing network. The core integrates a detail enhancement module and an adaptive feature recalibration module: the detail enhancement module strengthens the extraction of detailed features from small targets through multi-branch differential convolution and reparameterization techniques; the adaptive feature recalibration module consists of cascaded recalibrated attention units, adaptively fusing deep and shallow features to optimize feature representation. The detail enhancement module and the adaptive feature recalibration module provide a new feature enhancement and fusion scheme for infrared small target detection, which can be transferred to other deep learning models and has broad technical reuse value. Attached Figure Description
[0016] Figure 1 This is a schematic diagram of the image processing network described in an embodiment of the present invention; Figure 2 This is a schematic diagram of the detail enhancement module structure in an embodiment of the present invention; Figure 3 This is a schematic diagram of the adaptive feature recalibration module structure in an embodiment of the present invention; Figure 4 This is a schematic diagram of the recalibration attention unit structure in an embodiment of the present invention; Figure 5This is a comparison diagram of infrared image processing in an embodiment of the present invention. Detailed Implementation
[0017] The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the protection scope of the present invention.
[0018] Example 1 This embodiment provides an image processing method for enhancing infrared image details. By designing a dedicated detail enhancement module and an adaptive feature recalibration module, it strengthens the feature representation of small targets and optimizes multi-scale feature fusion. While ensuring detection speed, it significantly improves the recognition and localization accuracy of small infrared targets in complex scenes. Specifically: S1. Construction of multi-class datasets: Select publicly available infrared datasets as the basic data source, and perform annotation processing and dataset division; S2. Image Processing Network Construction: A combination of backbone and neck network is used to enhance the details of infrared images. A backbone network is used to extract infrared image features at four scales. The backbone network consists of five layers: a CBS module, two lightweight convolutional modules, and two detail enhancement modules. A spatial attention module is added after the second detail enhancement module. A neck network is used to fuse the extracted infrared image features at four scales to obtain four infrared images with enhanced details at different scales. The neck network adopts an FPN+PAN structure, which includes an upsampling part and a downsampling part. The upsampling part contains three adaptive feature recalibration modules, and the downsampling part contains three adaptive feature recalibration modules. The infrared images with enhanced details at four different scales are output by the three adaptive feature recalibration modules of the downsampling part and the last adaptive feature recalibration module of the upsampling part, respectively. S3. Image Processing Network Training: Configure the training environment based on the PyTorch framework, train the image processing network using the dataset divided in step S1, and use the validated network to perform detail enhancement processing on the infrared images.
[0019] Example 2 This embodiment further defines embodiment 1 and provides further explanation of step S1.
[0020] In this embodiment, two publicly available infrared datasets, BIT-SIRST and FLIR-ADAS-v2, are selected as the basic data sources: Multi-class annotation was performed on 10,478 images in the BIT-SIRST dataset, clearly classifying four target categories: cars, pedestrians, drones, and ships, while also annotating blank background areas; The original annotations of 10 target categories, including vehicles, pedestrians, and bicycles, in the FLIR-ADAS-v2 dataset were retained, and 14,452 valid thermal imaging images were selected. Data was divided according to a reasonable ratio of "training set: validation set: test set" (the BIT-SIRST dataset was divided according to the ratio of conventional remote sensing data, and the FLIR-ADAS-v2 dataset selected 7334 images as the training set, 1048 images as the validation set, and 2096 images as the test set) to ensure a balanced distribution of targets of each category. The annotation process employs a multi-person, multi-verification mechanism, with professionals calibrating the target bounding box and category labels to ensure annotation accuracy.
[0021] Example 3 This embodiment further defines embodiment 1 and provides further explanation of step S2.
[0022] like Figure 1 The diagram shows the structure of an image processing network (named RADFRNet), which includes a backbone network and a neck network.
[0023] A backbone network is used to extract infrared image features at four scales. The backbone network consists of five layers: a CBS module, two lightweight convolutional modules, and two detail enhancement modules. A spatial attention module is added after the second detail enhancement module. A neck network is used to fuse features extracted from infrared images at four scales, resulting in four infrared images with enhanced details at different scales. The neck network employs an FPN+PAN structure, comprising an upsampling part and a downsampling part. The upsampling part sequentially includes three adaptive feature recalibration modules, and the downsampling part sequentially includes three adaptive feature recalibration modules. The four infrared images with enhanced details at different scales are output by the three adaptive feature recalibration modules in the downsampling part and the last adaptive feature recalibration module in the upsampling part, respectively. The four infrared images with enhanced details at different scales correspond to... Figure 1 Among P1, P2, P3, and P4, their detail display capabilities increase sequentially.
[0024] Backbone network enables detailed feature enhancement of small objectives: The CBS module consists of a convolutional layer, a batch normalization layer, and a SiLU activation layer. The CBS module is used to perform initial feature extraction and nonlinear transformation on the input infrared image.
[0025] The lightweight convolution module is used to improve the efficiency of basic feature extraction. It is implemented using existing depthwise separable convolution.
[0026] The detail enhancement module (named C3DEConv module) is a self-designed module used for enhancing the detail features of small targets. Its structure is as follows: Figure 2 As shown: A dual-branch structure is adopted. The first branch uses a multi-layer convolutional feature extraction module (named DEConv module) in parallel with the standard convolution, and then adds three convolutional layers. The second branch uses a multi-layer convolutional feature extraction module. The outputs of the first branch and the second branch are superimposed, and then the output features are obtained through a convolutional layer.
[0027] With the goal of efficient feature extraction, a detail enhancement module is designed as the core feature extraction unit. The detail enhancement module integrates a multi-convolutional layer feature extraction module, which includes four parallel deployments: central difference convolution (CDC), angular difference convolution (ADC), horizontal difference convolution (HDC), and vertical difference convolution (VDC), which extract gradient-level information and intensity-level information respectively. The first branch uses a structure where a multi-layer convolutional feature extraction module is connected in parallel with a standard convolution. Through reparameterization, the five types of convolution are equivalently converted to a standard convolution. Its output feature map can be represented as:
[0028] in, Indicates input features, Indicates output features, These are five types of convolution kernels. For convolution operations, This is the equivalent kernel after conversion.
[0029] Neck network design: An FPN+PAN (Feature Pyramid Network + Pyramid Attention Network) structure is adopted, embedding an adaptive feature recalibration module (named AFRE module). The adaptive feature recalibration module is self-designed, and its structure is as follows: Figure 3 As shown, it is used to achieve multi-scale feature fusion and calibration to suppress background interference. Its specific structure is: it consists of three recalibrated attention units (named RAU) connected in series. The first and second recalibrated attention units receive the input deep features and shallow features, and the third recalibrated attention unit receives the features input from the first and second recalibrated attention units.
[0030] Combination Figure 1 and Figure 3 It can be seen that the initial adaptive feature recalibration module takes the deep semantic features (Lb) and shallow boundary features (Ls) of the backbone network as input, and the subsequent modules take the output features of the previous module and the shallow features of the next layer as input; Recalibrate attention unit structure such as Figure 4 As shown, adaptive extraction and mutual representation of input features can be represented as follows: using two-way features and As input, , , ,in, and Represents a linear mapping. To recalibrate the output of the attention unit, This indicates element-wise multiplication.
[0031] The outputs of the three recalibrated attention units are fused through 3×3 convolution, batch normalization, and ReLU activation layers to output calibrated multi-scale features, thus optimizing the feature fusion effect of the feature pyramid network.
[0032] This method constructs a novel image processing network, integrating a detail enhancement module and an adaptive feature recalibration module at its core. The detail enhancement module leverages multi-branch differential convolution and reparameterization techniques to enhance the extraction of detailed features from small targets. The adaptive feature recalibration module consists of cascaded recalibrated attention units, adaptively fusing features from both deep and shallow layers to optimize feature representation. These two approaches provide a novel feature enhancement and fusion scheme for infrared small target detection, are transferable to other deep learning models, and possess broad technical reuse value.
[0033] Example 4 This embodiment further defines embodiments 1 and 2. Step 3 is further explained.
[0034] When training the image processing network, four detection heads are connected at positions P1, P2, P3 and P4 respectively. The detection heads use existing three-scale decoupled detection heads, which output target class probabilities and bounding box coordinates based on the low, medium and high resolution feature maps output by the backbone network.
[0035] Step 3 involves the following specific operations: Hardware configuration: NVIDIA GeForce RTX 4060 GPU (15.9GB VRAM), Windows 11 operating system; Hyperparameter settings: Number of target classes 4 (BIT-SIRST dataset) / 10 (FLIR-ADAS-v2 dataset), 100 training epochs, batch size 8, input image size 640×640, number of worker threads 4, SGD optimizer (initial learning rate 0.01, momentum 0.9, weight decay 0.0005). Data augmentation: Mosaic (image stitching) and Mixup (image blending) techniques are used during training to enhance the model's adaptability to changes in target scale and background diversity; Training process: The learning rate decreases linearly for the first 50 rounds, and a cosine annealing strategy is used to adjust the learning rate for the next 50 rounds; after each training round, the mAP@0.5 and F1 score are evaluated on the validation set, and the weights of the model with the best performance are saved.
[0036] Validation method: Five-fold cross-validation was used, with four subsets selected for training and one subset for testing each time, repeated five times to comprehensively evaluate model performance.
[0037] After the model training is successful, the detector head is removed, and the infrared image to be enhanced is input to obtain infrared images with enhanced details at four scales. In one embodiment, the feature map size and target size range of the four images are as follows: P1: 160×160, <8×8 pixels (suitable for small targets); P2: 80×80, 8-16 pixels (suitable for small objects); P3: 40×40, 16-32 pixels (suitable for medium targets); P4: 20×20, >32 pixels (suitable for large targets).
[0038] In the trained image processing network, the image is processed by the CBS module and nonlinear feature transformation is performed through the SiLU activation function; the feature map is gradually extracted by the lightweight convolution module and detail enhancement module of the backbone network to enhance the detail features of small targets; the neck network realizes multi-scale feature fusion and calibration through the adaptive feature recalibration module to suppress background noise interference; finally, infrared detail-enhanced images at different scales are output.
[0039] Take the same infrared image and perform detail enhancement processing using other methods and the method described in this invention, such as... Figure 5 As shown, Figures (a)-(d) are enhanced images obtained by other existing infrared image processing methods, and Figure (e) is an enhanced image processed by the method described in this invention. It can be observed that Figure (e) obtained by the method described in this invention has a more obvious enhancement of image details.
Claims
1. An image processing method for enhancing details in infrared images, characterized in that, The method includes the following steps: S1. Construction of multi-category datasets: Select publicly available infrared datasets as the basic data source and divide the datasets; S2. Image Processing Network Construction: A combination of backbone and neck network is used to enhance the details of infrared images. A backbone network is used to extract infrared image features at four scales. The backbone network consists of five layers: a CBS module, two lightweight convolutional modules, and two detail enhancement modules. A spatial attention module is added after the second detail enhancement module. The neck network is used to fuse the extracted infrared image features at four scales to obtain infrared images with enhanced details at four different scales. The neck network adopts an FPN+PAN structure, which includes an upsampling part and a downsampling part. The upsampling part contains three adaptive feature recalibration modules in sequence, and the downsampling part contains three adaptive feature recalibration modules in sequence. The four infrared images with enhanced details at different scales are output by the three adaptive feature recalibration modules in the downsampling part and the last adaptive feature recalibration module in the upsampling part, respectively. S3. Image Processing Network Training: Configure the training environment based on the PyTorch framework, train the image processing network using the dataset divided in step S1, and use the validated network to perform detail enhancement processing on the infrared images.
2. The image processing method for enhancing infrared image details according to claim 1, characterized in that, When splitting the dataset, it is divided into a training set, a validation set, and a test set.
3. The image processing method for enhancing infrared image details according to claim 2, characterized in that, The CBS module consists of a convolutional layer, a batch normalization layer, and a SiLU activation layer. The CBS module is used to perform initial feature extraction and nonlinear transformation on the input infrared image.
4. The image processing method for enhancing infrared image details according to claim 3, characterized in that, The lightweight convolution module is used to improve the efficiency of basic feature extraction, and it is implemented using existing depthwise separable convolution.
5. The image processing method for enhancing infrared image details according to claim 4, characterized in that, The detail enhancement module is a self-designed module used to enhance the detail features of small targets. Its structure is as follows: it adopts a dual-branch structure. The first branch uses a multi-layer convolutional feature extraction module connected in parallel with a standard convolution, and then adds three convolutional layers. The second branch uses a multi-layer convolutional feature extraction module. The outputs of the first and second branches are superimposed and then passed through a convolutional layer to obtain the output features.
6. The image processing method for enhancing infrared image details according to claim 5, characterized in that, The multi-layer convolutional feature extraction module includes parallel-connected central difference convolution, angular difference convolution, horizontal difference convolution, and vertical difference convolution.
7. The image processing method for enhancing infrared image details according to claim 6, characterized in that, The adaptive feature recalibration module is independently designed to achieve multi-scale feature fusion and calibration and suppress background interference. Its specific structure consists of three recalibration attention units connected in series. The first and second recalibration attention units receive the input deep and shallow features, and the third recalibration attention unit receives the features input from the first and second recalibration attention units.
8. The image processing method for enhancing infrared image details according to claim 7, characterized in that, The recalibration attention unit is a self-designed component used for adaptive extraction and mutual representation of input features. The specific data processing flow involves using two feature vectors... and As input, , , ,in, and Represents a linear mapping. To recalibrate the output of the attention unit, This indicates element-wise multiplication.