Infrared image super-resolution method based on global context channel attention

Abstract

The application discloses an infrared image super-resolution method based on global context channel attention, and comprises the following steps: step one, acquiring a training data set; step two, expanding the training data set; step three, constructing a global context channel attention network, which is based on the RCAN network architecture, and is improved on the residual group structure in the deep feature extraction stage; in the residual group structure, m global context channel attention modules are integrated in series; step four, constructing a structure loss function; step five, inputting the expanded training data set into the global context channel attention network, and training the network by taking the structure loss function as an optimization target; using an ADAM optimizer to update model parameters until the loss no longer decreases, and obtaining a trained infrared image super-resolution reconstruction model; and step six, performing super-resolution reconstruction on low-resolution images in a test set by using the trained infrared image super-resolution reconstruction model, and obtaining infrared super-resolution images.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, specifically relating to an infrared image super-resolution algorithm based on global contextual channel attention. Background Technology

[0002] Infrared imaging, with its unique advantages, has become an irreplaceable branch of imaging technology compared to visible light, and is widely used in fields such as security monitoring, military reconnaissance, and medical imaging. All objects above absolute zero emit infrared thermal radiation. Due to its unique imaging method and longer wavelength compared to visible light, infrared imaging technology can produce clear images even in low-light or foggy conditions. This makes it crucial for nighttime navigation, border security, military reconnaissance, and biomedical applications. However, limited by the manufacturing processes and materials of infrared imaging technology, its pixel count cannot yet reach the level of visible light imaging. Furthermore, increasing resolution by improving imaging equipment performance comes at a high cost. These factors collectively drive the demand for infrared super-resolution technology.

[0003] Image super-resolution reconstruction is a fundamental task in computer vision, aiming to reconstruct a clearer high-resolution (HR) image from a low-resolution (LR) input image. Following the significant success of AlexNet in object classification, deep learning has become a focal point in image processing, with various vision tasks attempting to use it as a new method to replace traditional approaches. This is also true in the field of image super-resolution, where various deep learning-based image super-resolution reconstruction algorithms have emerged. However, many current deep learning super-resolution reconstruction algorithms primarily focus on visible light images and have not been specifically optimized for the signal-to-noise ratio, contrast, edge structure richness, and non-uniformity characteristics of infrared images. Therefore, it is particularly necessary to design efficient and improved super-resolution reconstruction network algorithms for infrared images to enhance their super-resolution reconstruction performance. Summary of the Invention

[0004] In view of this, the main objective of the present invention is to provide an infrared image super-resolution method based on global contextual channel attention.

[0005] To achieve the above objectives, the technical solution of the present invention is implemented as follows:

[0006] This invention provides an infrared image super-resolution method based on global contextual channel attention, the method comprising the following steps:

[0007] Step 1: Obtain the training dataset;

[0008] Step 2: Expand the training dataset;

[0009] Step 3: Construct the Global Context Channel Attention Network (GCCAN). The GCCAN network is based on the RCAN network architecture. While keeping the overall architecture of the RCAN network unchanged, it improves the construction of the residual group in the deep feature extraction stage. In the residual group structure, m interconnected Global Context Channel Attention Blocks (GCCAB) are incorporated.

[0010] Step 4: Construct the structural loss function;

[0011] Step 5: Input the expanded training dataset obtained in Step 2 into the global context channel attention network constructed in Step 3, and train it using the structural loss function constructed in Step 4 as the optimization objective. Use the ADAM optimizer to update the model parameters until the loss no longer decreases, and obtain the trained infrared image super-resolution reconstruction model.

[0012] Step Six: Use the infrared image super-resolution reconstruction model trained in Step Five to perform super-resolution reconstruction on the low-resolution images in the test set to obtain infrared super-resolution images.

[0013] In the above scheme, obtaining the training dataset specifically includes: acquiring original high-resolution images from public databases or personal collections, and then processing the acquired high-resolution images through I... LR =D w B l I HR +n o The described image degradation model is obtained by degradation, where I HR Represents the original high-resolution infrared image, B l D represents fuzzy interference. w This represents the downsampling process of a high-resolution image, n o I represents noise during the imaging process. LR For the acquired low-resolution infrared images, a Gaussian blur kernel with a standard deviation of 1 was used in the degradation model, and Bicubic interpolation was used for downsampling. Additive Gaussian noise was used for the noise. Finally, the set of these one-to-one corresponding high- and low-resolution image pairs was used as the training dataset.

[0014] In the above scheme, expanding the training dataset specifically includes: applying a data augmentation strategy to the acquired training dataset, that is, randomly flipping the dataset images horizontally or rotating them by 90°, 180° and 270°.

[0015] In the above scheme, the Global Context Channel Attention Network (GCCAN) specifically includes: GCCAN comprises three parts: shallow feature extraction, deep feature extraction, and image reconstruction. In the shallow feature extraction stage, the model uses a convolution operation to process a single-channel infrared image with an input size of H×W×1, mapping it to a feature map with deeper channels. In GCCAN, the channel depth after mapping is set to 64. Subsequently, the deep feature extraction stage includes n interconnected residual groups and a convolution operation at the end. The image reconstruction part operates on sub-pixel convolution upsampling to upscale the extracted feature map to the target resolution.

[0016] The GCCAN network is based on the RCAN network architecture. While maintaining the overall architecture of the RCAN network, it improves the construction of the residual group in the deep feature extraction stage. In the residual group structure, m interconnected Global Context Channel Attention Blocks (GCCABs) are incorporated. In the GCCAN model, n and m are adjustable hyperparameters, and n = m = 10 is set.

[0017] In the above scheme, the Global Context Channel Attention Module (GCCAB) specifically includes: GCCAB is based on the residual channel attention module of the RCAN algorithm, which optimizes the mechanism for calculating channel attention, introduces global context channel attention, and embeds parameterless channel shift operations to expand the receptive field of the network.

[0018] In the above scheme, the global context channel attention specifically includes: for an input feature map F of size H×W×C, the global context channel attention uses 1×1 convolution to generate a spatial self-attention map shared by all channels. Then, the spatial self-attention map was analyzed. The dimensions are transformed from H×W×1 to 1×1×HW. Then, through... Spatial self-attention map of the generation probability distribution Where a represents The original value in Spatial self-attention graph representing the generation probability distribution Elements in;

[0019] Obtain the spatial self-attention map of the probability distribution. Then, the feature map F is transformed into an HW×C size and multiplied by its matrix, i.e., according to... The channel descriptor tensor Z is calculated, where F′ represents the deformed feature map, which is a tensor of size HW×C. Z is the spatial self-attention map of the probability distribution, with a size of 1×1×HW, and Z is the final channel descriptor tensor, with a size of 1×1×C.

[0020] After extracting the descriptors of each channel, the interdependencies between the channels are captured.

[0021] In the above scheme, the channel shifting operation specifically includes: shifting the input feature map F of size H×W×C through... The four channels are moved one unit distance up, down, left, and right in the width and height dimensions of space, respectively, where γ is the proportion of the channels moved. This is a feature map after the channel shift.

[0022] In the above scheme, constructing the structural loss function specifically includes: using the L1 loss function: With structural loss function: The weighted sum form is used as the loss function when training the network: Loss = L L1 (I HR ,I G )+λL structure (I HR ,I G );

[0023] Where W and H represent the width and height of the image, respectively, and x ij y represents the pixel values ​​of the reconstructed image. ij Represents the pixel value of the target image, where i and j are the image indices, I HR and I G Let E represent the reconstructed high-resolution infrared image and the original high-resolution infrared image, respectively. Let E represent the edge information image, i and j represent coordinate indices, W and H represent the width and height of the image, respectively, and L represent the edge information image. structure (·) represents the calculated structural loss function.

[0024] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0025] This invention employs a Global Context Channel Attention (GCCAN)-based infrared image super-resolution algorithm. By enhancing the network's understanding and reconstruction capabilities of infrared features, and efficiently fusing global and local information, it helps to overcome the limitations of current super-resolution technologies in reconstructing infrared image details and characteristics, thereby improving the overall quality and practicality of the reconstructed images. Attached Figure Description

[0026] Figure 1 This is a diagram showing the overall structure of the GCCAN network in this invention;

[0027] Figure 2This is a schematic diagram of GCCAB in this invention;

[0028] Figure 3 This refers to the global context channel attention structure in this invention;

[0029] Figure 4 This is a schematic diagram of channel shifting in the present invention;

[0030] Figure 5 The curve showing the change in PSNR during training. Detailed Implementation

[0031] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0032] This invention provides an infrared image super-resolution method based on global contextual channel attention, such as... Figure 1-4 As shown, the method includes the following steps:

[0033] Step 1: Obtain the training dataset;

[0034] Specifically, raw high-resolution images are obtained from public databases or personal collections, and the obtained high-resolution images are then processed through I... LR =D w B l I HR +n o The described image degradation model is obtained by degradation, where I HR Represents the original high-resolution infrared image, B l D represents fuzzy interference. w This represents the downsampling process of a high-resolution image, n o I represents noise during the imaging process. LR For the acquired low-resolution infrared images, a Gaussian blur kernel with a standard deviation of 1 was used in the degradation model, and Bicubic interpolation was used for downsampling. Additive Gaussian noise was used for the noise. Finally, the set of these one-to-one corresponding high- and low-resolution image pairs was used as the training dataset.

[0035] Step 2: Expand the training dataset;

[0036] Specifically, a data augmentation strategy is adopted for the acquired training dataset, that is, the dataset images are randomly flipped horizontally or rotated by 90°, 180° and 270°.

[0037] Step 3: Construct the Global Context Channel Attention Network (GCCAN). The GCCAN network is based on the RCAN network architecture. While keeping the overall architecture of the RCAN network unchanged, it improves the construction of the residual group in the deep feature extraction stage. In the residual group structure, m interconnected Global Context Channel Attention Blocks (GCCAB) are incorporated.

[0038] Specifically, GCCAN comprises three parts: shallow feature extraction, deep feature extraction, and image reconstruction. In the shallow feature extraction stage, the model uses a convolution operation to process a single-channel infrared image with an input size of H×W×1, mapping it to a feature map with deeper channels. In GCCAN, the channel depth after mapping is set to 64. Subsequently, the deep feature extraction stage includes n interconnected residual groups and a convolution operation at the end. The image reconstruction part operates on subpixel convolution upsampling to upscale the extracted feature map to the target resolution.

[0039] The GCCAN network is based on the RCAN network architecture. While maintaining the overall architecture of the RCAN network, it improves the construction of the residual group in the deep feature extraction stage. In the residual group structure, m interconnected Global Context Channel Attention Blocks (GCCABs) are incorporated. In the GCCAN model, n and m are adjustable hyperparameters, and n = m = 10 is set.

[0040] GCCAB is based on the residual channel attention module of the RCAN algorithm. It optimizes the mechanism for calculating channel attention, introduces global context channel attention, and embeds parameterless channel shift operations to expand the receptive field of the network.

[0041] For an input feature map F of size H×W×C, the global context channel attention uses a 1×1 convolution to generate a spatial self-attention map shared by all channels. Then, the spatial self-attention map was analyzed. The dimensions are transformed from H×W×1 to 1×1×HW. Then, through... Spatial self-attention map of the generation probability distribution Where a represents The original value in Spatial self-attention graph representing the generation probability distribution Elements in;

[0042] Obtain the spatial self-attention map of the probability distribution. Then, the feature map F is transformed into an HW×C size and multiplied by its matrix, i.e., according to... The channel descriptor tensor Z is calculated, where F′ represents the deformed feature map, which is a tensor of size HW×C. Z is the spatial self-attention map of the probability distribution, with a size of 1×1×HW, and Z is the final channel descriptor tensor, with a size of 1×1×C.

[0043] For an input feature map F of size H×W×C, the global context channel attention uses a 1×1 convolution to generate a spatial self-attention map shared by all channels. The spatial self-attention map The dimensions are transformed from H×W×1 to 1×1×HW;

[0044] Spatial self-attention maps of probability distributions are generated using the Softmax function.

[0045] in, This represents the spatial self-attention map after the input softmax function has been transformed, where 'a' represents... The original value in Spatial self-attention graph representing the generation probability distribution Elements in;

[0046] The feature map F is transformed into an HW×C size and multiplied by its matrix to determine the channel descriptor tensor Z. Where F′ represents the deformed feature map, which is a tensor of size HW×C. The spatial self-attention map is a probability distribution with a size of 1×1×HW.

[0047] After extracting the descriptors of each channel, the interdependencies between the channels are captured.

[0048] The channel shifting operation specifically includes: shifting the input feature map F of size H×W×C through... The four channels are moved one unit distance up, down, left, and right in the width and height dimensions of space, respectively, where γ is the proportion of the channels moved. This is a feature map after the channel shift.

[0049] Step 4: Construct the structural loss function;

[0050] Specifically, the L1 loss function is used: With structural loss function: The weighted sum form is used as the loss function when training the network: Loss = L L1 (I HR ,I G)+λL structure (I HR ,I G );

[0051] Where W and H represent the width and height of the image, respectively, and x ij y represents the pixel values ​​of the reconstructed image. ij Represents the pixel value of the target image, where i and j are the image indices, I HR and I G Let E represent the reconstructed high-resolution infrared image and the original high-resolution infrared image, respectively. Let E represent the edge information image, i and j represent coordinate indices, W and H represent the width and height of the image, respectively, and L represent the edge information image. structure (·) represents the calculated structural loss function. The weighting factor λ of the structural loss function is set to 0.2 in this invention.

[0052] According to g x (i,j)=(I*K x (i,j), g y (i,j)=(I*K y (i,j) obtains the edge information in the horizontal and vertical directions, where I represents the high-resolution image or super-resolution reconstructed image after Gaussian filtering, and K x and K y The detection operators are divided into horizontal and vertical directions, where i and j are the coordinates of pixels in the high-resolution image or super-resolution reconstructed image, and g... x and g y These represent the extracted edge information in the horizontal and vertical directions, respectively.

[0053] The final key edge information E is obtained based on the edge information in the horizontal and vertical directions.

[0054] Step 5: Input the expanded training dataset obtained in Step 2 into the global context channel attention network constructed in Step 3, and train it using the structural loss function constructed in Step 4 as the optimization objective. Use the ADAM optimizer to update the model parameters until the loss no longer decreases, and obtain the trained infrared image super-resolution reconstruction model.

[0055] Step Six: Use the infrared image super-resolution reconstruction model trained in Step Five to perform super-resolution reconstruction on the low-resolution images in the test set to obtain infrared super-resolution images.

[0056] The test example of this invention is as follows:

[0057] When training the GCCAN algorithm network with a magnification of ×2, PSNR is evaluated on the acquired test set every thousand iterations. Simultaneously, the test results of the GCCAN algorithm are compared with those of the RCAN algorithm. The test and comparison results are as follows: Figure 5 As shown.

[0058] from Figure 5 As training progresses, the test results of both GCCAN and RCAN algorithms gradually increase and stabilize. GCCAN ultimately achieves a higher Peak Signal-to-Noise Ratio (PSNR), indicating its stronger learning ability. Furthermore, the GCCAN algorithm model exhibits high stability during iteration due to its global contextual channel attention mechanism, which effectively captures and utilizes features at different levels, thus providing more accurate detail recovery in super-resolution tasks. These results demonstrate that the GCCAN model exhibits superior performance during training compared to the RCAN model, validating the algorithm's effectiveness and potential.

[0059] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention.

Claims

1. A super-resolution method for infrared images based on global contextual channel attention, characterized in that, The method includes the following steps: Step 1: Obtain the training dataset; Step 2: Expand the training dataset; Step 3: Construct the Global Context Channel Attention Network (GCCAN). The GCCAN network is based on the RCAN network architecture. While maintaining the overall architecture of the RCAN network, improvements are made to the construction of the residual set in the deep feature extraction stage. The residual set structure incorporates… m A series of interconnected global context channel attention modules; Step 4: Construct the structural loss function; Step 5: Input the expanded training dataset obtained in Step 2 into the global context channel attention network constructed in Step 3, and train it using the structural loss function constructed in Step 4 as the optimization objective. Use the ADAM optimizer to update the model parameters until the loss no longer decreases, and obtain the trained infrared image super-resolution reconstruction model. Step 6: Use the infrared image super-resolution reconstruction model trained in Step 5 to perform super-resolution reconstruction on the low-resolution images in the test set to obtain infrared super-resolution images; The Global Context Channel Attention Network (GCCAN) specifically includes three parts: shallow feature extraction, deep feature extraction, and image reconstruction. In the shallow feature extraction stage, the model uses a convolution operation to process the input size. H×W× A single-channel infrared image of size 1 is mapped to a feature map with deeper channels; in GCCAN, the mapped channel depth is set to 64, and then the deep feature extraction stage includes... n The image reconstruction part consists of a series of interconnected residual groups and a convolution operation at the end. The subpixel convolution upsampling operation is used to upscale the extracted feature maps to the target resolution. The GCCAN network is based on the RCAN network architecture. While maintaining the overall architecture of the RCAN network, it improves the construction of the residual set in the deep feature extraction stage by incorporating [a specific feature] into the residual set structure. m In the GCCAN model, there are several interconnected global context channel attention modules. n and m All are adjustable hyperparameters, set n = m = 10, The global context channel attention module specifically includes: GCCAB is based on the residual channel attention module of the RCAN algorithm, which optimizes the mechanism for calculating channel attention, introduces global context channel attention, and embeds parameterless channel shift operations to expand the receptive field of the network.

2. The infrared image super-resolution method based on global contextual channel attention according to claim 1, characterized in that: The acquisition of the training dataset specifically includes: obtaining raw high-resolution images from public databases or personal collections, and then processing the obtained high-resolution images through... The described image degradation model is obtained by degradation, where, I HR This represents the original high-resolution infrared image. B l Indicates fuzzy interference. D w This represents the downsampling process for high-resolution images. n o This represents noise during the imaging process. I LR For the acquired low-resolution infrared images, a Gaussian blur kernel with a standard deviation of 1 was used in the degradation model. Bicubic Interpolation downsampling is performed, and additive Gaussian noise is used for the noise. Finally, the set of these one-to-one corresponding high- and low-resolution image pairs is used as the training dataset.

3. The infrared image super-resolution method based on global contextual channel attention according to claim 1 or 2, characterized in that: The expansion of the training dataset specifically includes: applying data augmentation strategies to the acquired training dataset, namely, randomly flipping the dataset images horizontally or rotating them by 90°, 180°, and 270°.

4. The infrared image super-resolution method based on global contextual channel attention according to claim 3, characterized in that: The global context channel attention specifically includes: for an input size of H×W×C Feature map F Global context channel attention uses 1×1 convolutions to generate a spatial self-attention map shared by all channels. S Then, for spatial self-attention maps S Deformation, changing the size from H×W× 1 transforms into 1×1× HW Then, through Spatial self-attention map of the generation probability distribution ,in, a express S The original value in Spatial self-attention graph representing the generation probability distribution Elements in; Obtain the spatial self-attention map of the probability distribution. Then, the feature map F Transformed into HW×C The size is multiplied by its matrix, i.e., according to Calculate the channel descriptor tensor Z， in, The deformed feature map is a HW×C Tensors of size, The spatial self-attention map is a probability distribution with a size of [size missing]. 1×1×HW , Z This is the final channel descriptor tensor, with a size of [size missing]. 1×1×C ; After extracting the descriptors of each channel, the interdependencies between the channels are captured.

5. The infrared image super-resolution method based on global contextual channel attention according to claim 4, characterized in that: The channel shifting operation specifically includes: shifting the input size to a value of 10 ... H × W × C Feature map F China passed Move its four channels one unit distance up, down, left, and right in both the width and height dimensions of space. This represents the proportion of the moving channel. This is a feature map after the channel shift.

6. The infrared image super-resolution method based on global contextual channel attention according to claim 5, characterized in that: The construction of the structural loss function specifically includes: using the L1 loss function: With structural loss function: The weighted sum form is used as the loss function when training the network: ; in, W and H These represent the width and height of the image, respectively. Represents the pixel values ​​of the reconstructed image. Represents the pixel values ​​of the target image. i and j It is an index of the image. I HR and I G These represent the reconstructed high-resolution infrared image and the original high-resolution infrared image, respectively. E Represents edge information image, i and j Indicates coordinate index, W and H These represent the width and height of the image, respectively. This is the calculated structural loss function.

Images