Cold reflection correction method of refrigeration infrared continuous zooming system based on attention network

Abstract

The invention discloses a refrigeration infrared continuous zooming system cold reflection correction method based on an attention network, relates to the technical field of optical imaging and deep learning, and aims to solve the problem that an existing cold reflection correction method cannot correct cold reflection of an infrared continuous zooming system under different focal lengths. Step 2, constructing a U-Net network cold reflection correction module in which an attention mechanism is introduced; step 3, constructing a loss function of the cold reflection correction method; the attention mechanism has three cold reflection correction modes and can be adaptive to cold reflection areas in images shot by the focusing refrigeration type infrared continuous zooming system under different focal lengths. An attention mechanism is applied to different network levels of a decoding path in the U-Net network. The U-Net network with the attention mechanism is introduced, cold reflection correction of the refrigeration type infrared continuous zooming system in images under different focal lengths is achieved, cold reflection artifacts are basically and completely eliminated, and a good cold reflection correction effect is achieved.

Description

Technical Field

[0001] This invention relates to the field of deep learning technology, and in particular to a cold reflection correction method for a cooled infrared continuous zoom system based on attention networks. Background Technology

[0002] Traditional cooled infrared continuous zoom systems suppress cold reflections by controlling the lens curvature radius, increasing the number of lenses and even-order aspherical surfaces, and applying anti-reflection coatings to the lens surfaces. However, this method only reduces the intensity of cold reflections and still results in some cold reflection artifacts, and requires a more complex system structure. By combining cooled infrared continuous zoom systems with computational imaging, a simplified and lightweight system with high image quality is achieved, demonstrating great potential in the design and cold reflection correction of cooled infrared zoom systems.

[0003] See Chinese Patent Publication No. CN117437145A, entitled "A Deep Learning-Based Method for Suppressing Cold Reflections in Infrared Images." This method designs a cold reflection suppression method for cooled infrared optical systems. It first uses MATLAB to generate a random number A representing the intensity of cold reflection in the range of 30-50 and a random number representing the range of cold reflection in the range of 0.4-0.6. Then, based on the generated random numbers indicating the intensity and range of cold reflections, infrared images with cold reflection noise are generated and used as a paired dataset. A lightweight neural network is then constructed and trained, incorporating convolutional modules, an encoder, a decoder, a downsampling module, and an upsampling module. Finally, the trained lightweight neural network is used to process infrared images with strong cold reflections in the center, resulting in infrared images with suppressed cold reflections. However, this method is only applicable to cooled infrared fixed-focus systems, and the cold reflection energy is concentrated in the central region of the image, making it unable to correct cold reflections at different focal lengths in infrared continuous zoom systems. Summary of the Invention

[0004] To address the problem that existing cold reflection correction methods cannot correct cold reflection in infrared continuous zoom systems at different focal lengths, this invention provides a cold reflection correction method for cooled infrared continuous zoom systems based on attention networks. This method can simultaneously correct cold reflection in infrared continuous zoom systems at different focal lengths, and the cold reflection correction effect is good.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] A method for cold reflection correction in a cooled infrared continuous zoom system based on attention networks, comprising the following steps:

[0007] Step 1: Construct the attention mechanism module;

[0008] Step 2: Construct a U-Net network cold reflection correction module that incorporates an attention mechanism;

[0009] Step 3: Construct the loss function for the cold reflection correction method.

[0010] Step one involves constructing an attention mechanism module. Based on the characteristics of cold reflection energy distribution at different focal lengths in a cooled infrared continuous zoom system, three cold reflection correction modes are designed:

[0011] Center-focused mode: assigns higher weight to the central region of the image and can automatically detect cold reflection areas;

[0012] Edge-sensitive mode: Generates a high-gradient attention response at the boundary of cold reflection spot to ensure a smooth transition from the cold reflection artifact area to the normal image area;

[0013] Content-aware mode: assigns the highest weight to completely dark areas in cold reflection artifacts, a medium weight to partially visible texture areas, and a weight close to zero to normal areas, in order to differentiate repair priorities.

[0014] Step two involves constructing a U-Net network cold reflection correction module that incorporates an attention mechanism. The attention mechanism module constructed in step one analyzes the global characteristics of cold reflections based on encoder features and combines them with decoder features to achieve precise localization of cold reflections. By fusing the features from both the encoder and decoder paths, an attention weight map is generated to guide the U-Net network to enhance its feature response to cold reflection areas and suppress feature activation in normal areas, forming a feature selection mechanism that allows the network to focus on processing cold reflection areas.

[0015] The U-Net network consists of an encoding path for feature extraction and a decoding path for feature processing and fusion. The feature extraction part of the encoding path adopts a deep convolutional block structure. Through a series of convolutional blocks and downsampling operations, it extracts multi-level features from low-level texture to high-level semantics of the input image. The decoding path processes the cold reflection area according to the attention weight map and preserves the texture details of the original image. Through a series of upsampling and feature fusion operations, it gradually restores the spatial resolution of the image and reconstructs the image with corrected cold reflection.

[0016] The loss function of the cold reflection correction method described in step three includes L1(p) i ,t i Loss, edge loss L edge and structural similarity loss L ssim (p,t), the three loss functions constitute the combined loss function L. t It is determined by the following formula:

[0017]

[0018]

[0019]

[0020]

[0021] In the formula: Let i be the total number of pixels and i be the pixel number. To predict the pixel values ​​of an image, This refers to a pixel value in the actual target image. For predicting images Orientation gradient, For predicting images Orientation gradient, For real target image Orientation gradient, For real target image Orientation gradient; , represent the local window mean values ​​of the predicted image and the real target image, respectively. , These represent the standard deviations of the predicted image and the actual target image, respectively. w2 represents the covariance between the predicted image and the real target image, and w2 is the correlation coefficient. , , It is a parameter used to control the loss weights of each part.

[0022] Simultaneously, during training, the validation set PSNR is used as the primary metric for model selection. The mean squared error (MSE) is used to evaluate the similarity between the distorted image and the original image, which is the average of the sum of squared energy errors of all pixels in the distorted and original images. This is then converted into a logarithmic scale to evaluate image quality, ensuring the acquisition of model parameters with optimal performance. The expression is as follows:

[0023]

[0024]

[0025] In the formula: MAX is the maximum grayscale value that the color bits in the image can display; M and N are the length and width of the image, respectively, and I(i,j) is the energy value of the corresponding row and column pixels in the image.

[0026] The beneficial technical effects of the present invention are as follows:

[0027] The attention mechanism designed in this invention features three cold reflection correction modes, enabling adaptive correction of cold reflection regions in images captured by a cooled infrared continuous zoom system at different focal lengths. Simultaneously, the designed attention mechanism is applied to different network layers within the decoding path of the U-Net network. Utilizing the U-Net network incorporating the attention mechanism, cold reflection correction can be achieved in images of a cooled infrared continuous zoom system at different focal lengths, essentially eliminating cold reflection artifacts and demonstrating excellent cold reflection correction performance.

[0028] Compared to existing methods that use random numbers to determine the intensity and range of cold reflections, this invention uses images of the cold reflection energy distribution of a real infrared continuous zoom system at different focal lengths as its dataset. The trained model can simultaneously correct the cold reflections of the infrared continuous zoom system at different focal lengths. The U-Net network cold reflection correction model constructed using this method, which incorporates an attention mechanism, has low computational cost, high computational efficiency, and strong real-time performance, providing a new solution for cold reflection correction in cooled infrared continuous zoom systems. Attached Figure Description

[0029] Figure 1 This is a flowchart of the cold reflection correction method for a cooled infrared continuous zoom system based on attention networks according to the present invention;

[0030] Figure 2 This is a schematic diagram of the attention mechanism construction module described in this invention;

[0031] Figure 3 This is a schematic diagram of the U-Net network cold reflection correction with the introduction of an attention mechanism as described in this invention;

[0032] Figure 4 The images shown are comparisons of infrared images at different focal lengths before and after cold reflection correction according to the present invention. (a) shows the images before and after cold reflection correction at the shortest focal length, (b) shows the images before and after cold reflection correction at a relatively short focal length, and (c) shows the images before and after cold reflection correction at a long focal length. Detailed Implementation

[0033] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0034] like Figure 1 As shown, a cold reflection correction method for a cooled infrared continuous zoom system based on attention networks is described. This method includes the following steps:

[0035] Step 1: Construct the attention mechanism module;

[0036] Based on the characteristics of cold reflection energy distribution in a cooled infrared continuous zoom system at different focal lengths, an attention mechanism module is constructed, such as... Figure 2 As shown, there are three modes for cold reflection correction:

[0037] Center-focused mode: assigns higher weight to the central region of the image and can automatically detect cold reflection areas;

[0038] Edge-sensitive mode: Generates a high-gradient attention response at the boundary of cold reflection spot to ensure a smooth transition from the cold reflection artifact area to the normal image area;

[0039] Content-aware mode: assigns the highest weight to completely dark areas in cold reflection artifacts, a medium weight to partially visible texture areas, and a weight close to zero to normal areas, in order to differentiate repair priorities.

[0040] This attention mechanism module calculates a gating signal, enabling the model to adaptively focus on cold reflection areas in the image, prioritizing the processing of cold reflection areas while preserving detailed information in normal imaging areas.

[0041] Step 2: Construct a U-Net network cold reflection correction module that incorporates an attention mechanism;

[0042] The attention mechanism module constructed in step one is applied to the cold reflection correction module of the U-Net network, such as... Figure 3 As shown, based on encoder features, the global characteristics of cold reflections are analyzed to identify their boundary relationship with normal areas. Then, decoder features are combined to achieve pixel-level precise localization while preserving the texture details of the original image. By fusing the features from both encoding and decoding, an attention weight map between 0 and 1 is generated: a weight close to 1 indicates cold reflection areas requiring focused repair, while a weight close to 0 indicates normal areas that do not require modification. This weight map enhances the feature response to cold reflection areas and suppresses feature activation in normal areas, thus avoiding unnecessary modifications and forming a feature selection mechanism that allows the network to focus on processing cold reflection areas. The attention mechanism is further applied to different network layers in the decoding path: the deep network attention mechanism is responsible for identifying the overall shape and position of cold reflection artifacts over a large area; the mid-level network attention mechanism focuses on the edge transition areas of cold reflections, coordinating the natural fusion of the repaired area with surrounding pixels; and the shallow network attention mechanism is used to restore detailed textures, ensuring the visual realism and consistency of the repair results.

[0043] A U-Net network cold reflection correction module, incorporating an attention mechanism, is constructed by inputting infrared images at different focal lengths captured by a cooled infrared continuous zoom system. The input infrared image first passes through a feature extraction encoding path, which extracts multi-level features from low-level texture to high-level semantics through a series of convolutional blocks and downsampling operations. Then, it passes through a feature processing and fusion decoding path. This decoding path processes the cold reflection region according to the attention weight map while preserving the texture details of the original image. Through a series of upsampling and feature fusion operations, the decoding path gradually restores the image's spatial resolution and reconstructs an image with corrected cold reflections. Figure 3 As shown.

[0044] Step 3: Construct the loss function for the cold reflection correction method.

[0045] The loss function of the cold reflection correction method adopts a multi-component combined loss function L. t It includes L1(p i ,t i Loss, edge loss L edge and structural similarity loss L ssim (p,t). This combined loss function L t It cleverly combines constraints at different levels: L1(p i ,t i The loss is responsible for macroscopic pixel alignment; the edge loss L edge Responsible for microscopic edge sharpness; structural similarity loss L ssim (p,t) are responsible for perceptual image quality. These three components work together to guide the model in learning how to generate accurate, clear, and natural output images without cold reflections from input images with cold reflection spots. This composite loss function design allows the model to simultaneously consider multiple optimization objectives such as pixel accuracy, edge quality, and structural fidelity. The formula is as follows:

[0046]

[0047]

[0048]

[0049]

[0050] In the formula: Let i be the total number of pixels and i be the pixel number. To predict the pixel values ​​of an image, This refers to a pixel value in the actual target image. For predicting images Orientation gradient, For predicting images Orientation gradient, For real target image Orientation gradient, For real target image Orientation gradient; , represent the local window mean values ​​of the predicted image and the real target image, respectively. , These represent the standard deviations of the predicted image and the actual target image, respectively. w2 represents the covariance between the predicted image and the real target image, and w2 is the correlation coefficient. , , It is a parameter used to control the loss weights of each part.

[0051] During training, the validation set PSNR is used as the primary metric for model selection. The mean squared error (MSE) is used to evaluate the similarity between the distorted image and the original image, which is the average of the sum of squared energy errors of all pixels in the distorted and original images. This is then converted to a logarithmic scale to evaluate image quality, ensuring the acquisition of model parameters with optimal performance. The expression is as follows:

[0052]

[0053]

[0054] In the formula: MAX is the maximum grayscale value that the color bits in the image can display; M and N are the length and width of the image, respectively, and I(i,j) is the energy value of the corresponding row and column pixels in the image.

[0055] Example:

[0056] A method for cold reflection correction in a cooled infrared continuous zoom system based on attention networks, comprising the following steps:

[0057] Step 1: Construct the attention mechanism module:

[0058] Based on the cold reflection energy distribution characteristics of a cooled infrared 10x continuous zoom system with an F-number of 2 at different focal lengths, this invention constructs an attention mechanism module with three cold reflection correction modes: center focusing, edge sensitivity, and content awareness. The structure of the attention mechanism module constructed in the embodiment is as follows: Figure 2As shown, the left side, from top to bottom, displays cold reflection images captured by the infrared continuous zoom system at the shortest, shorter, and longer focal lengths. The right side presents an attention mechanism module with three cold reflection correction modes, constructed based on the cold reflection energy distribution characteristics in the images at the shortest, shorter, and longer focal lengths on the left. Based on the characteristic that the cold reflection energy in the central region of the shortest and shorter focal length images is much higher than in other regions, a center-focusing mode is designed, assigning a higher weight to the central region of the image to detect whether there is strong cold reflection in the center of the input image. Based on the fact that the cold reflection energy in the longer focal length image is weaker from the center outwards, and considering the shorter focal length, shorter, and longer focal lengths... Given the weaker cold reflection energy in the peripheral regions of shorter focal length images, an edge-sensitive mode is designed to generate a high-gradient attention response at the cold reflection boundaries, ensuring a smooth transition from cold reflection artifact areas to normal image areas. Based on the characteristic that the cold reflection energy gradually decreases from the center to the edges in the shortest and shorter focal length images (i.e., from completely dark to partially visible to normal areas without cold reflection), a content-aware mode is designed to assign the highest weight to completely dark areas in cold reflection artifacts, a medium weight to partially visible texture areas, and a weight close to zero to normal areas, thus differentiating the restoration priorities.

[0059] Step 2: Construct a U-Net network cold reflection correction module incorporating an attention mechanism:

[0060] By constructing a U-Net network consisting of an encoder, a decoder, and skip connections, and applying the aforementioned attention mechanism module to different network layers in the decoder, the construction of a U-Net network cold reflection correction module incorporating an attention mechanism is completed. The specific structure of the U-Net network cold reflection correction module incorporating an attention mechanism in this embodiment is as follows:

[0061] as follows Figure 3As shown, infrared images captured at different focal lengths by a cooled infrared 10-fold continuous zoom system with an F-number of 2 are input into an encoder. The encoder consists of four convolutional blocks and max pooling. Each convolutional block is composed of two sets of 3x3 convolutional kernels (Conv2d), batch normalization (BatchNorm2d), and ReLU activation functions. Each convolutional block is linked by max pooling using a 2x2 window. The number of convolutional kernels increases sequentially, performing downsampling operations with progressively increasing channel numbers to extract multi-level features from the input image, from low-level texture to high-level semantics. The max pooling result of the fourth encoder layer is convolved to obtain the feature map of the bottleneck layer, which is used as the upsampling module of the fourth decoding layer. The input feature map of the block is then processed and fused by a decoder consisting of four layers: an upsampling module, an attention mechanism module, a feature processing module, and a fusion module. The attention mechanism module and skip links process the feature maps of the convolutional blocks in the encoding layer and the feature maps of the upsampling module in the decoding layer according to three cold reflection correction modes. The Sigmoid function is used to generate corresponding attention weight maps between 0 and 1 in different decoding layers. The feature processing and fusion module of the decoding layer processes the cold reflection region according to the attention weight map and preserves the texture details of the original image. Through a series of upsampling and feature fusion operations of the decoder, the spatial resolution of the image is gradually restored and the image with corrected cold reflection is reconstructed.

[0062] Step 3: Construct the loss function for the cold reflection correction method:

[0063] The cold reflection correction method in this embodiment combines loss functions including: L1 loss, edge loss, and structural similarity (SSIM) loss. The weights of the three loss functions are... The images used in this embodiment are 8-bit color deep, with a maximum grayscale value. The image's length and width are , .

[0064] After 200 training rounds, the training loss curve and the validation loss curve converge, and the difference is very small. After 200 training rounds, the training loss function finally converges to around 0.025, and the validation loss function finally converges to around 0.017. Simultaneously, the training PSNR curve and the validation PSNR curve obtained after 200 training rounds converge, with the training PSNR finally converging to around 28.3 and the validation PSNR finally converging to around 34.

[0065] Finally, this invention performs cold reflection correction on infrared images captured at different focal lengths by a cooled infrared 10x continuous zoom system with an f / 2 aperture. The cold reflection image of the infrared continuous zoom system at its shortest focal length is... Figure 4(a) The upper image clearly shows that the cold reflection energy is strongest in the central area of ​​the image, which is manifested as a black spot; the cold reflection energy gradually decreases from the central area of ​​the image to the surrounding area, which is manifested as a light black ring; and the cold reflection energy is weakest at the edge of the image.

[0066] The cold reflection image of an infrared continuous zoom system at a shorter focal length is Figure 4 (b) In the upper image, it can be clearly seen that the cold reflection energy in the central region of the image is still the strongest, and stronger than that in the central region of the short focal length, which is manifested as a very dark black spot; the cold reflection energy decreases significantly from the central region of the image to the surrounding area, which is lower than that in the short focal length, which is manifested as a light black ring; while there is almost no cold reflection at the edge of the image.

[0067] Cold reflection images at long focal lengths from an infrared continuous zoom system are Figure 4 (c) In the upper image, it can be clearly seen that the cold reflection energy from the center of the image outwards is weaker, much lower than that of short focal length and shorter focal length, and appears as faint black circular spots.

[0068] The image after cold reflection correction using this method is: Figure 4 (a), (b), and (c) are the lower images. It can be seen that the cold reflection in the original image is almost completely eliminated, and the image quality after cold reflection correction is good. The image details in the areas without cold reflection are also almost entirely preserved.

Claims

1. A cold reflection correction method for a cooled infrared continuous zoom system based on attention networks, characterized in that, The method includes the following steps: Step 1: Construct the attention mechanism module; Step 2: Construct a U-Net network cold reflection correction module that incorporates an attention mechanism; Step 3: Construct the loss function for the cold reflection correction method.

2. The cold reflection correction method for a cooled infrared continuous zoom system based on attention networks according to claim 1, characterized in that, Step one involves constructing an attention mechanism module. Based on the characteristics of cold reflection energy distribution at different focal lengths in a cooled infrared continuous zoom system, three cold reflection correction modes are designed: Center-focused mode: assigns higher weight to the central region of the image and can automatically detect cold reflection areas; Edge-sensitive mode: Generates a high-gradient attention response at the boundary of cold reflection spot to ensure a smooth transition from the cold reflection artifact area to the normal image area; Content-aware mode: assigns the highest weight to completely dark areas in cold reflection artifacts, a medium weight to partially visible texture areas, and a weight close to zero to normal areas, in order to differentiate repair priorities.

3. The cold reflection correction method for a cooled infrared continuous zoom system based on attention network as described in claim 1, wherein step two involves constructing a U-Net network cold reflection correction module that incorporates an attention mechanism. The attention mechanism module constructed in step one analyzes the global characteristics of cold reflection based on encoder features and combines them with decoder features to achieve precise localization of cold reflection. By fusing the features from both the encoding and decoding paths, an attention weight map is generated to guide the U-Net network to enhance its feature response to the cold reflection region and suppress feature activation in the normal region, thus forming a feature selection mechanism that enables the network to focus on processing the cold reflection region.

4. The cold reflection correction method for a cooled infrared continuous zoom system based on attention networks according to claim 3, characterized in that, The U-Net network consists of an encoding path for feature extraction and a decoding path for feature processing and fusion. The feature extraction part of the encoding path adopts a deep convolutional block structure, which extracts multi-level features from low-level texture to high-level semantics of the input image through a series of convolutional blocks and downsampling operations. The decoding path processes the cold reflection region based on the attention weight map, while preserving the texture details of the original image. Through a series of upsampling and feature fusion operations, it gradually restores the image spatial resolution and reconstructs an image with corrected cold reflections.

5. The cold reflection correction method for a cooled infrared continuous zoom system based on attention networks according to claim 1, characterized in that, The loss function of the cold reflection correction method described in step three includes L1(p) i ,t i Loss, edge loss L edge and structural similarity loss L ssim (p,t), the three loss functions constitute the combined loss function L. t It is determined by the following formula: ； ； ； ； In the formula: Let i be the total number of pixels and i be the pixel number. To predict the pixel values ​​of an image, This refers to a pixel value in the actual target image. For predicting images Orientation gradient, For predicting images Orientation gradient, For real target image Orientation gradient, For real target image Orientation gradient; , represent the local window mean values ​​of the predicted image and the real target image, respectively. , These represent the standard deviations of the predicted image and the actual target image, respectively. w2 represents the covariance between the predicted image and the real target image, and w2 is the correlation coefficient. , , These are parameters used to control the loss weights of each part; Simultaneously, during training, the validation set PSNR is used as the primary metric for model selection. The mean squared error (MSE) is used to evaluate the similarity between the distorted image and the original image, which is the average of the sum of squared energy errors of all pixels in the distorted and original images. This is then converted into a logarithmic scale to evaluate image quality, ensuring the acquisition of model parameters with optimal performance. The expression is as follows: ； ； In the formula: MAX is the maximum grayscale value that the color bits in the image can display; M and N are the length and width of the image, respectively, and I(i,j) is the energy value of the corresponding row and column pixels in the image.

Images