An infrared image enhancement method based on thermal signal decomposition

Abstract

The present application relates to a kind of infrared image enhancement methods based on thermal signal decomposition, belong to computer image processing technical field.The method includes: S1, according to Stefan-Boltzmann law, from temperature information T reconstructs thermal radiation field image I' D , modeling direct thermal emission component I D Without texture information, original infrared image I is decomposed into direct thermal emission component I D And environmental reflection component I E Two parts;S2, from original infrared image I subtract thermal radiation field image I' D , separate out the part I' E Containing texture information, modeling environmental reflection component I E ;S3, by infrared-texture mapping model, in-depth mining potential texture information in the part I' E Containing texture information, obtain enhanced texture information I' ET , by redundancy information separation network G n From the part I' E Containing texture information, extract out the redundancy information I' En Affecting the expression of infrared texture information;S4, thermal radiation field image I' D And enhanced texture information I' ET Fusion, obtain enhanced infrared image I'.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, specifically to an infrared image enhancement method based on thermal signal decomposition. Background Technology

[0002] Infrared imaging technology plays a crucial role in several key areas, including military reconnaissance, security monitoring, and industrial inspection. However, the lack of texture information and low contrast in infrared images pose challenges for target recognition and image analysis. Infrared image enhancement techniques significantly improve these problems by enhancing the clarity of image features. Infrared image enhancement techniques are mainly divided into two categories: traditional methods and deep learning-based methods. Traditional infrared image enhancement methods are primarily divided into spatial domain processing and frequency domain processing. Spatial domain image enhancement methods enhance contrast by redistributing pixel values, including techniques such as histogram equalization and contrast stretching. Spatial domain image enhancement methods are intuitive, easy to operate, and real-time, making them particularly suitable for processing local areas of an image. However, they struggle to consider the overall contrast and details of the image. Frequency domain image enhancement methods enhance the image by transforming it from the spatial domain to the frequency domain and applying frequency domain filtering. Finally, an inverse Fourier transform restores the image to the spatial domain. This method offers excellent noise reduction capabilities and precise control over different frequency components, but it has limitations in processing local details and may also lead to frequency aliasing.

[0003] With the rapid development of deep learning technology, deep learning-based infrared image enhancement methods have demonstrated outstanding performance. Researchers have effectively improved the contrast between the target and the background by using convolutional neural networks (CNNs) to learn filters and predict the features of the target and background, making the target stand out more in the image. Furthermore, to further improve the quality of infrared images, some scholars have introduced generative adversarial networks (GANs). This technique, through adversarial training, not only enhances the texture and edge details of the image but also improves the overall visual effect. However, current infrared image enhancement methods mainly focus on noise reduction, contrast enhancement, and the recombination of texture information. These operations do not essentially introduce new information into the image, thus limiting further improvements in enhancement effects. Therefore, how to effectively mine and utilize the potential texture information in infrared images has become a crucial problem that urgently needs to be solved. Summary of the Invention

[0004] To address the aforementioned technical problems and effectively mine and utilize the potential texture information in infrared images, this application proposes an infrared image enhancement method based on thermal signal decomposition. Specifically, the infrared image I is decomposed into a direct thermal emission component I according to the source of the thermal signal. D (No texture) and ambient reflection component I E(Carrying texture) Two parts. The specific scheme is as follows: S1, According to the Stefan-Boltzmann thermal radiation law, reconstruct the thermal radiation field image I' from the temperature information T. D Modeling the direct emission component I D The original infrared image I is decomposed into direct thermal emission component I. D and environmental reflection component I E Two parts; S2, subtracting the reconstructed thermal radiation field image I' from the original infrared image I. D Separate the part containing texture information I' E Modeling environment reflection component I E S3. Design an infrared-texture mapping model using visible light images. v Delving deeper into the parts containing texture information E The latent texture information is used to obtain the enhanced texture information I'. ET Further, through redundant information separation network G n From the part I' containing texture information E Extract redundant information that affects the representation of infrared texture information. En S4. Image I' of the thermal radiation field. D and enhanced texture information I' ET The images are then fused to obtain the enhanced infrared image I'. It should be understood that, based on the source of the thermal signal, the original infrared image I can be decomposed into a direct thermal emission component I'. D (No texture) and ambient reflection component I E (Carrying texture) Two parts.

[0005] Further optional, in S4, the redundant information I' is extracted. En Image of the thermal radiation field after I' D and enhanced texture information I' En The fusion process includes: S4.1, based on the thermal radiation field image I' D and enhanced texture information I' ET Calculate adaptive weights

[0006] The weight w is represented as: Among them, w PIQE w represents the PIQE weighted components. AG w represents the weighted components of AG. NIQE w represents the NIQE weighted components. STD S4.2, Using adaptive weights w, fuse the components to obtain the enhanced infrared image I', where I' = I' D ·w+I' ET ·(1-w).

[0007] Further optional, wPIQE w AG w NIQE and w STD The expressions are as follows:

[0008]

[0009] Further optional, the infrared-texture mapping model in S3 is generated by generator G. T It consists of a discriminator D and a generator G. T The ResNet structure is adopted, consisting of 9 residual blocks; the discriminator D is based on PatchGAN and consists of 5 convolutional layers; the redundancy information separation network in S3 adopts the U-Net framework and consists of 8 downsampling and 8 upsampling convolutional layers.

[0010] Further, optionally, during the training of the infrared-texture mapping model in S3, a total loss function including adversarial loss, perceptual loss, pixel loss, reconstruction loss, and style loss is used for loss calculation; the adversarial loss is expressed as: Perceptual loss can be expressed as: Among them, W i,j H represents the width of the i-th layer and the j-th feature map. i,j φ represents the height of the i-th layer and the j-th feature map. i,j The function representing the i-th and j-th feature maps extracted from the VGG network; the pixel loss is expressed as: L pix =||I v -G T (I' E )||1; The reconstruction loss is represented as: L rec =||I' E -I” E ||1, where I” E =I' ET ×I' En , representing enhanced texture information I' ET With redundant information I' En Reconstructed pseudo-thermal radiation image; style loss is represented as: L style =||I v -G T (I v The total loss function is expressed as: ||1;

[0011] L=λ1L G +λ2L VGG / i.j +λ3L pix +λ4L rec +λ5L style .

[0012] The beneficial effects of this invention are: (1) This invention divides the original infrared image into direct emission components and environmental emission components according to the signal source, and performs physical modeling and processing on them respectively, thereby improving the denoising and texture enhancement effects. (2) Based on the infrared image texture degradation mechanism, this invention constructs a texture-redundancy information decomposition model, which effectively separates and extracts infrared texture information components. (3) This invention introduces adaptive fusion, which automatically adjusts the image weights according to the no-reference evaluation index, efficiently fuses the texture features of different images, and significantly improves the image reconstruction effect. (4) This invention uses a total loss function including adversarial loss, perceptual loss, pixel loss, reconstruction loss and style loss for loss calculation, which effectively restores the texture details of the infrared image. Attached Figure Description

[0013] Figure 1 This is a schematic flowchart of an infrared image enhancement method based on thermal signal decomposition in this invention.

[0014] Figure 2 This is a visualization of the image enhancement results of the infrared image enhancement method based on thermal signal decomposition in this invention compared with other methods. Detailed Implementation

[0015] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. All other embodiments obtained by those skilled in the art based on the embodiments in this application without inventive effort are within the scope of protection of this application.

[0016] Please see Figure 1 The infrared image enhancement method based on thermal signal decomposition in this invention includes:

[0017] S1. Reconstruct the thermal radiation field image I' from the temperature information T according to the Stefan-Boltzmann law. D Modeling the direct thermal emission component I without texture information D The original infrared image I is decomposed into direct thermal emission components I based on the signal source. D and environmental reflection component I E Two parts.

[0018] S2. Subtract the thermal radiation field image I' from the original infrared image I. D Separate the part containing texture information I' E Modeling environment reflection component I E .

[0019] S3. Design an infrared-texture mapping model, using visible light images I.v Delving deeper into the parts containing texture information E The latent texture information is used to obtain the enhanced texture information I'. ET By separating redundant information in network G n From the part I' containing texture information E Extract redundant information that affects the representation of infrared texture information. En .

[0020] For the infrared-texture mapping model in S3, it is generated by generator G. T It consists of a discriminator D and a generator G. T The ResNet architecture is adopted, consisting of 9 residual blocks; the discriminator D is based on PatchGAN and consists of 5 convolutional layers; the redundancy information separation network adopts the U-Net framework and consists of 8 downsampling and 8 upsampling convolutional layers.

[0021] S4, Image I' of the thermal radiation field D and enhanced texture information I' ET The images are fused to obtain the enhanced infrared image I'.

[0022] Specifically, optional, in S4, the thermal radiation field image I' D and enhanced texture information I' ET The fusion process includes: S4.1, based on the thermal radiation field image I' D and enhanced texture information I' ET The adaptive weight w is calculated as follows: Among them, w PIQE w represents the PIQE weighted components. AG w represents the weighted components of AG. NIQE w represents the NIQE weighted components. STD S4.2, Using adaptive weights w, fuse the components to obtain the enhanced infrared image I', where I' = I' D ·w+I' ET ·(1-w).

[0023] The above w PIQE w AG w NIQE and w STD The expressions are as follows:

[0024]

[0025] In this invention, the adaptive weights are calculated jointly by four parts: Perception-based Image Quality Evaluator (PIQE), Natural Image Quality Evaluator (NIQE), Average Gradient (AG), and Standard Deviation (STD).

[0026] Furthermore, during the training of the infrared-texture mapping model, a total loss function including adversarial loss, perceptual loss, pixel loss, reconstruction loss, and style loss is used for loss calculation; the adversarial loss is expressed as: Perceptual loss can be expressed as: Among them, W i,j H represents the width of the i-th layer and the j-th feature map. i,j φ represents the height of the i-th layer and the j-th feature map. i,j The function representing the i-th and j-th feature maps extracted from the VGG network; the pixel loss is expressed as: L pix =||I v -G T (I' E )||1; The reconstruction loss is represented as: L rec =||I' E -I” E ||1, where I” E =I' ET ×I' En , representing enhanced texture information I' ET With redundant information I' En Reconstructed pseudo-thermal radiation image; style loss is represented as: L style =||I v -G T (I v The total loss function is expressed as: L = λ1L G +λ2L VGG / i.j +λ3L pix +λ4L rec +λ5L style .

[0027] Specifically, perceptual loss is used to ensure that the deep texture encoding of the network output image is closer to that of the visible light image. In this invention, the perceptual loss is defined based on a pre-trained 19-layer Visual Geometry Group (VGG) network. Pixel loss is used to constrain the output image to closely approximate the visible light image. vMaintain consistency at the pixel level. Preserve the original features of the infrared image by constraining the output data through reconstruction loss. Constrain the input and output data of the model to express style consistency through style loss.

[0028] Furthermore, this embodiment of the invention is implemented based on the PyTorch framework, with an initial learning rate set to 0.001, which gradually decays after 20 training rounds until it reaches 0 after 160 epochs. Model training is performed on a single NVIDIA RTX 4090 GPU, while testing is conducted on a single NVIDIA RTX 4060 Ti GPU. 861 pairs of RGB-T images were randomly selected from the publicly available jungle dataset OdinMJ for training.

[0029] This patent also provides a visual comparison of the experimental results of the proposed method with those of the classic histogram equalization method. The comparison results are as follows: Figure 2 As shown in Table 1, further quantitative evaluation was conducted using no-reference metrics such as Average Gradient (AG), Measure of Enhancement by Entropy (EME), Spatial Frequency (SF), and Perception-based Image Quality Evaluator (PIQE). AG and SF represent the sharpness of the image; higher values ​​indicate sharper images. EME evaluates the algorithm's performance in contrast improvement; higher values ​​indicate higher image contrast. The PIQE metric calculates the image quality score based on the image's block structure and noise characteristics; lower PIQE values ​​indicate higher image quality. The evaluation results are shown in Table 1.

[0030] Depend on Figure 2 As can be seen, the method proposed in this patent exhibits better enhancement effects compared to histogram equalization. Table 1 shows that the method proposed in this patent outperforms histogram equalization in all listed evaluation metrics, indicating that the method proposed in this patent restores more texture details and provides better perceptual effects, thus surpassing histogram equalization in all aspects.

[0031] Table 1 Quantitative Assessment of Infrared Texture Information Recovery

[0032]

[0033] The specific embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Within the scope of knowledge possessed by those skilled in the art, various changes can be made without departing from the spirit of the present invention.

Claims

1. An infrared image enhancement method based on thermal signal decomposition, characterized in that, Includes the following steps: S1. Reconstruct the thermal radiation field image I' from the temperature information T according to the Stefan-Boltzmann law. D Modeling the direct thermal emission component I without texture information D The original infrared image I is decomposed into direct thermal emission components I based on the signal source. D and environmental reflection component I E Two parts; S2. Subtract the thermal radiation field image I' from the original infrared image I. D Separate the part containing texture information I' E Modeling environment reflection component I E ; S3. Design an infrared-texture mapping model, using visible light images I. v Delving deeper into the parts containing texture information E The latent texture information is used to obtain the enhanced texture information I'. ET By separating redundant information in network G n From the part I' containing texture information E Extract redundant information that affects the representation of infrared texture information. En ; In S3, the infrared-texture mapping model is generated by generator G. T It consists of a discriminator D and a generator G. T The ResNet architecture is used, consisting of 9 residual blocks; the discriminator D is based on PatchGAN and consists of 5 convolutional layers. The redundancy separation network in S3 adopts the U-Net framework and consists of 8 downsampling and 8 upsampling convolutional layers; During the training of the infrared-texture mapping model in S3, a total loss function including adversarial loss, perceptual loss, pixel loss, reconstruction loss and style loss is used to calculate the loss. Adversarial loss is represented as: Perceptual loss is represented as: Among them, W i,j H represents the width of the j-th feature map in the i-th layer. i,j φ represents the height of the j-th feature map in the i-th layer. i,j The function represents the j-th feature map extracted from the i-th layer of the VGG network; Pixel loss is represented as: L pix =||I v -G T (I' E )||1; Reconstruction loss is represented as: L rec =||I' E -I″ E ||1, where I″ E =I' ET ×I' En , representing the enhanced texture information I' ET With redundant information I' En Reconstructed pseudo-thermal radiation image; Style loss is represented as: L style =||I v -G T (I v )||1; The total loss function is expressed as: L = λ1L G +λ2L VGG / i.j +λ3L pix +λ4L rec +λ5L style ; S4, Image I' of the thermal radiation field D and enhanced texture information I' ET The images are fused to obtain the enhanced infrared image I'. Image I' of thermal radiation field in S4 D and enhanced texture information I' ET To integrate, including: S4.1, Based on the thermal radiation field image I' D and enhanced texture information I' ET The adaptive weight w is calculated as follows: Among them, w PIQE w represents the PIQE weighted components. AG w represents the weighted components of AG. NIQE w represents the NIQE weighted components. STD Indicates the STD weight components; S4.

2. Using adaptive weights w, the enhanced infrared image I' is obtained through fusion, where I' = I' D ·w+I' ET ·(1-w).

2. The infrared image enhancement method according to claim 1, characterized in that, w PIQE w AG w NIQE and w STD The expressions are as follows:

Images