A method for restoring infrared image texture information based on thermal radiation imaging principle

Abstract

The present invention relates to a method for restoring infrared image texture information based on the principle of thermal radiation imaging, and belongs to the field of computer image processing methods. The method first reconstructs the thermal radiation field image based on the Stefan-Boltzmann thermal radiation law, combined with temperature information and the emissivity of different substances in the image; during the reconstruction process, Gabor filter denoising is applied to enhance image quality; then, drawing on Retinex theory, the generative adversarial network GAN and the U-Net network are used to obtain texture information and redundant information from the thermal radiation field image, respectively, to restore infrared texture information; finally, loss calculation is performed based on two loss functions, adversarial loss and pixel loss, to ensure the texture restoration effect. The present invention is based on the principle of thermal radiation and uses visible light to guide infrared image texture restoration, deeply excavating the potential texture information of infrared images and significantly improving the visual effect of infrared images.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method for restoring infrared image texture information based on the principle of thermal radiation imaging. Background Art

[0002] Infrared imaging technology can provide stable imaging results all day and night in low-light conditions such as nighttime, rain, and fog. Therefore, it is widely used in a variety of fields, including military, search and rescue, industry, agriculture, wildlife conservation, and medical diagnosis. However, due to the limitations of thermal radiation imaging technology, ordinary medium- and long-wave infrared images often lack texture information. Although short-wave infrared images can provide clear details and outlines, their short wavelength makes them susceptible to environmental factors such as water vapor and dust when propagating in the atmosphere, and they are also relatively expensive. Therefore, current medium- and long-wave infrared imaging devices face the challenge of lacking texture information, and a low-cost infrared imaging technology with clear texture is urgently needed. Improving image quality from a hardware perspective is quite difficult, but restoring texture details through infrared image enhancement algorithms is a feasible solution.

[0003] Currently, conventional infrared image enhancement techniques, such as those based on generative adversarial networks (GANs) and Transformers, can only enhance existing texture details and are unable to add or mine new texture information. In the past two years, some researchers have proposed methods that utilize visible light to guide infrared image enhancement. These methods, by integrating visible light information, further enhance texture details based on the original infrared image. However, this method requires data from both infrared and visible light modalities, making it of limited practical value. Existing infrared enhancement techniques rarely conduct in-depth analysis based on the principles of thermal imaging and struggle to effectively mine the underlying texture information in infrared images. Therefore, there is an urgent need to develop new infrared texture information recovery techniques to address these issues. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide an infrared texture restoration method guided by visible light based on thermal radiation imaging theory to solve the problem of insufficient texture information in medium and long wave infrared imaging technologies.

[0005] The technical solution of this invention relates to a method for restoring infrared texture information based on the principle of thermal radiation imaging. Specifically, the method first reconstructs the thermal radiation field image according to the Stefan-Boltzmann thermal radiation law. Then, a GAN is used to learn how to encode texture details in visible light images, thereby restoring the underlying texture information in the infrared image.

[0006] The specific steps of this technical solution are:

[0007] S1: The thermal radiation field image is reconstructed according to the Stefan-Boltzmann thermal radiation law. During the reconstruction process, Gabor filters are used for denoising, and the emissivity parameters are adjusted to enhance the contrast of the target information.

[0008] S2: Based on the Retinex theory, a texture restoration model is constructed, and the generative adversarial network GAN and U-Net network are used to obtain texture information and redundant information from the thermal radiation field image to perform infrared texture information restoration.

[0009] S3: During the model training process of the texture restoration model, the total loss function constructed based on the adversarial loss and pixel loss loss functions is used to perform loss calculation to ensure the texture restoration effect.

[0010] More specifically, the specific steps of reconstructing the thermal radiation field image according to the Stefan-Boltzmann thermal radiation law in S1 are as follows:

[0011] S1.1: Combined with the shooting environment parameters, temperature data is obtained from the infrared data. The temperature data is processed using a Gabor filter to remove noise introduced during thermal radiation transmission and imaging.

[0012] S1.2: Obtain the material category corresponding to different pixel points by performing semantic segmentation on the infrared image, determine the emissivity corresponding to the material category, and adjust the emissivity value to enhance the contrast of the infrared target.

[0013] S1.3: Reconstruct the infrared radiation field using the Stefan-Boltzmann law of thermal radiation, where the Stefan-Boltzmann law of thermal radiation is: , where E represents the radiant energy per unit area, measured in watts per square meter (W / m²), ε is the average emissivity of a non-blackbody, a positive number less than 1, representing the ratio of the radiation capacity of a non-blackbody to that of a blackbody. σ is the Stefan-Boltzmann constant, which is 5.67×10 −8 W / (m²·K 4 ). T is the absolute temperature, the unit is Kelvin (K).

[0014] S1.4: Use the Gabor filter again to eliminate the noise information generated or amplified during the thermal radiation field image reconstruction process.

[0015] Specifically, in S2 above, a texture restoration model is constructed based on the Retinex theory, and the generative adversarial network (GAN) and the U-Net network are used to obtain texture information and redundant information from the thermal radiation field image, respectively. The specific steps for infrared texture information restoration are as follows:

[0016] S2.1: Preprocess the thermal radiation field image and visible light image first:

[0017] S2.1.1: Adjust image contrast by dynamic histogram equalization;

[0018] S2.1.2: Adjust image brightness: Use a visible light image taken under normal lighting conditions as a reference image. Adjust the brightness of the thermal radiation field image and visible light image used for training to match the brightness of the reference image to eliminate the difference in visible light image brightness under different lighting conditions.

[0019] S2.1.3: Apply Gabor filter to remove noise from visible light images and enhance texture details of visible light images.

[0020] S2.2: Input the thermal radiation field image to the first generator At the same time, the visible light image is input into the discriminator D.

[0021] First Generator The ResNet structure is used, which consists of 9 residual blocks. After training, the first generator Capable of making thermal radiation field images Learn the texture encoding method of visible light images and output texture restoration images The discriminator D is built based on PatchGAN and consists of 5 convolutional layers.

[0022] S2.3: Input the thermal radiation field image into the second generator Separate the redundant information that affects the expression of texture information . The generator The U-Net framework is adopted, which consists of 8 downsampling and 8 upsampling convolutional layers.

[0023] To be more specific, during the model training process, the total loss function includes adversarial loss and pixel loss.

[0024] The adversarial loss is expressed as: , where I v is the visible light image, and I is the thermal radiation field image. It represents the judgment of the discriminator on the input. The output value is between 0 and 1. The value close to 1 represents real data, and the value close to 0 represents generated data. Represents the data generated by the generator. By generating adversarial losses, the generator and discriminator are continuously optimized to improve the model training effect.

[0025] The pixel loss is expressed as: . Represents the L1 norm. Pixel loss is used to constrain the infrared texture restoration image to be consistent with the visible light image at the pixel level.

[0026] The total loss function is expressed as: . represents the weight parameter of the adversarial loss, Represents the weight parameter of pixel loss. The training effect of the model is guaranteed by combining adversarial loss and pixel loss.

[0027] The beneficial effects of the present invention are as follows: (1) the present invention delves into the principle of thermal radiation imaging, reconstructs the radiation field based on the Stefan-Boltzmann method, effectively eliminates the noise introduced in the infrared imaging process through the Gabor filter, and enhances the contrast of the infrared target by adjusting the ε value; (2) the present invention guides infrared texture encoding by visible light, effectively recovering the potential texture information; (3) the present invention learns the texture expression of visible light through GAN instead of directly fusing visible light texture features, which can be used to achieve low-cost and clear-texture medium and long-wave infrared imaging. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 It is a flow chart of the method for restoring texture information of infrared images of the present invention;

[0029] Figure 2 This is the visualization result of the infrared image texture information restoration method of the present invention and other comparative methods. DETAILED DESCRIPTION

[0030] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments, but the protection scope of the present invention is not limited to the contents described above.

[0031] like Figure 1 As shown in FIG, a method for restoring infrared image texture information based on the principle of thermal radiation imaging is described, and the specific steps are as follows:

[0032] S1: The thermal radiation field image is reconstructed according to the Stefan-Boltzmann thermal radiation law. During the reconstruction process, Gabor filters are used for denoising, and the emissivity parameters are adjusted to enhance the contrast of the target information.

[0033] S1 specifically includes the following steps:

[0034] S1.1: Extract temperature data from infrared images and process the temperature data using Gabor filters to remove noise introduced during thermal radiation transmission and imaging.

[0035] S1.2: Obtain the material category corresponding to each pixel by performing semantic segmentation on the infrared image, determine the emissivity corresponding to the material category, and adjust the emissivity value to enhance the contrast of the infrared target;

[0036] S1.3: Reconstruct the infrared radiation field based on the Stefan-Boltzmann thermal radiation law.

[0037] The Stefan-Boltzmann law of thermal radiation is expressed as: , E represents the radiant energy per unit area, in watts per square meter (W / m²), ε is the average emissivity of a non-blackbody, which represents the ratio of the radiation capacity of a non-blackbody to that of a blackbody; σ is the Stefan-Boltzmann constant, which is 5.67×10−8 W / (m²·K 4 ), T is the absolute temperature, the unit is Kelvin (K).

[0038] S1.4: Use the Gabor filter again to eliminate the noise information generated or amplified during the thermal radiation field image reconstruction process.

[0039] S2: Based on the Retinex theory, a texture restoration model is constructed, and the generative adversarial network GAN and U-Net network are used to obtain texture information and redundant information from the thermal radiation field image to restore infrared texture information.

[0040] S2 specifically includes the following steps:

[0041] S2.1: First, perform the following three types of preprocessing operations on the thermal radiation field image and the visible light image:

[0042] S2.1.1: Adjust image contrast by dynamic histogram equalization;

[0043] S2.1.2: Adjust image brightness: Use a visible light image under normal lighting as a reference image. Adjust the brightness of the radiation field image and visible light image used for training to match the brightness of the reference image to eliminate brightness differences between visible light images under different lighting conditions.

[0044] S2.1.3: Use Gabor filter to remove noise information in visible light images and enhance texture details of visible light images;

[0045] S2.2: Input the thermal radiation field image to the first generator At the same time, the visible light image is input into the discriminator D, the first generator The ResNet structure is composed of 9 residual blocks. After training, it can enable the thermal radiation field image to learn the texture encoding method of the visible light image and output the texture restored image.

[0046] S2.3: Input the thermal radiation field image into the second generator Separate the redundant information that affects the expression of texture information, where the second generator The U-Net framework is adopted, which consists of 8 downsampling and 8 upsampling convolutional layers.

[0047] S3: During the model training process of the texture restoration model, the total loss function constructed based on the adversarial loss and pixel loss is used to calculate the loss to ensure the texture restoration effect.

[0048] During the model training process, considering the specific requirements of the task, two different loss functions were designed, namely adversarial loss and pixel loss.

[0049] Among them, the adversarial loss is expressed as , where I v is the visible light image, and I is the thermal radiation field image. The pixel loss is expressed as: The total loss function is expressed as: ,in, , .

[0050] Optimize the generator according to the loss function calculation results ( 、 ) and the discriminator The network parameters of the training model are adjusted to make them converge.

[0051] This example is implemented in the PyTorch framework. The initial learning rate is set to 0.001 and gradually decays after 20 rounds of training, reaching 0 after 160 epochs. The model is trained on a single NVIDIA RTX 4090 GPU, while testing is performed on a single NVIDIA RTX 4060 Ti GPU. 861 pairs of RGB-T images are randomly selected from the public ODinMJ dataset for training.

[0052] The present invention also makes a visual comparison between the experimental results of the proposed method and the classic histogram equalization method. Figure 2As shown in Figure 3 , the algorithm was further quantitatively evaluated using non-reference metrics such as average gradient, entropy enhancement, spatial frequency, natural image quality estimation, and perception-based image quality estimation. Average gradient (AG) and spatial frequency (SF) indicate image clarity; larger values ​​indicate clearer images. The entropy enhancement metric (EME) was used to evaluate the algorithm's performance in contrast improvement. A larger value indicates higher image contrast. The natural image quality evaluator (NIQE) metric measures the naturalness or realism of an image; smaller NIQE values ​​indicate higher image quality. The perception-based image quality evaluator (PIQE) metric calculates an image's quality score based on its block structure and noise characteristics; smaller PIQE values ​​indicate higher image quality. The evaluation results are shown in Table 1.

[0053] Depend on Figure 2 As can be seen, compared with histogram equalization, the proposed method restores more texture details in the image and provides better visual effects. Table 1 shows that for each of the evaluation indicators listed, the proposed method outperforms histogram equalization, demonstrating that the proposed method restores more texture details, achieves higher contrast, and achieves better naturalness and perceptual effects, outperforming histogram equalization in all aspects.

[0054] Table 1: Quantitative evaluation of infrared texture information recovery

[0055] method Average gradient Entropy-enhanced metrics Spatial frequency Natural image quality estimation Perception-based image quality estimation Original image 1.6183 1.2807 3.0736 17.8287 0.8854 Histogram equalization 3.5843 2.8052 6.82936 18.1241 60.3885 The proposed method 11.2272 5.11258 21.6713 11.9682 4.27784

[0056] The above describes the specific embodiments of the present invention in detail with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Various changes can be made within the knowledge of ordinary technicians in this field without departing from the scope of the present invention.

Claims

1. A method for restoring infrared image texture information based on the principle of thermal radiation imaging, characterized in that: The following steps are involved: S1: Reconstruct the thermal radiation field image according to the Stefan-Boltzmann thermal radiation law. During the reconstruction process, Gabor filter is used for denoising and the emissivity parameters are adjusted to enhance the contrast of the target information. S1 specifically includes the following steps: S1.1: Extract temperature data from infrared images and process the temperature data using Gabor filters to remove noise introduced during thermal radiation transmission and imaging. S1.2: Obtain the material category corresponding to each pixel by performing semantic segmentation on the infrared image, determine the emissivity corresponding to the material category, and adjust the emissivity value to enhance the contrast of the infrared target; S1.3: Reconstruct the infrared radiation field based on the Stefan-Boltzmann thermal radiation law; S1.4: Use the Gabor filter again to eliminate the noise information generated or amplified during the thermal radiation field image reconstruction process; S2: Based on the Retinex theory, a texture restoration model is constructed. The generative adversarial network (GAN) and the U-Net network are used to obtain texture information and redundant information from the thermal radiation field image to restore infrared texture information. S2 specifically includes the following steps: S2.1: First, perform the following three types of preprocessing operations on the thermal radiation field image and the visible light image: S2.1.1: Adjust image contrast by dynamic histogram equalization; S2.1.2: Adjust image brightness: Use a visible light image under normal lighting as a reference image. Adjust the brightness of the radiation field image and visible light image used for training to match the brightness of the reference image to eliminate brightness differences between visible light images under different lighting conditions. S2.1.3: Use Gabor filter to remove noise information in visible light images and enhance texture details of visible light images; S2.2: Input the thermal radiation field image to the first generator At the same time, the visible light image is input into the discriminator D, the first generator The ResNet structure is used, which can enable the thermal radiation field image to learn the texture encoding method of the visible light image after training, and output the texture restoration image; S2.3: Input the thermal radiation field image into the second generator Separate the redundant information that affects the expression of texture information, where the second generator Adopting U-Net framework; S3: During the model training process of the texture restoration model, the total loss function constructed based on the adversarial loss and pixel loss is used to calculate the loss to ensure the texture restoration effect.

2. The infrared image texture information restoration method according to claim 1, characterized in that: The adversarial loss is expressed as: , I v is the visible light image, I is the thermal radiation field image, Represents the discriminator's judgment on the input; Represents the data generated by the generator; pixel loss is expressed as: , represents the L1 norm; the total loss function is expressed as: , represents the weight parameter of the adversarial loss, Weight parameter representing pixel loss.

Images