Thermal radiation characteristic guided infrared image basic model construction method

Abstract

The invention discloses a thermal radiation characteristic guided infrared image basic model construction method. The method comprises the following steps: 1) modeling key physical characteristics in thermal radiation generation and transmission and sensor imaging processes; 2) constructing a multi-scale thermal infrared image basic model by taking image reconstruction as a skeleton; 3) constructing a frequency domain feature auxiliary loss function to train the model; and 4) carrying out image feature extraction by using the trained model, and carrying out thermal infrared image segmentation, detection and depth estimation. According to the method, a neural radiation field is used for fitting a thermal radiation generation process, and emissivity and temperature of an object are extracted as supplements of image information; thermal radiation transmission loss is modeled by space-guided deformable cavity convolution, and an atmospheric transmission attenuation coefficient is extracted as image information correction; a multi-head attention mechanism is used for fitting sensor imaging characteristics, and a sensor thermal inertia coefficient is extracted to improve reconstruction precision; a low-pass filtering mechanism is introduced to promote attention of the network to semantics, and robustness and generalization performance are improved.

Description

Technical Field

[0001] This invention relates to a method for constructing a basic model of infrared images guided by thermal radiation characteristics, belonging to the fields of artificial intelligence and computer vision, and has broad application prospects in various systems based on thermal infrared images for segmentation, detection, depth estimation, etc. Background Technology

[0002] Thermal infrared imaging acquires images by detecting the thermal radiation of a target. It features all-weather operation, strong anti-interference capabilities, and good concealment, playing an important role in fields such as autonomous driving, precision guidance, and coastal and border defense. However, due to its imaging principle differing from visible light images, thermal infrared images suffer from low resolution, weak texture, and lack of detail, severely impacting segmentation and detection performance. In recent years, benefiting from the rapid development of self-supervised learning, the MoCo (Momentum Contrast) framework based on contrastive learning has significantly improved the discriminative ability of feature representations by constructing a dynamic dictionary and momentum update mechanism and mining semantic consistency features using a large amount of unlabeled data (see reference: He et al., Momentum contrast for unsupervised visual representation learning. IEEE Conference on Computer Vision and Pattern Recognition, 2020, 1234-1243. (He K, Fan H, Wu Y, et al. Momentum contrast for unsupervised visual representation learning[C] / / Proceedings of the IEEE / CVF conference on computer vision and pattern recognition. 2020: 1234-1243.)). The Masked Autoencoder (MAE) architecture proposed by He et al. forces the model to learn contextual information from the visible region by randomly occluding image regions, possessing strong feature modeling capabilities and significantly improving the performance of downstream tasks (see: He et al., Masked autoencoders are scalable vision learners. IEEE Conference on Computer Vision and Pattern Recognition, 2022, 2345-2354.). However, the above scheme is mainly designed for visible light images, and its direct application to thermal infrared images suffers from insufficient generalization ability and insensitivity to perceptual structures.

[0003] In the field of thermal infrared image analysis, physics-guided model building methods can effectively improve the robustness and generalization ability of feature representation by incorporating imaging physics mechanisms into model design. Saragadam et al. combined the non-uniformity physical model of thermal infrared sensors with deep network regularization in the DeepIR framework, decomposing sensor characteristics into scene-independent non-uniformity terms and scene-dependent radiative flux, thereby achieving high-quality denoising and super-resolution restoration (see: Saragadam et al., Thermal image processing via physics-inspired deep networks[C] / / Proceedings of the IEEE / CVF International Conference on ComputerVision. 2021: 4057-4065.). The physical-driven diffusion model proposed by Yue et al. introduces atmospheric transmission and thermal radiation constraints into the visible-to-thermal-infrared image conversion, making the generated results significantly superior to traditional style transfer methods in terms of physical consistency (see reference: Yue et al., Dif-fusion: Toward high color fidelity in infrared and visible image fusion based on diffusion models. IEEE Transactions on Image Processing, 2023, 32: 5705-5720.).For segmentation and detection tasks, Chen et al. used Meta AI's Segment Anything Model (SAM) to generate thermal infrared pseudo-labels and performed distillation pre-training on a large-scale SATIR dataset, significantly improving the mask segmentation accuracy of thermal infrared images (see: Chen et al., Thermal Infrared Image Segmentation Based on SAM Distillation. arXiv Preprint, 2023.). The Thermal3D-GS method proposed by Chen et al. explicitly models the physical processes of atmospheric transmission and heat conduction and incorporates temperature consistency constraints into the optimization objective, achieving a significant improvement in detail reconstruction and edge fidelity in the synthesis of new thermal infrared perspectives, providing a new approach for the coupled analysis of three-dimensional geometry and radiation characteristics. (See: Chen et al., Thermal3D-GS: Physics-induced 3D Gaussians for thermal infrared novel-view synthesis and 3D reconstruction. European Conference on Computer Vision, 2024, 253-269. These works demonstrate that integrating sensor physical models, atmospheric and thermal conduction mechanisms, and other thermal radiation characteristics into deep learning networks can not only compensate for the low contrast and weak texture of thermal infrared imaging, but also provide more discriminative feature representations for downstream tasks such as segmentation and detection. However, most of the above methods are designed for single downstream tasks and are difficult to apply to a wide range of downstream tasks.)

[0004] To address the problems existing in the aforementioned research, this invention proposes a method for constructing a basic infrared image model guided by thermal radiation features. In terms of structural design, a windowed occlusion strategy is introduced to extend the encoder into a multi-scale feature extraction module, and a frequency domain filtering enhancement module is designed to guide the model to focus more on key high-level semantic features in the image. Furthermore, this invention also encodes the key physical processes of thermal radiation in its generation, transmission, and sensor imaging into the network structure in formula form, constructing a thermal radiation feature guidance mechanism to guide the model to learn image feature representations in a more physically consistent manner. Summary of the Invention

[0005] The purpose of this invention is to address the aforementioned problems by designing a method for constructing a basic model of infrared images guided by thermal radiation features. To achieve more efficient and robust extraction of thermal infrared image features, a thermal radiation feature-guided mechanism is constructed. This mechanism combines key physical processes of radiation generation, transmission, and sensor imaging to guide the model in learning image feature representations in a more physically consistent manner.

[0006] To achieve this objective, the overall technical solution of this invention employs a self-supervised learning strategy based on a Masked Autoencoder. The image is randomly masked, and the unmasked portion is input into an encoder primarily composed of a Swin-Transformer network to extract image features. These features are then fed into a lightweight convolutional neural network for decoding, with the reconstruction of the masked portion as the learning objective for training. The technical approach proposed in this invention is mainly reflected in the following three aspects: 1) Design a window masking strategy to enable the encoder to extract multi-scale features, thereby taking into account both local details and global contextual information.

[0007] 2) In the encoder part, the Stefan Boltzmann law generated by thermal radiation and the Lambert-Beer law of atmospheric transmission are encoded by introducing a physical module. In the decoder, the thermal inertia effect of the thermal radiation sensor is introduced to jointly guide the network training.

[0008] 3) Introduce frequency domain filtering into the training loss and use the filtered image as the ground truth for reconstruction, so that the network pays more attention to information such as shape and semantics, thereby improving the robustness of feature extraction.

[0009] This invention relates to a method for constructing a basic model of an infrared image guided by thermal radiation characteristics. The specific steps of this method are as follows: Step 1: First, divide the thermal infrared image into segments. The pixel blocks are processed by adding sine and cosine position codes to each pixel block, and then 75% of the pixel blocks are randomly masked in the form of block masking, with the masked parts replaced by special markers.

[0010] Step 2: For the unmasked pixel block, the physical process of heat radiation generation and transmission is modeled using neural radiation field and spatial adaptive dilated convolution, respectively. The results are then superimposed on the initial pixel block through concatenation and multiplication.

[0011] Step 3: Input the processed pixel block into the multi-scale Swin-Transformer encoder network to extract features at four different scales. Then, input the smallest scale features and the special labels of the mask part into the multi-head attention mechanism network to fit the thermal inertia correction coefficient of the sensor imaging. Finally, input the corrected features into the decoder convolutional neural network.

[0012] Step 4: Construct a loss function to train the network.

[0013] Output: Using the trained base model as a feature extractor, connected to the existing segmentation, detection, and depth estimation decoder convolutional neural network, and fine-tuned on the corresponding task dataset, high-precision results of thermal infrared image semantic segmentation, object detection, and depth estimation can be obtained.

[0014] Step two is as follows: Step 2.1: The neural radiation field consists of two multilayer perceptrons (MLPs), whose inputs are the two-dimensional coordinates x and y of pixels, and whose outputs are predicted values ​​of temperature. and emissivity , recorded as ,in, These are pixel plane coordinates. and The neural radiation field parameters, corresponding to the fitted emissivity and temperature respectively, are combined with the Stefan-Boltzmann law of thermal radiation. in, This is the thermal radiation constant. The final output is the fitted thermal radiation intensity value. It is superimposed on the original pixel block in a splicing manner.

[0015] Step 2.2: Spatial Adaptive Dilated Convolution First, the spatial location encoding of the pixel is concatenated with the original pixel, and then input into a dilated convolution module with a dilation rate of 0 and 1 respectively. The outputs are the predicted transmission attenuation coefficients. and transmission distance d， Record Where I represents a pixel block and pos represents the spatial location code. and These correspond to the convolutional network parameters for fitting the transmission attenuation coefficient and the transmission distance, respectively. This is described using the Lamb-Beer law. The final decrease in thermal radiation intensity And the thermal radiation intensity predicted by the neural radiation field is multiplicatively added to it. The final thermal radiation intensity result after attenuation correction is obtained. ,Right now .

[0016] Step three is as follows: Step 3.1: Input the processed feature sequence into a Swin-Transformer-based multi-scale encoder. The encoder adopts a hierarchical architecture, containing four consecutive processing stages, each outputting the original... Figure 1 Feature maps at scales of 1 / 4, 1 / 8, 1 / 16, and 1 / 32. Each stage consists of multiple stacked Swin-Transformer blocks, with the base layer having a window size of 7, 8 multi-head attention heads, and a hidden layer dimension of 256. During feature extraction, a thermal radiation characteristic module is run concurrently, containing three parallel physical feature extraction branches: fitting the thermal radiation generation process using a neural radiation field to output temperature T and emissivity ε; modeling the thermal radiation transmission process using spatially adaptive dilated convolution to predict the transmission attenuation coefficient γ and transmission distance d; and fitting sensor imaging characteristics using a multi-head attention mechanism to extract thermal inertia correction coefficients. These physical features guide the encoder's image feature learning through feature concatenation and element-wise multiplication, respectively, enhancing the physical consistency of the features.

[0017] Step 3.2: The network for the multi-head attention mechanism consists of... It consists of stacked attention layers, whose input is the minimum scale feature. and mask special marker sequence The two are concatenated along the channel dimension and used as the input to the attention network. In each attention layer, through calculation Attention weights between internal elements are fused with mask position context information, and the output is the fused feature sequence. The input S' is fed into two multilayer perceptron (MLP) networks, whose output predictions are the historical radiation values ​​fitted by the microbolometer. and current radiation feature, .in, and The MLP network parameters are respectively fitted to historical radiation and current radiation. The representative decoder convolutional neural network, combined with a sensor thermal inertial imaging physical model, addresses the severe historical radiation superposition effect caused by sensor thermal inertia, which leads to the current radiation value of a pixel facing the final image reconstruction result. This can be represented as historical radiation. and current radiation Superposition:

[0018] in, , Indicates the start and end times of sensor exposure during the imaging process. The absorption rate of radiation by the thermal infrared camera sensor. , These represent the heat capacity and thermal conductivity of the thermal infrared camera sensor, respectively. These parameters are constants related to the thermal infrared camera and can be obtained through calibration. In the formula... It is the natural index.

[0019] Step 3.3: The decoder convolutional neural network adopts a symmetrical encoder-decoder structure, containing four upsampling stages. Each stage consists of a transposed convolutional layer, a group normalization layer, and a ReLU activation function. Specifically, the decoder convolutional neural network first performs a 2x upsampling through two 3×3 transposed convolutional layers, then refines features through two 3×3 residual blocks, and finally adjusts the number of channels through a 5×5 convolutional layer, outputting a reconstructed result of the same size as the original image. During the training phase, a frequency domain-assisted supervision mechanism is introduced: the original image is processed through a Gaussian low-pass filter to remove high-frequency noise components while retaining the main semantic structural information, serving as the supervision target for the reconstruction task.

[0020] Step four is as follows: The loss function uses the mean squared error (MSE) to calculate the difference between the predicted result and the filtered true value. The MSE loss function can be expressed by the formula:

[0021] in, These are the height and width of the output image, respectively. These are the two-dimensional coordinates of the pixels. This is the result obtained after performing Fourier transform, low-pass frequency domain filtering, and inverse Fourier transform on a real image. The Fourier transform can be achieved using the following formula:

[0022] in, These are frequency domain coordinates. It is an imaginary number. This is a real image. The resulting image is then subjected to a low-pass filter, using a Gaussian function, which can be expressed as:

[0023] in, The center of the frequency domain is used. Then, filtering is performed in the frequency domain using a dot product, which can be expressed by the following formula:

[0024] Finally, the filtered result is transformed back into an image using the following formula:

[0025] Training was performed using the AdamW optimizer, with an initial learning rate set to [value missing]. The weight decay was 0.05, a cosine annealing learning rate scheduling strategy was adopted, the batch size was 32, and a total of 300 training rounds were conducted.

[0026] The advantages and beneficial effects of this invention are as follows: This invention proposes a method for constructing a basic infrared image model guided by thermal radiation features. Using a Masked Autoencoder as the basic architecture, it introduces key physical processes of thermal infrared radiation generation, transmission, and imaging as guidance, enabling the network to focus on the physical characteristics of thermal radiation and extract more robust thermal infrared image features. Through a windowed masking strategy, a multi-scale encoder is constructed to better balance local details and global contextual information. By introducing frequency domain filtering into the loss function, the network focuses more on shape, semantic, and other information, improving the robustness of feature extraction. This invention can be combined with various thermal infrared image-based application systems, showing broad application prospects and value in various thermal infrared image-based segmentation, detection, and depth estimation systems. Attached Figure Description

[0027] Figure 1 This invention presents the basic structure and workflow of the infrared image basic model guided by thermal radiation characteristics.

[0028] Figure 2 The physical laws related to the thermal radiation characteristics utilized in this invention are as follows.

[0029] Figure 3 This is the structure of the basic model network proposed in this invention.

[0030] Figure 4 a, Figure 4 b、 Figure 4 c. Figure 4 d、 Figure 4 e Figure 4 f demonstrates the effectiveness of the present invention in tasks such as thermal infrared image segmentation, detection, and monocular depth estimation. Detailed Implementation

[0031] To better understand the technical solution of the present invention, the embodiments of the present invention will be further described below with reference to the accompanying drawings.

[0032] This invention is a method for constructing a basic model of infrared images guided by thermal radiation characteristics. Its algorithm framework and process are as follows: Figure 1 As shown, the specific implementation steps for each part are as follows: Step 1: First, the input thermal infrared image is divided into 7×7 pixel blocks. The position information of each block is marked using sine and cosine position coding. Then, 75% of the pixel blocks are randomly selected for masking. The masked parts are replaced with special markers, while the unmasked parts retain the original pixel values.

[0033] Specifically, step one is as follows: The input thermal infrared image is first divided into blocks using a fixed-size 7×7 pixel block partitioning strategy. This block size has been experimentally verified to achieve an optimal balance between preserving local detail features and computational efficiency. Each pixel block is spatially labeled using a two-dimensional sine and cosine positional encoding function, defined as:

[0034] in, Encoding spatial location, For encoding dimensions, This encoding method provides the model with absolute and relative positional information, enhancing its ability to perceive spatial structure. The encoded pixel blocks are transformed into 256-dimensional feature vectors through a linear projection layer, serving as input to the subsequent Swin-Transformer encoder. Then, 75% of all pixel blocks are randomly selected for masking; the masked portion is replaced with a learnable special label, while the unmasked portion retains the original projected feature vector. This high 75% masking ratio forces the model to learn rich contextual representations from limited visible information, effectively improving the model's generalization ability and feature extraction quality.

[0035] Step 2: Input the unmasked portion into the Swin-Transformer encoder. Then, using the radiation features extracted by the thermal radiation characteristics module, extract the key features for the thermal radiation generation, transmission, and sensor imaging stages, respectively. The corresponding process diagrams are shown below. Figure 2 As shown, the physical processes from left to right are thermal radiation generation, transmission, and sensor imaging, where thermal radiation generation is mainly caused by temperature. and emissivity The transmission process is primarily determined by the transmission attenuation coefficient γ and the transmission distance d, while sensor imaging is mainly determined by historical radiation. and current radiation The superposition of factors affects the final imaging result.

[0036] Step 3: The extracted physical features guide the image features of the encoder, which are then input into the decoder convolutional neural network to obtain the reconstructed result. The original image is then processed by a frequency domain filtering module, and the result is used as the ground truth predicted by the decoder convolutional neural network. The mean squared error loss of the reconstruction is calculated by comparing this result with the model's prediction, and this loss function is used as the network's training objective for iterative optimization. The corresponding model structure is as follows: Figure 3 As shown, during the encoding stage, the unmasked portion of the image is input into the neural radiation field, i.e., the MLP in the figure, to estimate the emissivity. and temperature Stefan-Boltzmann law after thermal radiation The fitted thermal radiation intensity value is output. The data is also input into the convolution module to fit the transmission attenuation coefficient γ and the transmission distance d, and to calculate the attenuation of the thermal radiation intensity. And the thermal radiation intensity predicted by the neural radiation field is multiplicatively added to it. The final thermal radiation intensity result after attenuation correction is obtained. During the decoding stage, the output features of the Swin-Transformer are used to estimate historical radiation through an attention mechanism. and current radiation After formula correction, the final reconstructed image is obtained. In the loss function calculation part, as shown in the figure below, the original image is processed by a Fourier transform and a low-pass filter, then inverse Fourier transform is performed back to the image, and the mean square error loss between the original image and the reconstructed image is calculated as a supervision signal.

[0037] Step 4: For the already trained thermal infrared image base model, the encoder part is used as a general feature extractor and connected to the task decoder convolutional neural network. The effectiveness of the model performance is verified on segmentation, detection, and depth estimation tasks. End-to-end fine-tuning is performed on downstream task datasets, with the learning rate set to [value missing]. Fine-tuning 50 rounds.

[0038] To visually demonstrate the effects of the present invention, Figure 4 a, Figure 4 b、 Figure 4 c. Figure 4 d、 Figure 4 e Figure 4 f demonstrates the effectiveness of the invention under different tasks, here Figure 4 a, Figure 4 b corresponds to the thermal infrared image segmentation task. Figure 4 'a' represents the original image input into the model. Figure 4 b represents the corresponding semantic segmentation result, which shows that the model accurately segmented the pedestrians, bicycles, and distant marker posts in the input image. Figure 4 c. Figure 4 d corresponds to the thermal infrared image target detection task. Figure 4 c represents the original image input into the model. Figure 4 d represents the corresponding target detection result, and it can be seen that the model accurately detected the car target in the image. Figure 4 e Figure 4 f corresponds to the result of the thermal infrared depth estimation task. Figure 4 e represents the original image input into the model. Figure 4 f represents the corresponding depth estimation result, showing that the model estimates the depth relationships of objects in the image. The figure demonstrates that this invention achieves excellent performance across different tasks.

Claims

1. A method for constructing a basic model of an infrared image guided by thermal radiation characteristics, characterized in that: The steps are as follows: Step one: the thermal infrared image is first divided into pixel blocks, and a sinusoidal position coding is attached to each pixel block. Then, 75% of the pixel blocks are randomly masked in the form of a block mask, and the masked parts are replaced by special markers. Step two: the panchromatic image is first divided into pixel blocks, and a sinusoidal position coding is attached to each pixel block. Then, 75% of the pixel blocks are randomly masked in the form of a block mask, and the masked parts are replaced by special markers. Step three: the thermal image is divided into pixel blocks, and a sinusoidal position coding is attached to Step two: for the pixel block not masked, the physical processes of thermal radiation generation and transmission are modeled by neural radiance field and spatially adaptive dilated convolution respectively, and the results are superimposed on the initial pixel block in the form of splicing and multiplication; Step three: the processed pixel block is input into a multi-scale Swin-Transformer encoder network for feature extraction, and then the smallest scale feature and the special mark of the mask part are jointly input into the network of multi-head attention mechanism to fit the thermal inertia correction coefficient of sensor imaging, and the corrected feature is input into the decoder convolutional neural network; Step four: construct a loss function to train the network; use the trained basic model as a feature extractor, connect it to the existing decoder convolutional neural network of segmentation, detection and depth estimation, and fine-tune it on the corresponding task dataset to obtain high-precision thermal infrared image semantic segmentation, target detection and depth estimation results.

2. The method of claim 1, wherein the method further comprises: determining a thermal radiation characteristic of the object; and determining a thermal radiation characteristic of the object based on the determined thermal radiation characteristic of the object. In step one, the input thermal infrared image is processed by block, and each pixel block is labeled by two-dimensional sine-cosine position coding. The coding function is defined as: ； wherein, is a spatial position encoding, is an encoding dimension, is a dimension index; the encoded pixel block is converted into a 256-dimensional feature vector through a linear projection layer, which is used as the input of the subsequent Swin-Transformer encoder; then 75% of all pixel blocks are randomly selected for mask processing, the masked part is replaced by a learnable special marker, and the unmasked part retains the original feature vector after projection.

3. The method of claim 1, wherein the method further comprises: determining a thermal radiation characteristic of the object; and determining a thermal radiation characteristic of the object based on the determined thermal radiation characteristic of the object. In step two, the neural radiance field is composed of two multi-layer perceptrons (MLPs), with inputs being the two-dimensional coordinates x and y of the pixels, and the predicted values of the outputs being the temperature and the emissivity , denoted as , where is the pixel plane coordinate, and correspond to the neural radiance field parameters of the fitted emissivity and temperature, respectively, combined with the Stefan-Boltzmann law of thermal radiation where is the thermal radiation constant; the fitted thermal radiation intensity values output are superimposed on the original pixel block in the form of splicing.

4. The method according to claim 1 or 3, characterized in that: In step two, the spatial adaptive hole convolution first encodes the spatial position of the pixel and splices the original pixel, and then inputs into the hole convolution module with hole rates of 0 and 1 respectively, and the output is the predicted transmission attenuation coefficient and transmission distance d， is denoted as , wherein I is a pixel block, pos is a spatial position encoding, and correspond to the neural radiance field parameters fitting the transmission attenuation coefficient and the transmission distance respectively; combined with the Lambert-Beer law, ; the final attenuation of the thermal radiation intensity is obtained and superimposed in the form of multiplication on the thermal radiation intensity predicted by the neural radiance field , to obtain the final thermal radiation intensity result after attenuation correction , that is .

5. The method of claim 1, wherein the method further comprises: determining a thermal radiation characteristic of the object; and determining a thermal radiation characteristic of the object based on the thermal radiation characteristic of the object and the thermal radiation characteristic of the object. In step three, the processed feature sequence is input into a multi-scale encoder based on Swin-Transformer; the encoder adopts a hierarchical architecture and contains 4 consecutive processing stages, which respectively output feature maps of 1 / 4, 1 / 8, 1 / 16 and 1 / 32 of the original image; each stage is stacked by multiple Swin-Transformer blocks, wherein the basic layer sets the window size to 7, the number of multi-head attention heads to 8, and the hidden layer dimension to 256; during feature extraction, a thermal radiation characteristic module is run synchronously, which contains three parallel physical feature extraction branches: a neural radiance field is used to fit the thermal radiation generation process, outputting temperature T and emissivity ε; a spatially adaptive dilated convolution is used to model the thermal radiation transmission process, predicting the transmission attenuation coefficient γ and the transmission distance d; a multi-head attention mechanism is used to fit the sensor imaging characteristics and extract the thermal inertia correction coefficient; these physical features guide the image feature learning of the encoder in the form of feature splicing and element multiplication, enhancing the physical consistency of the features.

6. The method according to claim 1 or 5, characterized in that: In step three, the network of multi-head attention mechanism is composed of stacked attention layers, with the input of the minimum scale feature and the mask special token sequence , both of which are concatenated in the channel dimension as the input of the attention network ; in each attention layer, the attention weight between the internal elements is calculated to fuse the mask position context information, and the output is the fused feature sequence ; input S' into two multi-layer perceptron (MLP) networks, and the output predicted value is the fitted historical radiation and current radiation of the microbolometer ; wherein, and respectively correspond to the MLP network parameters of the fitted historical radiation and current radiation, representing the decoder convolutional neural network, reconstructing the image result is represented as the superposition of the historical radiation and the current radiation : ； wherein, , denotes the start and end time of the sensor exposure during the imaging process, is the absorption rate of the thermal infrared camera sensor for radiation, , denote the heat capacity and the heat conduction of the thermal infrared camera sensor, respectively, is the natural exponential.

7. The method of claim 6, wherein the method further comprises: determining a thermal radiation characteristic of the object; and determining a thermal radiation characteristic of the object based on the determined thermal radiation characteristic of the object. In step three, the decoder convolutional neural network adopts a symmetrical encoder-decoder structure and contains 4 upsampling stages, each of which consists of a transpose convolution layer, a group normalization layer and a ReLU activation function; specifically, the decoder convolutional neural network first performs 2 times upsampling through 2 3x3 transpose convolution layers, then refines the features through 2 residual blocks with a convolution kernel size of 3x3, and finally adjusts the channel number through a 5x5 convolution layer to output a reconstruction result with the same size as the original image; in the training stage, a frequency domain auxiliary supervision mechanism is introduced: the original image is processed by a Gaussian low-pass filter to filter out high-frequency noise components and retain semantic structure information as the supervision target of the reconstruction task.

8. The method of claim 1, wherein the method further comprises: determining a thermal radiation characteristic of the object; and determining a thermal radiation characteristic of the object based on the thermal radiation characteristic of the object and the thermal radiation characteristic of the object. In step four, the loss function uses mean square error to calculate the difference between the predicted result and the filtered true value; wherein the mean square error loss function MSE is calculated by the formula: ； wherein, are respectively a height and a width of the output image, are respectively two-dimensional coordinates of the pixel, is a result obtained after Fourier transformation, low-pass frequency domain filtering, and inverse Fourier transformation of the real image. The loss function adopts mean square error to calculate the difference between the prediction result and the filtered true value; wherein, the mean square error loss function MSE is calculated by the following formula: ； wherein, are respectively a height and a width of the output image, are respectively two-dimensional coordinates of the pixel, is a result obtained by performing Fourier transformation, low-pass frequency domain filtering, and inverse Fourier transformation on the real image.

9. The method according to claim 1 or 8, wherein: In step four, the Fourier transformation is realized by the following formula: ； wherein is a frequency domain coordinate, is an imaginary number, is a real image; the result of the change is then low-pass filtered, here with a Gaussian function, expressed as: ； wherein is the frequency domain center; then filtering in the frequency domain by point multiplication, expressed by the following formula: ； Finally, the filtered result is inverse transformed back to the image, and the following formula is used to calculate: 。 10. The method of claim 1, wherein: In step four, the training uses the AdamW optimizer, sets the initial learning rate to , the weight decay is 0.05, the cosine annealing learning rate scheduling strategy is adopted, the batch size is 32, and a total of 300 rounds of training are performed.

Images