A compressed thermal infrared image temperature calculation method based on an improved Transformer architecture

Abstract

The application discloses a compressed thermal infrared image temperature calculation method based on an improved Transform architecture, and relates to the technical field of thermal infrared image processing and temperature calculation. The method comprises the following steps: constructing an improved Transform coding-decoding network, wherein the improved Transform coding-decoding network comprises a dynamic position coding module introduced in an encoder, a multi-scale feature fusion module embedded in a decoder, and a cross-level attention interaction layer arranged between a coding path and a decoding path; training the improved Transform coding-decoding network, wherein a loss function adopted in the training is a hybrid loss function; normalizing a compressed single-channel thermal infrared image and an adjacent frame infrared image thereof as original images to obtain a first image; inputting the first image into the trained improved Transform coding-decoding network to generate a first temperature map; and dynamically calibrating the first temperature map based on an image compression factor and an actual temperature map to generate a second temperature map. The application solves the technical problem of low temperature calculation precision in the prior art.

Description

Technical Field

[0001] This invention belongs to the field of thermal infrared image processing and temperature calculation technology, specifically relating to a method for calculating the temperature of compressed thermal infrared images based on an improved Transformer architecture. Background Technology

[0002] Accurate temperature calculations are crucial in many fields, including fire safety, building inspection, and healthcare. Thermal infrared imaging technology, due to its ability to acquire temperature distribution information from object surfaces non-contactly, has become an indispensable detection method in these fields. However, raw thermal infrared image data is enormous, facing bandwidth and capacity limitations during transmission and storage, thus typically requiring compression.

[0003] Compressed thermal infrared images inevitably lose some information, leading to a decrease in image quality and posing a significant challenge to subsequent accurate temperature calculations. Traditional temperature calculation methods have obvious shortcomings when processing compressed thermal infrared images. Some methods based on simple mathematical models fail to fully capture the complex temperature distribution characteristics and long-range dependencies in the image, making accurate temperature calculations difficult.

[0004] While traditional convolutional neural networks (CNNs) have made some progress in image processing, their limited local receptive fields restrict their ability to grasp global information in thermal infrared images. CNNs cannot effectively recover and compensate for problems such as image blurring and information loss caused by compression, thus affecting the accuracy of temperature calculations.

[0005] In recent years, the Transformer architecture has emerged in the field of computer vision. Its self-attention mechanism can capture global information, providing new ideas for image processing. However, directly applying the original Transformer architecture to temperature calculation of compressed thermal infrared images presents many problems. For example, traditional fixed-position encoding cannot adapt to the complex temperature distribution changes in thermal infrared images; there is a lack of effective fusion mechanisms for temperature features at different scales; the information exchange between encoding and decoding paths is insufficient; and during model training, there is a lack of loss functions and training strategies designed specifically for the characteristics of thermal infrared images, resulting in the model's inability to fully learn the temperature features of thermal infrared images and failing to meet the requirements for temperature calculation accuracy and stability in practical applications. Therefore, developing a compressed thermal infrared image temperature calculation method based on an improved Transformer architecture has significant practical implications. Summary of the Invention

[0006] The technical problem to be solved by the present invention is to provide a method for calculating temperature of compressed thermal infrared images based on an improved transformer-based architecture, so as to at least solve some of the above-mentioned technical problems.

[0007] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0008] A method for calculating temperature from compressed thermal infrared images based on an improved Transformer architecture includes the following steps:

[0009] Step 1: Construct an improved Transformer encoder-decoder network. The improved Transformer encoder-decoder network includes: a dynamic positional encoding module introduced in the encoder, a multi-scale feature fusion module embedded in the decoder, and a cross-level attention interaction layer set between the encoder-decoder paths.

[0010] Step 2: Train the improved Transformer encoder-decoder network. The loss function used for training is a hybrid loss function, which includes: a weighted L1 loss term for high-temperature regions, a loss term for temperature gradient structure similarity, and a L2 regularization term for model parameters.

[0011] Step 3: Take the compressed single-channel thermal infrared image and its adjacent frame infrared images as the original image, normalize the original image to obtain the first image;

[0012] Step 4: Input the first image into the trained improved Transformer encoder-decoder network to generate the first temperature map;

[0013] Step 5: Dynamically calibrate the first temperature map based on the image compression factor and the real temperature map to generate a single-channel second temperature map with the same resolution as the original image.

[0014] Furthermore, the calculation method of the dynamic position encoding module is as follows:

[0015] Calculate the temperature gradient magnitude between each pixel in the input image and its 8 neighboring pixels;

[0016] The temperature gradient magnitude is mapped to a location feature vector through a fully connected layer, and the dimension of the location feature vector is consistent with the embedding dimension of the Transformer.

[0017] The location feature vector is added to the pixel features of the input image to generate a feature map with dynamic location encoding.

[0018] Furthermore, the multi-scale feature fusion module is characterized by:

[0019] Three parallel dilated convolution branches, with dilation rates set to 1, 2, and 4 respectively, are used to extract temperature features from the input feature map;

[0020] A feature aggregation layer uses a 1×1 convolution to weight and fuse the output feature maps of three parallel dilated convolution branches to obtain a fused feature map.

[0021] A residual connection adds the fused feature map to the input feature map.

[0022] Furthermore, the cross-level attention interaction layer is implemented as follows:

[0023] The output features of the i-th layer of the encoder and the j-th layer input features of the decoder The interaction feature F is calculated using the cross-attention mechanism. inter :

[0024]

[0025] Where Q, K, and V are the query, key, and value mapping functions, respectively; Softmax is the activation function; T is the transpose; and d k For feature dimensions;

[0026] Interaction feature F inter Input features of the decoder The components are assembled by channel and then dimensionality is reduced using 1×1 convolution.

[0027] Furthermore, the hybrid loss function L total for:

[0028] L total =λ1L temp +λ2L grad +λ3L reg ;

[0029] Among them, L temp Weighted L1 loss for high-temperature regions, N is the number of samples, and i is the index of the sample. Let i be the predicted temperature value for the i-th sample. This represents the true temperature value of the i-th sample. The indicator function is the actual temperature value of the i-th sample. When it exceeds the set threshold, The function value is 1 if it is 1, otherwise it is 0.

[0030] L grad For temperature gradient structural similarity loss, To predict the temperature gradient, For the actual temperature gradient, It is a structural similarity index;

[0031] Lreg For the L2 regularization term of the model parameters, L is the total number of model parameters, Θ i Let λ1, λ2, and λ3 be the i-th parameter of the model, and let λ1 + λ2 + λ3 = 1, and λ1 ≥ 0.6.

[0032] Furthermore, dynamic calibration includes:

[0033] Coarse calibration: A nonlinear transformation is performed based on the compression factor CF. The nonlinear transformation formula is as follows:

[0034]

[0035] Among them, T coarse This is the temperature value after coarse calibration; T raw The temperature value of the first temperature graph; μ raw and σ raw These represent the mean and standard deviation of the temperature in the first temperature plot, respectively; CF is the compressibility factor; μ gt This is the average of the actual temperatures;

[0036] Precise calibration: Temperature range based on real temperature map [T] min T max Perform a piecewise linear mapping, the formula for which is:

[0037]

[0038] Among them, T final The temperature value is shown in the second temperature graph; T min and T max The minimum and maximum temperatures are represented by the actual temperature map; T coarse_min and T coarse_max These are the minimum and maximum temperatures on the first temperature graph; T mid This is the median temperature of the first temperature map.

[0039] Furthermore, in step 2, the training configuration includes: using AdamW as the optimizer and setting the initial learning rate to 2×10. -4 The cosine annealing strategy is used for dynamic adjustment; the batch size is set to 8, and the training rounds are 150.

[0040] Furthermore, in step 2, the training configuration also includes: training data: original uncompressed infrared images; real temperature maps corresponding to the original uncompressed infrared images, which are obtained by acquiring them through a calibrated thermal imager; compressed images obtained by compressing the original uncompressed infrared images at different compression ratios; the training data covers indoor scenes, outdoor scenes, targets of different materials, different temperature ranges, and dynamic / static targets, and includes fire infrared images.

[0041] Compared with the prior art, the present invention has the following beneficial effects:

[0042] This invention introduces a dynamic position encoding module into the encoder, replacing the traditional fixed position encoding, enabling the model to better adapt to temperature distribution variations in thermal infrared images. A multi-scale feature fusion module is embedded in the decoder, utilizing parallel dilated convolution branches to capture temperature features from different receptive fields. By dynamically fusing features through an attention mechanism, temperature features at different scales can be extracted, improving the model's ability to model complex temperature distributions. A cross-level attention interaction layer strengthens information exchange between the encoding and decoding paths, helping the model to better recover temperature information. Simultaneously, a hybrid loss function, including a high-temperature region weighted L1 loss term, a temperature gradient structure similarity loss term, and a model parameter L2 regularization term, is used for model training, improving the model's accuracy in calculating temperature from compressed thermal infrared images.

[0043] In this invention, coarse calibration uses a nonlinear transformation of the compression factor to initially correct temperature deviations caused by compression; fine calibration uses piecewise linear mapping to further refine the adjustment based on the actual temperature range, resulting in a final temperature value that is closer to the actual temperature and improving the accuracy of temperature calculation. Attached Figure Description

[0044] Figure 1 This is a flowchart of the steps of the present invention. Detailed Implementation

[0045] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0046] In the description of this invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0047] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; of course, they can also refer to a mechanical connection or an electrical connection; furthermore, they can refer to a direct connection, an indirect connection through an intermediate medium, or a connection within two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0048] like Figure 1 As shown, a method for calculating temperature from compressed thermal infrared images based on an improved transformer-based network includes the following steps:

[0049] Step 1: Construct an improved Transformer encoder-decoder network. The improved Transformer encoder-decoder network includes: a dynamic positional encoding module introduced in the encoder, a multi-scale feature fusion module embedded in the decoder, and a cross-level attention interaction layer set between the encoder-decoder paths.

[0050] Step 2: Train the improved Transformer encoder-decoder network. The loss function used for training is a hybrid loss function, which includes: a weighted L1 loss term for high-temperature regions, a loss term for temperature gradient structure similarity, and a L2 regularization term for model parameters.

[0051] Step 3: Take the compressed single-channel thermal infrared image and its adjacent frame infrared images as the original image, normalize the original image to obtain the first image;

[0052] Step 4: Input the first image into the trained improved Transformer encoder-decoder network to generate the first temperature map;

[0053] Step 5: Dynamically calibrate the first temperature map based on the image compression factor and the real temperature map to generate a single-channel second temperature map with the same resolution as the original image.

[0054] This embodiment introduces a dynamic position encoding module into the encoder to replace the traditional fixed position encoding, enabling the model to better adapt to the temperature distribution changes in thermal infrared images. By embedding a multi-scale feature fusion module in the decoder, temperature features at different scales can be extracted, improving the model's ability to model complex temperature distributions. The cross-level attention interaction layer strengthens the information interaction between the encoding and decoding paths, helping the model to better recover temperature information and thus improving the accuracy of temperature calculation.

[0055] In some embodiments, the dynamic position encoding module calculates the position as follows:

[0056] Calculate the temperature gradient magnitude between each pixel in the input image and its 8 neighboring pixels;

[0057] The temperature gradient magnitude is mapped to a location feature vector through a fully connected layer, and the dimension of the location feature vector is consistent with the embedding dimension of the Transformer.

[0058] The location feature vector is added to the pixel features of the input image to generate a feature map with dynamic location encoding.

[0059] The multi-scale feature fusion module includes:

[0060] Three parallel dilated convolution branches, with dilation rates set to 1, 2, and 4 respectively, are used to extract temperature features from the input feature map;

[0061] A feature aggregation layer uses a 1×1 convolution to weight and fuse the output feature maps of three parallel dilated convolution branches to obtain a fused feature map.

[0062] A residual connection adds the fused feature map to the input feature map.

[0063] The implementation method of the cross-level attention interaction layer is as follows:

[0064] For the i-th layer output features of the encoder and the j-th layer input features of the decoder The interaction feature F is calculated using the cross-attention mechanism. inter :

[0065]

[0066] Where Q, K, and V are the query, key, and value mapping functions, respectively; Softmax is the activation function; T is the transpose; and d k For feature dimensions;

[0067] Interaction feature F inter Input features of the decoder The components are assembled by channel and then dimensionality is reduced using 1×1 convolution.

[0068] This embodiment introduces a dynamic position encoding module into the encoder to replace the traditional fixed position encoding, enabling the model to better adapt to the temperature distribution changes in thermal infrared images. A multi-scale feature fusion module is embedded in the decoder, using parallel dilated convolution branches to capture temperature features from different receptive fields. By dynamically fusing features through an attention mechanism, temperature features at different scales can be extracted, improving the model's ability to model complex temperature distributions. A cross-level attention interaction layer strengthens information exchange between the encoding and decoding paths, helping the model to better recover temperature information and thus improving the accuracy of temperature calculations.

[0069] In some embodiments, the hybrid loss function L total for:

[0070] L total =λ1L temp +λ2L grad +λ3L reg ;

[0071] Among them, L temp Weighted L1 loss for high-temperature regions, N is the number of samples, and i is the index of the sample. Let i be the predicted temperature value for the i-th sample. Let i be the true temperature value of the i-th sample. The indicator function is the actual temperature value of the i-th sample. When the value exceeds the set threshold, The function value is 1 if it is not 0 otherwise; in scenarios such as fire early warning, Threshold can be dynamically set according to actual needs, for example, set to 400K.

[0072] L grad For temperature gradient structural similarity loss, To predict the temperature gradient, For the actual temperature gradient, It is a structural similarity index used to measure the degree of similarity between the predicted temperature gradient and the actual temperature gradient;

[0073] L reg For the L2 regularization term of the model parameters, L is the total number of model parameters, Θ i Let λ1, λ2, and λ3 be the i-th parameter of the model, and let λ1 + λ2 + λ3 = 1, and λ1 ≥ 0.6.

[0074] This embodiment improves the accuracy of temperature calculation for compressed thermal infrared images by using a hybrid loss function that includes a weighted L1 loss term for high-temperature regions, a loss term for temperature gradient structure similarity, and a regularization term for model parameters L2 for model training.

[0075] In some embodiments, dynamic calibration includes:

[0076] Coarse calibration: A nonlinear transformation is performed based on the compression factor CF. The nonlinear transformation formula is as follows:

[0077]

[0078] Among them, T coarse This is the temperature value after coarse calibration; T rawThe temperature value of the first temperature graph; μ raw and σ raw These represent the mean and standard deviation of the temperature in the first temperature plot, respectively; CF is the compressibility factor; μ gt This is the average of the actual temperatures;

[0079] Precise calibration: Temperature range based on real temperature map [T] min T max Perform a piecewise linear mapping, the formula for which is:

[0080]

[0081] Among them, T final The temperature value is shown in the second temperature graph; T min and T max The minimum and maximum temperatures are represented by the actual temperature map; T coarse_min and T coarse_max These are the minimum and maximum temperatures on the first temperature graph; T mid The value represents the median temperature of the first temperature map. In this embodiment, coarse calibration uses a nonlinear transformation based on the compression factor to initially correct temperature deviations caused by compression; fine calibration uses piecewise linear mapping to further refine the adjustment based on the true temperature range, resulting in a final temperature value that is closer to the true temperature and improving the accuracy of temperature calculation.

[0082] Preferably, in step 2, the training configuration includes: using AdamW as the optimizer and setting the initial learning rate to 2×10. -4 The training process is dynamically adjusted using a cosine annealing strategy. The batch size is set to 8, and the training rounds are 150. In step 2, the training configuration also includes: training data: original uncompressed infrared images; real temperature maps corresponding to the original uncompressed infrared images, which are obtained by acquiring them through a calibrated thermal imager; compressed images obtained by compressing the original uncompressed infrared images at different compression ratios; the training data covers indoor scenes, outdoor scenes, targets of different materials, different temperature ranges, and dynamic / static targets, and includes fire infrared images.

[0083] Finally, it should be noted that the above embodiments are merely preferred embodiments of the present invention used to illustrate the technical solutions of the present invention, and are not intended to limit the invention, nor are they intended to limit the patent scope of the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. These modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention. That is to say, any changes or refinements made to the main design concept and spirit of the present invention that are not of substantial significance, but whose technical problems are still consistent with the present invention, should be included within the protection scope of the present invention. In addition, the direct or indirect application of the technical solutions of the present invention to other related technical fields are similarly included within the patent protection scope of the present invention.

Claims

1. A method for calculating temperature from compressed thermal infrared images based on an improved Transformer architecture, characterized in that, Includes the following steps: Step 1: Construct an improved Transformer encoder-decoder network. The improved Transformer encoder-decoder network includes: a dynamic positional encoding module introduced in the encoder, a multi-scale feature fusion module embedded in the decoder, and a cross-level attention interaction layer set between the encoder-decoder paths. Step 2: Train the improved Transformer encoder-decoder network. The loss function used for training is a hybrid loss function, which includes: a high-temperature region weighted L1 loss term, a temperature gradient structure similarity loss term, and a model parameter L2 regularization term. Step 3: Take the compressed single-channel thermal infrared image and its adjacent frame infrared images as the original image, normalize the original image to obtain the first image; Step 4: Input the first image into the trained improved Transformer encoder-decoder network to generate the first temperature map; Step 5: Dynamically calibrate the first temperature map based on the image compression factor and the real temperature map to generate a single-channel second temperature map with the same resolution as the original image. The calculation method of the dynamic position encoding module is as follows: Calculate the temperature gradient magnitude between each pixel in the input image and its 8 neighboring pixels; The temperature gradient magnitude is mapped to a location feature vector through a fully connected layer, and the dimension of the location feature vector is consistent with the embedding dimension of the Transformer. The position feature vector is added to the pixel features of the input image to generate a feature map with dynamic position encoding; The multi-scale feature fusion module includes: Three parallel dilated convolution branches, with dilation rates set to 1, 2, and 4 respectively, are used to extract temperature features from the input feature map; A feature aggregation layer uses a 1×1 convolution to weight and fuse the output feature maps of three parallel dilated convolution branches to obtain a fused feature map. A residual connection adds the fused feature map to the input feature map; The cross-level attention interaction layer is implemented as follows: For the encoder's first Layer output features and the decoder's Layer input features Interaction features are calculated through a cross-attention mechanism. : ； in, , , These are query, key-value mapping functions, respectively. For activation function, For transpose, For feature dimensions; Interaction features Input features of the decoder Concatenate the data by channel and reduce the dimensionality using 1×1 convolution; Dynamic calibration includes: Coarse calibration: based on compression factor CF The nonlinear transformation is performed, and the formula for the nonlinear transformation is: ； in, T coarse This is the temperature value after coarse calibration; T raw These are the temperature values ​​from the first temperature graph; μ raw and σ raw These are the temperature mean and standard deviation of the first temperature plot, respectively. CF It is the compression factor; μ gt This is the average of the actual temperatures; Precise calibration: Temperature range based on real temperature maps [ T min , T max Perform a piecewise linear mapping, the formula for which is: ； in, T final The temperature values ​​are from the second temperature chart. T min and T max These are the minimum and maximum temperatures of the actual temperature map; T coarse_min and T coarse_max These represent the minimum and maximum temperatures of the first temperature graph. T mid This is the median temperature of the first temperature map.

2. The method for calculating temperature from compressed thermal infrared images based on an improved Transformer architecture according to claim 1, characterized in that, Hybrid loss function for: ； in, Weighted L1 loss for high-temperature regions, , N For the sample size, For the index of the sample, For the first The predicted temperature value for each sample. For the first The true temperature value of each sample For indicator functions, when the first The true temperature value of each sample Greater than the set threshold hour, The function value is 1 if it is 1, otherwise it is 0. For temperature gradient structural similarity loss, , To predict the temperature gradient, For the actual temperature gradient, It is a structural similarity index; For the L2 regularization term of the model parameters, , This represents the total number of model parameters. For the model's first One parameter, , , These are the weighting coefficients. λ 1+ λ 2+ λ 3=1, and λ 1≥0.

6.

3. The method for calculating temperature from compressed thermal infrared images based on an improved Transformer architecture according to claim 1, characterized in that, In step 2, the training configuration includes: using AdamW as the optimizer and setting the initial learning rate to 2×10. −4 The cosine annealing strategy is used for dynamic adjustment; the batch size is set to 8, and the training rounds are 150.

4. The method for calculating temperature from compressed thermal infrared images based on an improved Transformer architecture according to claim 3, characterized in that, In step 2, the training configuration also includes: training data: raw uncompressed infrared images; real temperature maps corresponding to the raw uncompressed infrared images, which are obtained by a calibrated thermal imager; compressed images obtained by compressing the raw uncompressed infrared images at different compression ratios; the training data covers indoor scenes, outdoor scenes, targets of different materials, different temperature ranges and dynamic / static targets, and includes infrared images of fires.

Images