Method and system for correcting infrared images of different emissivity based on image enhancement techniques

Abstract

This application relates to a method and system for infrared imaging correction with different emissivity based on image enhancement technology. The method involves acquiring a target infrared image and fitting a relationship function between the grayscale value and temperature value of the infrared image based on its temperature and pixel value files. The target infrared image is then divided into a target region and a reference region. A cross-attention module is used to fuse the features of the target region and the reference region to obtain fused features. These fused features are then input into a convolutional neural network of a calibration model, with the output of the last convolutional layer serving as the input to a nonlinear iterative module. Within this module, the target infrared image is fused to obtain an enhanced image. Finally, the enhanced image's temperature curve is obtained based on this enhanced image. The Euclidean distance between the enhanced and original image temperature curves is used to determine if the calibration temperature is acceptable.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method and system for correcting infrared imaging with different emissivity based on image enhancement technology. Background Technology

[0002] Image emissivity is an important parameter describing the radiation characteristics of an object, reflecting its ability to radiate energy at a given temperature. Emissivity is closely related to the surface properties of an object (such as composition and structure), and under given temperature conditions, the emissivity of any object is numerically equal to its absorptivity. Emissivity ranges from zero to less than 1, and its value is related to factors such as the object's material, shape, surface roughness, unevenness, degree of oxidation, color, and thickness. Generally speaking, the amount of radiant energy received by an infrared thermometer from an object is directly proportional to the object's emissivity.

[0003] Existing infrared image temperature calibration for target objects only calibrates the distortion of the target object's temperature, resulting in inaccurate calibration results and a significant difference between the calibrated temperature and the actual temperature distribution of the target object. Summary of the Invention

[0004] Therefore, it is necessary to address the problem that existing infrared image temperature calibration for target objects only calibrates the distortion of the target object's temperature, resulting in inaccurate calibration results and a large difference between the calibrated temperature and the actual temperature distribution of the target object. To address this issue, a method and system for infrared imaging correction with different emissivity based on image enhancement technology should be provided.

[0005] On the one hand, this application provides a method for correcting infrared imaging with different emissivity based on image enhancement technology, including:

[0006] Multiple infrared images of the target are acquired. Based on the temperature and pixel value of each pixel in each infrared image, a temperature file and a pixel value file for each infrared image are created. The temperature file and pixel value file of each infrared image are preprocessed to obtain the preprocessed temperature file and preprocessed pixel value file corresponding to each infrared image.

[0007] Based on the preprocessed temperature file and the preprocessed pixel value file, a mapping relationship between the preprocessed temperature file and the preprocessed pixel value file is established through a fitting function;

[0008] Create a calibration model; the calibration model includes a segmentation network, a cross-attention module, a convolutional neural network, and a nonlinear iterative module;

[0009] Multiple target infrared images are sequentially input into the calibration model. The calibration model enhances the images based on the input target infrared images and incorporates cross-attention features, then outputs an enhanced image.

[0010] All enhanced images are converted into a continuous temperature data sequence, and the temperature data sequence is defined as the enhanced temperature data sequence.

[0011] An enhanced image temperature curve is generated based on the enhanced temperature data sequence;

[0012] Each temperature data point in the enhanced temperature data sequence is restored to its original temperature range to generate the original image temperature curve.

[0013] Calculate the Euclidean distance between the temperature curve of the enhanced image and the temperature curve of the original image, and determine whether the calibration temperature is qualified based on the Euclidean distance between the temperature curve of the enhanced image and the temperature curve of the original image.

[0014] If the calibration temperature is satisfactory, an enhanced image will be output.

[0015] On the other hand, this application also provides an infrared imaging correction system for different emissivity based on image enhancement technology, the infrared imaging correction system for different emissivity based on image enhancement technology includes:

[0016] An acquisition component is used to acquire infrared images of the target;

[0017] The processing component is communicatively connected to the acquisition component, and performs the infrared imaging correction method based on image enhancement technology on the acquired infrared image through the processing component.

[0018] This application relates to a method and system for infrared imaging correction with different emissivity based on image enhancement technology. The method for infrared imaging correction with different emissivity based on image enhancement technology acquires a target infrared image and fits a relationship function between the grayscale value and the temperature value of the infrared image based on the temperature file and pixel value file of the target infrared image. Then, through a calibration model, the target infrared image is divided into a target region and a reference region. The cross-attention module of the calibration model fuses the features of the target region and the reference region to obtain fused features. The fused features are then input into the convolutional neural network of the calibration model, and the output of the last convolutional layer of the convolutional neural network is used as the input of a nonlinear iterative module. In the nonlinear iterative module, the target infrared image is fused to obtain an enhanced image. Finally, the temperature curve of the enhanced image is obtained based on the enhanced image, and the Euclidean distance between the temperature curve of the enhanced image and the temperature curve of the original image is used to determine whether the calibration temperature is qualified. Attached Figure Description

[0019] Figure 1 This is a flowchart illustrating an infrared imaging correction method with different emissivity based on image enhancement technology, provided in an embodiment of this application.

[0020] Figure 2 This is a schematic diagram of the structure of an infrared imaging correction system with different emissivity based on image enhancement technology, provided in an embodiment of this application.

[0021] Figure 3 This is a comparison image of the original and enhanced images of an infrared imaging correction method with different emissivity based on image enhancement technology, provided in an embodiment of this application.

[0022] Figure 4 This is a schematic diagram of the actual temperature curve and the estimated temperature curve of an object surface using an infrared imaging correction method with different emissivity based on image enhancement technology, provided in an embodiment of this application.

[0023] Figure 5 The temperature curves obtained by performing eight iterations under different parameters for an infrared imaging correction method with different emissivity based on image enhancement technology, as provided in an embodiment of this application.

[0024] Figure 6 Temperature curves obtained at different iteration numbers and with an iteration parameter of 0.7 for an embodiment of this application of an infrared imaging correction method with different emissivity based on image enhancement technology.

[0025] Figure label:

[0026] 100. Acquisition component; 200. Processing component. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0028] This application provides a method and system for correcting infrared imaging with different emissivity based on image enhancement technology.

[0029] like Figure 1 As shown, in one embodiment of this application, a method for correcting infrared imaging with different emissivity based on image enhancement technology is provided. The method includes:

[0030] S100 acquires multiple target infrared images, and creates a temperature file and a pixel value file for each target infrared image based on the temperature and pixel value of each pixel in each target infrared image. It then preprocesses the temperature file and pixel value file for each target infrared image to obtain the preprocessed temperature file and preprocessed pixel value file corresponding to each target infrared image.

[0031] Specifically, S100 includes:

[0032] K101, Create the initial image database.

[0033] K102 acquires multiple infrared images of the target and places all acquired infrared images of the target into the initial image database.

[0034] K103 selects a target infrared image from the initial image database.

[0035] K104: Obtain the temperature value and pixel value corresponding to each pixel of the target infrared image, and create a temperature file and a pixel value file of the target infrared image based on the temperature value and pixel value corresponding to each pixel of the target infrared image.

[0036] K105 normalizes the temperature value of each pixel in the temperature file of the infrared image of the target and normalizes the pixel value of each pixel in the pixel value file of the infrared image of the target, thus obtaining the normalized temperature file and the normalized pixel value file.

[0037] K106, return to K103, until every target infrared image in the initial image database has been selected, and the temperature file and pixel value file of each target infrared image are obtained.

[0038] S200 establishes a mapping relationship between the preprocessed temperature file and the preprocessed pixel value file using a fitting function, based on the preprocessed temperature file and the preprocessed pixel value file.

[0039] Specifically, S200 includes:

[0040] K107. Create a Cartesian coordinate system. Using normalized temperature and pixel value files as data, with the normalized pixel value files as the x-axis and the normalized temperature files as the y-axis, mark the points corresponding to each infrared image in the Cartesian coordinate system to generate a target infrared image scatter plot. The x-axis of the target infrared image scatter plot represents the pixel value of each pixel, and the y-axis represents the temperature value of each pixel.

[0041] K108, based on the scatter plot of the target infrared image, connects each scatter point in the scatter plot of the target infrared image in sequence, and fits the function form y=ax+b, and solves for a and b.

[0042] Specifically, y can be solved using the least squares method to minimize the sum of squared residuals between the objective function value and the true value.

[0043] S300, Create a calibration model. The calibration model includes a segmentation network, a cross-attention module, a convolutional neural network, and a nonlinear iterative module.

[0044] The S400 sequentially inputs multiple target infrared images into the calibration model. Based on the input target infrared images, the calibration model combines cross-attention features to enhance the images and outputs an enhanced image.

[0045] S500 converts all enhanced images into a continuous temperature data sequence, and defines the temperature data sequence as an enhanced temperature data sequence.

[0046] S600, an enhanced image temperature curve is generated based on the enhanced temperature data sequence.

[0047] S700, restore each temperature data in the enhanced temperature data sequence to the original temperature range to generate the original image temperature curve.

[0048] S800 calculates the Euclidean distance between the enhanced image temperature curve and the original image temperature curve, and determines whether the calibration temperature is qualified based on the Euclidean distance between the enhanced image temperature curve and the original image temperature curve.

[0049] If the calibration temperature is within acceptable limits, the S900 will output an enhanced image.

[0050] In this embodiment, by acquiring a target infrared image and fitting a relationship function between the grayscale value and temperature value of the infrared image based on the temperature file and pixel value file of the target infrared image, the target infrared image is divided into a target region and a reference region using a calibration model. The features of the target region and the reference region are fused using the cross-attention module of the calibration model to obtain fused features. The fused features are then input into the convolutional neural network of the calibration model, and the output of the last convolutional layer of the convolutional neural network is used as the input of the nonlinear iterative module. The enhanced image is obtained by fusing the target infrared image within the nonlinear iterative module. Finally, the temperature curve of the enhanced image is obtained based on the enhanced image. If the Euclidean distance between the enhanced image temperature curve and the original image temperature curve is satisfactory, the calibration temperature is considered acceptable.

[0051] In one embodiment of this application, S400 includes:

[0052] S410 extracts the mask of the target region and the mask of the reference region from the infrared image of the target through a segmentation network.

[0053] S420: In the target infrared image, the mask of the target area and the mask of the reference area are masked to obtain the masked image of the target infrared image.

[0054] S430 extracts cross-attention features of the target region and the reference region from the masked image of the target infrared image.

[0055] S440 inputs the cross-attention features, which are fused from the target region and the reference region, into the convolutional neural network. The convolutional neural network then performs N consecutive convolutions on the fused cross-attention features, resulting in the feature map output by the convolutional neural network after each convolution. N is a positive integer and N is greater than 1.

[0056] S450: Input the masked image of the target infrared image into the nonlinear iteration module. Use the nonlinear iteration module to perform P consecutive iterations on the masked image of the target infrared image to enhance the characteristics of the target region in the masked image of the target infrared image. Output the image after P consecutive iterations as the enhanced image.

[0057] In one embodiment of this application, S410 includes:

[0058] S411, Select an infrared image of the target.

[0059] S412, input the target infrared image into the calibration model.

[0060] S413, the segmentation network of the calibration model is used to segment and label the different parts of each input target infrared image, so as to obtain the segmentation label map of each target infrared image.

[0061] S414 performs the first binarization process on the segmentation label map of each target infrared image to extract the mask of the target region from the target infrared image.

[0062] Specifically, image binarization is the process of setting the grayscale value of pixels in an image to 0 or 255, thus presenting the entire image as a visual effect with only black and white.

[0063] In this embodiment, the region with a grayscale value of 255 after binarization is designated as the target region, and the region with a grayscale value of 0 after binarization is designated as the reference region.

[0064] S415 performs a second binarization process on the segmentation label map of each target infrared image to extract the mask of the reference region from the target infrared image.

[0065] Optionally, this embodiment is a processing procedure for a target containing two materials. When the target contains more types of materials, multiple binarization processes are required to segment out each material contained in the infrared image of the target.

[0066] S416, put the mask of the target region and the mask of the reference region into the initial image database.

[0067] S417, return to S411, until all target infrared images have been processed.

[0068] Specifically, a mask refers to using a selected image, graphic, or object to occlude (fully or partially) an image being processed, thereby controlling the area or process of image processing. The specific image or object used for covering is called a mask or template. In optical image processing, masks can be films, filters, etc. In digital image processing, masks are two-dimensional matrix arrays, and sometimes multi-valued images are also used.

[0069] In this embodiment, different materials are used as a reference, and the segmentation network of the calibration model is used to segment and label each material contained in the input target infrared image, thereby obtaining a segmentation label map of the target infrared image. The segmentation label map identifies the specific location of different material regions in the infrared image.

[0070] The segmentation label image is then binarized to divide it into target and reference regions based on the material, facilitating subsequent processing.

[0071] like Figure 3 As shown, in one embodiment of this application, S430 includes:

[0072] S431, Select a mask for the target region or a mask for the reference region from the initial image database.

[0073] S432, the mask of the selected target area or the mask of the reference area is masked using Formula 1 to obtain the masked image of the target area or the masked image of the reference area. The masked image of the target area or the masked image of the reference area are collectively referred to as the mask image.

[0074] R(x,y)=I(x,y)ΛM R (x,y) Formula 1.

[0075] Where I(x,y) is the gray value of the pixel at coordinates (x,y) in the infrared image, and M... R(x,y) is the gray value of the pixel at coordinates (x,y) in the mask of the target region and the reference region, Λ is the bitwise AND expression symbol, and R(x,y) is the gray value of the pixel at coordinates (x,y) in the mask image of the target region or the mask image of the reference region.

[0076] S433: Place the mask image of the target region or the mask image of the reference region into the initial image database, and return to select a mask of the target region or the mask of the reference region from the initial image database, until all images in the initial image database have been selected, to obtain multiple mask images of the target region and multiple mask images of the reference region.

[0077] In this embodiment, the target area mask or reference area mask obtained in the previous embodiment is used to mask the image except for the target area and reference area, so that only the target area and reference area in the infrared image are retained.

[0078] like Figure 3 As shown, in one embodiment of this application, S440 includes:

[0079] S441, Select a mask image of the target region and a mask image of the reference region from the initial image data.

[0080] S442 converts the mask images of the target area and the reference area from grayscale space to YCbCr color space.

[0081] Specifically, YCbCr is a color space commonly used in continuous image processing in film or in digital photography systems. CB and CR represent the intensity shifts of blue and red, respectively. Y stands for luminance, representing the intensity of light and is non-linear, using gamma correction encoding.

[0082] S443, obtain the emissivity of the material of the component corresponding to the target area, and normalize the target area using Formula 2.

[0083]

[0084] Among them, I' R1 It is the normalized mask image of the target region, I R1 It is a mask image of the target area after conversion to the YCbCr color space, ε R1 R1 is the emissivity of the target region.

[0085] Specifically, tanh represents the hyperbolic tangent function, which is a type of hyperbolic function.

[0086] Like trigonometric functions, hyperbolic functions are also divided into six types: hyperbolic sine, hyperbolic cosine, hyperbolic tangent, hyperbolic cotangent, hyperbolic secant, and hyperbolic cosecant. The hyperbolic tangent function is one of them.

[0087] Similar to the tangent function, the hyperbolic tangent function is computationally equal to the ratio of the hyperbolic sine to the hyperbolic cosine, i.e., tanh(x) = sinh(x) / cosh(x). In this embodiment,

[0088] S444: Obtain the emissivity of the material of the component corresponding to the reference area, and normalize it using Formula 3.

[0089]

[0090] Among them, I' R2 It is the normalized mask image of the reference region, I R2 It is a mask image of the reference area after conversion to the YCbCr color space, ε R2 R1 is the emissivity of the reference region, and R2 is the symbol for the reference region.

[0091] Specifically, if multiple materials exist, each material can be normalized using a formula similar to Formula 2 or Formula 3. This simply requires adjusting ε in the formula. R1 or ε R1 Simply replace it with the emissivity of the corresponding material.

[0092] S445, the normalized masked image of the target region and the normalized masked image of the reference region are downsampled four times respectively to extract the multi-scale features of the target region and the multi-scale features of the reference region. The multi-scale features of the target region include feature map Q, and the multi-scale features of the reference region include feature map K and feature map V.

[0093] Specifically, the reason for setting the downsampling count to four is that the downsampled image size is smaller, thus reducing the consumption of computational resources. Furthermore, the scaling ratio of four downsampling counts better suits the design requirements of convolutional neural networks, enabling the network to process efficiently on smaller feature maps while preserving sufficient feature information.

[0094] Multi-scale features refer to feature information extracted from different scales, which can be spatial, temporal, or other types of scales. By considering information at different scales, data can be described more comprehensively, thereby improving model performance. For example, in image processing, multi-scale features can help identify targets of different sizes, while in natural language processing, multi-scale features can capture information of different granularities in text.

[0095] S446, based on the multi-scale features of the target region and the multi-scale features of the reference region, the correlation between the target region and the reference region is calculated using Formula 4.

[0096]

[0097] in, Q represents the correlation between the target region and the reference region, where Q is the feature map of the target region, and K and V are both feature maps of the reference region. is the scaling factor, and T represents the transpose matrix.

[0098] Specifically, the softmax function is a normalized exponential function, a generalization of the logistic function. It "compresses" a K-dimensional vector z containing arbitrary real numbers into another K-dimensional real vector σ(z), such that each element is in the range (0,1) and the sum of all elements is 1. This function is often used in multi-class classification problems.

[0099] In this embodiment, since the reference region and the target region are made of different materials, the emissivity of the reference region and the emissivity of the target region are different. Therefore, the reference region and the target region can be normalized by using the emissivity of different regions.

[0100] After normalization, the normalized image is downsampled four times to progressively reduce the resolution of the feature maps. Specifically, downsampling the normalized image of the target region reduces the resolution of feature map Q, while downsampling the normalized image of the reference region reduces the resolution of feature maps K and V. Downsampling also extracts multi-scale features from the normalized image and reduces the computational complexity of the attention module, thus saving computational resources.

[0101] In one embodiment of this application, step S440 further includes:

[0102] S447, the correlation between the target region and the reference region is added as a convolution parameter to the convolutional layer used in each convolution in the convolutional neural network, and N consecutive convolutions are performed by Equation 5 to obtain the feature map output by the convolutional neural network after each convolution.

[0103]

[0104] Where ReLU is the function computation notation for the corrected linear unit function, and X is the input to the first convolutional layer. Y i Y is the output of the i-th convolution. j W is the output of the j-th convolution. i W is the weight of the i-th convolution. j b is the weight of the j-th convolution, b1 is the bias of the first convolution, and b i b is the bias of the i-th convolution. j is the bias of the j-th convolution, Concat is the symbol for the concatenation operation, and N is the total number of convolutions.

[0105] Specifically, the convolutional neural network in this application comprises seven convolutional layers. The first six layers each consist of 32 convolutional kernels, and the last layer consists of 8 convolutional kernels. All convolutional kernels are 3×3 in size with a stride of 1. Each convolution is followed by a ReLU activation function. The network retains and fuses information from different layers through skip connections, which helps to alleviate the problems of gradient vanishing and feature loss. Before the fifth convolutional layer, the feature maps of the fourth and third convolutional layers are concatenated. Before the sixth convolutional layer, the feature maps of the fifth and second convolutional layers are concatenated. The final convolutional layer outputs an 8-channel feature map.

[0106] In this embodiment, the result obtained in step S446 As the input to the first convolutional layer of a convolutional neural network, i.e. Thus, Y1 was obtained, and then Y was... i The input of the first convolutional layer is used as the input of the second convolutional layer, and the input of the previous convolutional layer is used as the input of the next convolutional layer. In each convolutional layer, a bias is added based on the input of the convolutional layer to obtain the output of the convolutional layer. The output of the convolutional layer is used as the input of the next convolutional layer, until the last convolutional layer, i.e. the seventh convolutional layer, is reached, and the final feature map is output.

[0107] In one embodiment of this application, S450 includes:

[0108] An iteration parameter is obtained based on the feature map output after each convolution.

[0109] S451, input the normalized mask image of the target region into the nonlinear iteration module.

[0110] S452 performs nonlinear iteration using formulas 6 and 7, and uses the nonlinear iteration module to perform P consecutive nonlinear iterations on the masked image of the target infrared image, outputting the image after P consecutive nonlinear iterations, and using the image after P consecutive nonlinear iterations as the enhanced image output.

[0111] C0(x)=I' R1 Formula 6.

[0112] C n (x)=C n-1 (x)+θ n (x)C n-1 (x)(1-C n-1 Formula 7. (x)

[0113] Among them, I' R1 This is the normalized masked image of the target region, where C0(x) is the initial data used in the first iteration, and C... n (x) is the output of the nth nonlinear iteration, where n is the iteration number and θ is the value of θ. n (x) is the parameter used in the p-th nonlinear iteration. During the p-th nonlinear iteration, θ is the parameter used in the iteration process. n (x) are the iterative parameters obtained based on the feature map output after the z-th convolution, where z = 1, 2, ..., N.

[0114] In this embodiment, the normalized masked image of the target region is used as the input of the nonlinear iteration module, thereby performing nonlinear iteration on the grayscale value of the input image to enhance the input image.

[0115] During nonlinear iteration, the closer the result of each nonlinear iteration is to 0 or 1, the less the grayscale value is adjusted. However, when the result of each nonlinear iteration is in the middle range of 0-1, the contrast of the grayscale value can be enhanced.

[0116] Since the parameters of each iteration of the nonlinear iteration are derived from the output of the convolutional neural network, the enhancement network can adaptively adjust the enhancement based on the local features of the input image, so that the gray values ​​of different regions can be appropriately enhanced.

[0117] like Figure 5 and Figure 6 As shown, the more nonlinear iterations there are and the larger the parameters, the more obvious the enhancement effect on the input image. In this embodiment, the maximum number of iterations is set to 8, i.e., n=8. However, in the actual iteration process, no limit is placed on the number of iterations.

[0118] In one embodiment of this application, prior to S450, the following is included:

[0119] K410 is used to train the loss function of the image enhancement network in the nonlinear iterative module.

[0120] K420, the training of the loss function of the image enhancement network in the nonlinear iterative module includes:

[0121] K430, the alignment loss is calculated using Formula 8.

[0122]

[0123] Among them, L stat It is the alignment loss, μ En It is the mean of the target region in the enhanced image, σ En It is the mean square error of the target region in the enhanced image, μ Fe It is the mean of the reference region of the enhanced image, σ Fe It is the mean squared error of the reference region for enhancing the image.

[0124] K440, calculate the probability density distance loss using Formula 9.

[0125]

[0126] Among them, L hist It is the probability density distance loss, H En It enhances the probability distribution of the target region in the image, H Fe It is the probability distribution of the reference region of the enhanced image.

[0127] Specifically, the goal of probability density distance loss is to make the distribution of gray values ​​in the target region of the enhanced image as similar as possible to the distribution of gray values ​​in the reference region of the enhanced image. Therefore, KL divergence is used to quantify the difference between the gray values ​​in the target region of the enhanced image and the gray values ​​in the reference region of the enhanced image. KL divergence is an asymmetric measure used to measure the difference between two probability distributions.

[0128] K450, calculate the loss function of the image enhancement network in the nonlinear iterative module using Equation 10.

[0129] L total =L stat +L hist Formula 10.

[0130] Among them, L total L is the loss function of the image enhancement network in the nonlinear iterative module. stat It is the alignment loss, L hist It is probability density distance loss.

[0131] In this embodiment, the loss function of the image enhancement network in the nonlinear iterative module is obtained by combining alignment loss and probability density distance loss, thereby generating an enhanced image that is more consistent with the reference region in multiple dimensions and improving the overall visual quality of the input.

[0132] When combining alignment loss and probability density distance loss to obtain the loss function of the image enhancement network in the nonlinear iterative module, the alignment loss is obtained by calculating the difference between the mean and standard deviation of the enhanced target region and the mean and variance of the reference region. The ultimate goal is to make the gray value distribution of the enhanced target region as similar as possible to the gray value distribution of the reference region.

[0133] In one embodiment of this application, S700 includes:

[0134] S701 calculates the true surface temperature of the object corresponding to each enhanced image by formula 11, based on the time sequence of the infrared images.

[0135]

[0136] Where ε is the surface emissivity of the object being measured, and T m It is the measured temperature of the object being measured, T. r It is the true surface temperature of the object corresponding to each enhanced image, which is equivalent to the actual temperature of the object being measured, T. u It is the background temperature, T a It is the atmospheric temperature, n is a constant related to the sensor material, and τ a It is atmospheric transmittance.

[0137] Specifically, in this embodiment, n = 4.09.

[0138] The atmospheric transmittance τ is calculated using formulas 12 and 13. a .

[0139]

[0140]

[0141] Where, τ a ω is the atmospheric transmittance, and h is the condensation number of water. The values ​​of ω at different temperatures are shown in Table 1. ω It is the relative humidity of the environment, k(h) ω ) is the atmospheric attenuation coefficient, and d is the distance between the infrared sensor and the object.

[0142] Table 1: Temperature-Water Condensation Number Comparison Table

[0143]

[0144] S702 creates a temperature Cartesian coordinate system and combines the actual surface temperature of the object with the actual surface temperature corresponding to each enhanced image to create an actual surface temperature curve of the object.

[0145] S703 obtains the estimated temperature of each infrared image based on the fitting function, and creates an estimated temperature curve by combining the estimated temperature of each infrared image.

[0146] Specifically, S703 includes:

[0147] S703a, randomly select an infrared image and an enhanced image of that infrared image.

[0148] S703b: Randomly select a region of size P*Q within the target area of ​​the infrared image. The selected region of size P*Q is the pixel value region.

[0149] S703c: Obtain the pixel value of each pixel within the pixel value region, calculate the average pixel value of the pixel value region based on the pixel value of each pixel, and use the average pixel value of the pixel value region as the pixel value of the infrared image.

[0150] S703d uses the pixel values ​​of the infrared image as input and calculates the estimated temperature of the infrared image using a fitting function.

[0151] S703e returns a randomly selected infrared image and its enhanced image until all infrared images have been selected, and obtains the estimated temperature of each infrared image.

[0152] The S703f, based on the capture time of each infrared image, inputs the estimated temperature of each infrared image into a temperature Cartesian coordinate system, and connects each estimated temperature point in sequence to create an estimated temperature curve.

[0153] S704 calculates the Euclidean distance between the actual temperature curve and the predicted temperature curve of the object's surface, and determines whether the actual temperature of the object's surface is normal based on the Euclidean distance.

[0154] Specifically, S704 includes:

[0155] S704a, set the temperature threshold.

[0156] Specifically, the temperature threshold can be set to 3°C.

[0157] S704b, randomly select the actual temperature point Q1(x1, y1) and the estimated temperature point Q2(x1, y2) on the same horizontal axis on the actual temperature curve and the estimated temperature curve of the object surface.

[0158] S704c calculates the Euclidean distance between the actual temperature point Q1(x1, y1) and the estimated temperature point Q2(x1, y2) using Formula 14.

[0159]

[0160] Where Q1 is the actual temperature point on the true temperature curve of the object's surface, Q2 is the estimated temperature point on the estimated temperature curve, x1 is the x-coordinate of the actual temperature point Q1, y1 is the y-coordinate of the actual temperature point Q1, x1 is the x-coordinate of the estimated temperature point X2, and y2 is the y-coordinate of the estimated temperature point X2.

[0161] S704d, return to S704b, until all points on the actual temperature curve and the estimated temperature curve of the object surface have been selected, and obtain the Euclidean distance between each actual temperature point on the actual temperature curve of the object surface and the estimated temperature point on the same horizontal axis on the estimated temperature curve.

[0162] S704e: Sum the Euclidean distances of all points to obtain the Euclidean distance between the actual temperature curve and the predicted temperature curve of the object surface.

[0163] S704f determines whether the Euclidean distance between the actual temperature curve and the estimated temperature curve of an object's surface is greater than or equal to a temperature threshold.

[0164] S704g: If the Euclidean distance between the actual temperature curve and the predicted temperature curve of an object's surface is greater than or equal to the temperature threshold, then the object's surface temperature is considered abnormal.

[0165] S704h: If the Euclidean distance between the actual temperature curve and the estimated temperature curve of the object surface is less than the temperature threshold, the object surface temperature is considered normal.

[0166] In this embodiment, to ensure that the temperature of the enhanced image accurately reflects the actual temperature of the object's surface, the object's surface temperature is first corrected using an infrared sensor. Therefore, based on the infrared thermal radiation model, the true surface temperature T of the object in the enhanced image corresponding to each infrared image is calculated using a formula. r .

[0167] like Figure 4 As shown, after calculating the true surface temperature of the object in the enhanced image corresponding to all infrared images, a temperature rectangular coordinate system is created. The horizontal axis of the temperature rectangular coordinate system is the shooting time of the infrared image, and the vertical axis is the temperature. According to the order of the shooting time of each infrared image, the true surface temperature of the object in the enhanced image corresponding to the infrared image is sequentially input into the temperature rectangular coordinate system, and the points are connected sequentially to create the true surface temperature curve of the object.

[0168] Then, using the fitting function obtained in step K108, the estimated temperature of each infrared image is calculated. After calculating the estimated temperature of each infrared image, the estimated temperature of each infrared image is input into the temperature rectangular coordinate system according to the order of the shooting time of each infrared image, and the points are connected in sequence to create the estimated temperature curve.

[0169] Then, the Euclidean distance between the actual temperature curve and the predicted temperature curve of the object surface is calculated, and it is determined whether the sum of the Euclidean distances exceeds the temperature threshold. If the sum of the Euclidean distances does not exceed the temperature threshold, then the target temperature is considered normal, and the enhanced image is output.

[0170] like Figure 2 As shown, in one embodiment of this application, a different emissivity infrared imaging correction system based on image enhancement technology is provided, which is applied to the different emissivity infrared imaging correction method based on image enhancement technology in the aforementioned embodiment. The different emissivity infrared imaging correction system based on image enhancement technology includes an acquisition component 100 and a processing component 200.

[0171] The acquisition component 100 acquires infrared images of the target. The acquisition component 100 is communicatively connected to the processing component 200, which performs the infrared imaging correction method based on image enhancement technology described in the foregoing embodiments on the acquired infrared images.

[0172] In this embodiment, the infrared image of the target is acquired by the acquisition component 100, and then all the acquired infrared images are input into the processing component 200. The processing component 200 performs the infrared imaging correction method based on image enhancement technology with different emissivity as described in the previous embodiment on the input infrared images.

[0173] The technical features of the above embodiments can be combined arbitrarily, and the execution order of the method steps is not restricted. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0174] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for correcting infrared imaging with different emissivity based on image enhancement technology, characterized in that, The infrared imaging correction method based on image enhancement technology with different emissivity includes: Multiple infrared images of the target are acquired. Based on the temperature and pixel value of each pixel in each infrared image, a temperature file and a pixel value file for each infrared image are created. The temperature file and pixel value file of each infrared image are preprocessed to obtain the preprocessed temperature file and preprocessed pixel value file corresponding to each infrared image. Based on the preprocessed temperature file and the preprocessed pixel value file, a mapping relationship between the preprocessed temperature file and the preprocessed pixel value file is established through a fitting function; Create a calibration model; the calibration model includes a segmentation network, a cross-attention module, a convolutional neural network, and a nonlinear iterative module; Multiple target infrared images are sequentially input into the calibration model. The calibration model enhances the images based on the input target infrared images and incorporates cross-attention features, then outputs an enhanced image. All enhanced images are converted into a continuous temperature data sequence, and the temperature data sequence is defined as the enhanced temperature data sequence. An enhanced image temperature curve is generated based on the enhanced temperature data sequence; Each temperature data point in the enhanced temperature data sequence is restored to its original temperature range to generate the original image temperature curve. Calculate the Euclidean distance between the temperature curve of the enhanced image and the temperature curve of the original image, and determine whether the calibration temperature is qualified based on the Euclidean distance between the temperature curve of the enhanced image and the temperature curve of the original image. If the calibration temperature is satisfactory, an enhanced image will be output.

2. The infrared imaging correction method with different emissivity based on image enhancement technology according to claim 1, characterized in that, The process involves sequentially inputting multiple target infrared images into a calibration model. The calibration model, based on the input target infrared images and incorporating cross-attention features, enhances the images and outputs an enhanced image, including: The target region mask and reference region mask are extracted from the target infrared image using a segmentation network; In the target infrared image, the mask of the target area and the mask of the reference area are masked to obtain the masked image of the target infrared image; The mask image extracted from the infrared image of the target incorporates cross-attention features of the target region and the reference region; The cross-attention features that fuse the target region and the reference region are input into the convolutional neural network. The convolutional neural network performs N consecutive convolutions on the cross-attention features that fuse the target region and the reference region, and obtains the feature map output by the convolutional neural network after each convolution; N is a positive integer and N is greater than 1; The masked image of the target infrared image is input into the nonlinear iteration module. The nonlinear iteration module performs P consecutive iterations on the masked image of the target infrared image to enhance the characteristics of the target region in the masked image of the target infrared image. The image after P consecutive iterations is output as the enhanced image.

3. The infrared imaging correction method with different emissivity based on image enhancement technology according to claim 2, characterized in that, The step of extracting the mask of the target region and the mask of the reference region from the target infrared image through a segmentation network includes: Select an infrared image of the target; Input the target infrared image into the calibration model; The segmentation network of the calibration model is used to segment and label different parts of each input target infrared image, resulting in a segmentation label map for each target infrared image. The segmentation label map of each target infrared image is first binarized to extract the mask of the target region from the target infrared image; A second binarization process is performed on the segmentation label map of each target infrared image to extract the mask of the reference region from the target infrared image; Place the mask of the target region and the mask of the reference region into the initial image database; Return to the selected target infrared image, and continue until all target infrared images have been processed.

4. The infrared imaging correction method with different emissivity based on image enhancement technology according to claim 3, characterized in that, The process of masking the target region and the reference region in the target infrared image to obtain a masked image of the target infrared image includes: Select a mask for the target region or a mask for the reference region from the initial image database; The mask of the selected target area or the mask of the reference area is masked using Formula 1 to obtain the masked image of the target area or the masked image of the reference area; the masked image of the target area or the masked image of the reference area are collectively referred to as the masked image. Formula 1: in, It is the grayscale value of the pixel with coordinates (x, y) in the infrared image. It is the grayscale value of the pixel with coordinates (x, y) in the mask of the target region and the reference region. It is the symbol for bitwise AND. It is the gray value of the pixel with coordinates (x, y) in the mask image of the target region or the mask image of the reference region; The mask image of the target region or the mask image of the reference region is placed into the initial image database, and a mask of the target region or the mask of the reference region is selected from the initial image database. This process continues until all images in the initial image database have been selected, resulting in multiple mask images of the target region and multiple mask images of the reference region.

5. The infrared imaging correction method with different emissivity based on image enhancement technology according to claim 4, characterized in that, The masked image from which the target infrared image is extracted incorporates cross-attention features of the target region and the reference region, including: Select a mask image of the target region and a mask image of the reference region from the initial image data; Convert the mask images of the target region and the reference region from grayscale space to YCbCr color space; Obtain the emissivity of the material of the component corresponding to the target area, and normalize the target area using Formula 2; Formula 2: in, It is the normalized mask image of the target region. It is a mask image of the target area after conversion to the YCbCr color space. R1 is the emissivity of the target region, and R1 is the symbol representing the target region. Obtain the emissivity of the material of the component corresponding to the reference area, and normalize it using Formula 3 for the reference area; Formula 3: in, It is the normalized mask image of the reference region. It is a mask image of the reference area after conversion to the YCbCr color space. R1 is the emissivity of the reference region, and R2 is the symbol for the reference region. The normalized masked image of the target region and the normalized masked image of the reference region are downsampled four times to extract the multi-scale features of the target region and the reference region. The multi-scale features of the target region include feature map Q, and the multi-scale features of the reference region include feature map K and feature map V. Based on the multi-scale features of the target region and the multi-scale features of the reference region, the correlation degree between the target region and the reference region is calculated using Formula 4. Formula 4: in, Q represents the correlation between the target region and the reference region, where Q is the feature map of the target region, and K and V are both feature maps of the reference region. is the scaling factor, and T represents the transpose matrix.

6. The infrared imaging correction method with different emissivity based on image enhancement technology according to claim 5, characterized in that, The method involves inputting the cross-attention features, which fuse the target region and the reference region, into a convolutional neural network. The convolutional neural network then performs N consecutive convolutions on these cross-attention features to obtain the feature map output by the convolutional neural network after each convolution. This includes: The correlation between the target region and the reference region is added as a convolution parameter to the convolutional layer used in each convolution in the convolutional neural network. N consecutive convolutions are performed using Equation 5 to obtain the feature map output by the convolutional neural network after each convolution. Formula 5: Where ReLU is the function computation notation for the corrected linear unit function, X is the input of the first convolutional layer, and X = , This is the output of the i-th convolution. This is the output of the j-th convolution. These are the weights of the i-th convolution. These are the weights of the j-th convolution. It is the bias of the first convolutional layer. It is the bias of the i-th convolution. is the bias of the j-th convolution, Concat is the symbol for the concatenation operation, and N is the total number of convolutions.

7. The infrared imaging correction method with different emissivity based on image enhancement technology according to claim 6, characterized in that, The process involves inputting the masked image of the target infrared image into a nonlinear iteration module, performing P consecutive iterations on the masked image of the target infrared image to enhance the characteristics of the target region in the masked image, and outputting the image after P consecutive iterations as the enhanced image. This includes: An iteration parameter is obtained based on the feature map output after each convolution. Input the normalized masked image of the target region into the nonlinear iteration module; Nonlinear iteration is performed using formulas 6 and 7. The masking image of the target infrared image is subjected to P consecutive nonlinear iterations using the nonlinear iteration module. The image after P consecutive nonlinear iterations is output and used as the enhanced image output. Formula 6: Formula 7: in, It is the normalized mask image of the target region. This is the initial data used in the first iteration. This is the output of the nth nonlinear iteration, where n is the iteration number. These are the parameters used in the p-th nonlinear iteration. They are used during the p-th nonlinear iteration process. The iteration parameters are obtained based on the feature map output after the z-th convolution, where z = 1, 2, ..., N.

8. The infrared imaging correction method with different emissivity based on image enhancement technology according to claim 7, characterized in that, Before inputting the masked image of the target infrared image into the nonlinear iteration module, performing P consecutive iterations on the masked image of the target infrared image to enhance the characteristics of the target region in the masked image, and outputting the image after P consecutive iterations as the enhanced image, the process further includes: The loss function of the image enhancement network in the nonlinear iterative module is trained; Training the loss function of the image enhancement network in the nonlinear iterative module includes: Calculate the alignment loss using Formula 8; Formula 8: in, It is alignment loss. It is the mean of the target region in the enhanced image. It enhances the mean square error of the target region in the image. It is the mean of the reference region of the enhanced image. It is the mean square error of the reference region of the enhanced image; Calculate the probability density distance loss using Formula 9; Formula 9; in, It is probability density distance loss. It enhances the probability distribution of the target region in the image. It is the probability distribution of the enhanced image reference region; The loss function of the image enhancement network in the nonlinear iterative module is calculated using Equation 10. Formula 10; in, It is the loss function of the image enhancement network in the nonlinear iterative module. It is alignment loss. It is probability density distance loss.

9. The infrared imaging correction method with different emissivity based on image enhancement technology according to claim 8, characterized in that, The step of restoring each temperature data point in the enhanced temperature data sequence to its original temperature range and generating the original image temperature curve includes: Based on the shooting order, the enhanced images corresponding to the infrared images are calculated sequentially using Formula 11 to determine the true surface temperature of the object corresponding to each enhanced image. Formula 11; in, It is the surface emissivity of the object being measured. It is the measured temperature of the object being measured. It represents the true surface temperature of the object corresponding to each enhanced image, which is equivalent to the actual temperature of the object being measured. It's the background temperature. Where is the atmospheric temperature, and n is a constant related to the sensor material. It is atmospheric transmittance; Create a temperature Cartesian coordinate system, and combine the actual surface temperature of the object with the actual surface temperature corresponding to each enhanced image to create an actual surface temperature curve of the object. The estimated temperature of each infrared image is obtained based on the fitting function, and an estimated temperature curve is created by combining the estimated temperature of each infrared image. Calculate the Euclidean distance between the actual temperature curve and the predicted temperature curve of the object's surface, and determine whether the actual temperature of the object's surface is normal based on the Euclidean distance.

10. A system for correcting infrared imaging with different emissivity based on image enhancement technology, applied to the method for correcting infrared imaging with different emissivity based on image enhancement technology as described in any one of claims 1 to 9, characterized in that, The infrared imaging correction system based on image enhancement technology with different emissivity includes: Acquisition component, used to acquire infrared images of the target; A processing component, wherein the acquisition component is communicatively connected to the processing component, performs the infrared imaging correction method based on image enhancement technology according to any one of claims 1 to 9 on the acquired infrared image through the processing component.

Images