Different-emissivity infrared imaging correction method and system based on image enhancement technology

Abstract

The invention relates to a different-emissivity infrared imaging correction method and system based on an image enhancement technology, and the method comprises the steps: collecting a target infrared image, fitting a relation function between a gray value of the infrared image and a temperature value of the infrared image according to a temperature file of the target infrared image and a pixel value file of the target infrared image, and correcting the gray value of the infrared image according to the relation function; dividing the target infrared image into a target area and a reference area, fusing the features of the target area and the features of the reference area through a cross attention module to obtain fused features, and inputting the fused features into a convolutional neural network of a calibration model to obtain a target image; and taking the output of the last convolutional layer of the convolutional neural network as the input of a nonlinear iteration module, combining the target infrared image in the nonlinear iteration module to obtain an enhanced image, and finally obtaining an enhanced image temperature curve based on the enhanced image. And judging that the calibration temperature is qualified according to the Euclidean distance between the enhanced image temperature curve and the original image temperature curve.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method and system for correcting infrared imaging of different emissivities based on image enhancement technology. Background Art

[0002] Image emissivity is an important parameter that describes the radiation characteristics of an object. It reflects the ability of an object to radiate energy at a given temperature. Emissivity is closely related to the surface properties of an object (such as composition and structure), and under given temperature conditions, the emissivity of any object is numerically equal to its absorptivity. The range of emissivity is greater than zero and less than 1, and its value is related to factors such as the material, shape, surface roughness, concavity, degree of oxidation, color, and thickness of the object. In general, the amount of radiation energy received by the infrared temperature measuring device from an object is proportional to the emissivity of the object.

[0003] The existing infrared image temperature calibration of the target object only calibrates the distortion of the temperature of the target object, resulting in an inaccurate final calibration result, which makes the calibration temperature of the target object differ greatly from the actual temperature distribution of the target object. Summary of the invention

[0004] Based on this, it is necessary to provide a method and system for infrared imaging correction of different emissivity based on image enhancement technology, in order to solve the problem that the existing infrared image temperature calibration of the target object only calibrates the distortion of the target object temperature, resulting in the final calibration result being inaccurate, and there is a large difference between the calibrated temperature of the target object and the actual temperature distribution of the target object.

[0005] On the one hand, the present application provides a method for correcting infrared imaging with different emissivity based on image enhancement technology, comprising:

[0006] Collect multiple target infrared images, create a temperature file and a pixel value file of each target infrared image according to the temperature of each pixel point and the pixel value of each pixel point in each target infrared image, preprocess the temperature file and the pixel value file of each target infrared image to obtain the preprocessed temperature file and the preprocessed pixel value file corresponding to each target infrared image;

[0007] Based on the preprocessed temperature file and the preprocessed pixel value file, a mapping relationship between the preprocessed temperature file and the preprocessed pixel value file is established through a fitting function;

[0008] Creating a calibration model; the calibration model includes a segmentation network, a cross-attention module, a convolutional neural network, and a nonlinear iteration module;

[0009] Inputting multiple target infrared images into the calibration model in sequence, the calibration model enhances the image based on the input target infrared image and combines the features of cross attention, and outputs the enhanced image;

[0010] Convert all enhanced images into a continuous temperature data sequence, and define the temperature data sequence as an enhanced temperature data sequence;

[0011] generating an enhanced image temperature curve according to the enhanced temperature data sequence;

[0012] Restoring each temperature data in the enhanced temperature data sequence to the original temperature range to generate an original image temperature curve;

[0013] Calculate the Euclidean distance between the enhanced image temperature curve and the original image temperature curve, and judge whether the calibration temperature is qualified according to the Euclidean distance between the enhanced image temperature curve and the original image temperature curve;

[0014] If the calibration temperature is qualified, the enhanced image is output.

[0015] On the other hand, the present application also provides a different emissivity infrared imaging correction system based on image enhancement technology, and the different emissivity infrared imaging correction system based on image enhancement technology includes:

[0016] A collection component, through which an infrared image of a target is collected;

[0017] The processing component is communicatively connected with the acquisition component, and the processing component executes the infrared imaging correction method with different emissivity based on the image enhancement technology on the acquired infrared image.

[0018] The present application relates to a method and system for correcting infrared imaging with different emissivity based on image enhancement technology. The method and system for correcting infrared imaging with different emissivity based on image enhancement technology collect target infrared images, and fit the relationship function between the grayscale value of the infrared image and the temperature value of the infrared image according to the temperature file of the target infrared image and the pixel value file of the target infrared image. Then, the target infrared image is divided into a target area and a reference area through a calibration model. The features of the target area and the features of the reference area are fused through a cross-attention module of the calibration model to obtain fused features. Then, the fused features are input into a convolutional neural network of the calibration model, and the output of the last convolutional layer of the convolutional neural network is used as the input of a nonlinear iteration module. The enhanced image is obtained by fusion with the target infrared image in the nonlinear iteration module. Finally, an enhanced image temperature curve is obtained based on the enhanced image, and whether the calibration temperature is qualified is judged according to the Euclidean distance between the enhanced image temperature curve and the original image temperature curve. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 A flowchart of a method for correcting infrared imaging with different emissivity based on image enhancement technology is provided in one embodiment of the present application.

[0020] Figure 2 A schematic diagram of the structure of a different emissivity infrared imaging correction system based on image enhancement technology provided in one embodiment of the present application.

[0021] Figure 3 A comparison diagram of an original image and an enhanced image of a method for correcting infrared imaging with different emissivities based on image enhancement technology provided in one embodiment of the present application.

[0022] Figure 4 A schematic diagram of a real temperature curve and an estimated temperature curve of an object surface according to a different emissivity infrared imaging correction method based on image enhancement technology provided in one embodiment of the present application.

[0023] Figure 5 A temperature curve obtained by performing 8 iterations under different parameters in a different emissivity infrared imaging correction method based on image enhancement technology provided in one embodiment of the present application.

[0024] Figure 6 The temperature curves obtained at different iteration times and when the iteration parameter is 0.7 for a different emissivity infrared imaging correction method based on image enhancement technology provided in one embodiment of the present application.

[0025] Reference numerals:

[0026] 100. Collection component; 200. Processing component. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0028] The present application provides a method and system for correcting infrared imaging of different emissivities based on image enhancement technology.

[0029] like Figure 1 As shown, in one embodiment of the present application, a method for correcting infrared imaging with different emissivity based on image enhancement technology is provided, and the method for correcting infrared imaging with different emissivity based on image enhancement technology includes:

[0030] S100, collecting multiple target infrared images, creating a temperature file and a pixel value file of each target infrared image according to the temperature of each pixel point and the pixel value of each pixel point in each target infrared image, preprocessing the temperature file and the pixel value file of each target infrared image to obtain the preprocessed temperature file and the preprocessed pixel value file corresponding to each target infrared image.

[0031] Specifically, the S100 includes:

[0032] K101, create the initial image database.

[0033] K102, collect multiple target infrared images, and put all collected target infrared images into the initial image database.

[0034] K103, select a target infrared image from the initial image database.

[0035] K104, obtaining the temperature value and pixel value corresponding to each pixel point of the target infrared image, and creating a temperature file of the target infrared image and a pixel value file of the target infrared image based on the temperature value and pixel value corresponding to each pixel point of the target infrared image.

[0036] K105, normalize the temperature value of each pixel point in the temperature file of the target infrared image, and normalize the pixel value of each pixel point in the pixel value file of the target infrared image to obtain a normalized temperature file and a normalized pixel value file.

[0037] K106, return to K103, until each target infrared image in the initial image database is selected, and the temperature file and pixel value file of each target infrared image are obtained.

[0038] S200, based on the preprocessed temperature file and the preprocessed pixel value file, establishing a mapping relationship between the preprocessed temperature file and the preprocessed pixel value file through a fitting function.

[0039] Specifically, the S200 includes:

[0040] K107, create a plane rectangular coordinate system, take the normalized temperature file and the normalized pixel value file as the data basis, take the normalized pixel value file as the horizontal coordinate, take the normalized temperature file as the vertical coordinate, mark the point corresponding to each infrared image in the plane rectangular coordinate system, and generate a target infrared image scatter plot. The horizontal coordinate of the target infrared image scatter plot is the pixel value of the pixel point, and the vertical coordinate is the temperature value of the pixel point.

[0041] K108, based on the target infrared image scatter plot, each scatter point in the target infrared image scatter plot is connected in sequence, and the fitting function form is y=ax+b, and a and b are solved.

[0042] Specifically, y may be solved by the least square method to minimize the sum of squares of the residuals between the objective function value and the true value.

[0043] S300, creating a calibration model. The calibration model includes a segmentation network, a cross-attention module, a convolutional neural network, and a nonlinear iteration module.

[0044] S400, multiple target infrared images are sequentially input into a calibration model, and the calibration model enhances the image based on the input target infrared image and combines the features of cross attention to output the enhanced image.

[0045] S500, converting all enhanced images into a continuous temperature data sequence, and defining the temperature data sequence as an enhanced temperature data sequence.

[0046] S600: generating an enhanced image temperature curve according to the enhanced temperature data sequence.

[0047] S700, restoring each temperature data in the enhanced temperature data sequence to the original temperature range to generate an original image temperature curve.

[0048] S800, calculating the Euclidean distance between the enhanced image temperature curve and the original image temperature curve, and judging whether the calibration temperature is qualified according to the Euclidean distance between the enhanced image temperature curve and the original image temperature curve.

[0049] S900: If the calibration temperature is qualified, an enhanced image is output.

[0050] In this embodiment, by collecting the target infrared image, and according to the temperature file of the target infrared image and the pixel value file of the target infrared image, the relationship function between the grayscale value of the infrared image and the temperature value of the infrared image is fitted, and then the target infrared image is divided into a target area and a reference area through a calibration model, and the features of the target area and the features of the reference area are fused through the cross-attention module of the calibration model to obtain a fused feature, and then the fused feature is input into the convolutional neural network of the calibration model, and the output of the last convolution layer of the convolutional neural network is used as the input of the nonlinear iterative module, and the target infrared image is combined with the fusion in the nonlinear iterative module to obtain an enhanced image, and finally the enhanced image temperature curve is obtained based on the enhanced image, and the calibration temperature is qualified according to the Euclidean distance between the enhanced image temperature curve and the original image temperature curve.

[0051] In one embodiment of the present application, the S400 includes:

[0052] S410, extracting a mask of a target area and a mask of a reference area from the target infrared image through a segmentation network.

[0053] S420, in the target infrared image, performing mask processing on the mask of the target area and the mask of the reference area to obtain a mask image of the target infrared image.

[0054] S430, extracting a cross-attention feature that fuses the target area and the reference area in the mask image of the target infrared image.

[0055] S440, inputting the cross-attention features fused with the target area and the reference area into a convolutional neural network, and using the convolutional neural network to perform N consecutive convolutions on the cross-attention features fused with the target area and the reference area, to obtain a feature map output by the convolutional neural network after each convolution. N is a positive integer and N is greater than 1.

[0056] S450, input the mask image of the target infrared image into a nonlinear iteration module, use the nonlinear iteration module to iterate the mask image of the target infrared image for P consecutive times to enhance the characteristics of the target area in the mask image of the target infrared image, output the image after P consecutive iterations, and output the image after P consecutive iterations as an enhanced image.

[0057] In one embodiment of the present application, the S410 includes:

[0058] S411, selecting a target infrared image.

[0059] S412, inputting the target infrared image into the calibration model.

[0060] S413, segmenting and marking different components of each input target infrared image through the segmentation network of the calibration model to obtain a segmentation label map of each target infrared image.

[0061] S414, performing a first binarization process on the segmentation label map of each target infrared image, and extracting a mask of the target area from the target infrared image.

[0062] Specifically, the binarization of an image is to set the grayscale value of the pixels on the image to 0 or 255, that is, to make the entire image appear to be only black and white.

[0063] In this embodiment, the region with a gray value of 255 after the binarization process is recorded as the target region, and the region with a gray value of 0 after the binarization process is recorded as the reference region.

[0064] S415, performing a second binarization process on the segmentation label image of each target infrared image, and extracting a mask of the reference area from the target infrared image.

[0065] Optionally, the present embodiment is directed to the processing of a target containing two materials. When the target contains more types of materials, multiple binarization processes are required to segment each material contained in the target infrared image.

[0066] S416, putting the mask of the target area and the mask of the reference area into the initial image database.

[0067] S417, return to S411, until all target infrared images are processed.

[0068] Specifically, masking refers to using a selected image, graphic, or object to block the processed image (all or part) to control the area or process of image processing. The specific image or object used for covering is called a mask or template. In optical image processing, the mask can be a film, filter, etc. In digital image processing, the mask is a two-dimensional matrix array, and sometimes a multi-value image is also used.

[0069] In this embodiment, different materials are used as references, and each material contained in the input target infrared image is segmented and marked through the segmentation network of the calibration model, thereby obtaining a segmentation label map of the target infrared image. The segmentation label map identifies the specific locations of different material areas in the infrared image.

[0070] The segmentation label map is then binarized, thereby dividing the segmentation label map into a target area and a reference area according to the material to facilitate subsequent processing.

[0071] like Figure 3 As shown, in one embodiment of the present application, the S430 includes:

[0072] S431, selecting a mask of a target area or a mask of a reference area from an initial image database.

[0073] S432, masking the selected target area mask or reference area mask by formula 1 to obtain a mask image of the target area or a mask image of the reference area. The mask image of the target area or the mask image of the reference area is uniformly recorded as a mask image.

[0074] R(x,y)=I(x,y)ΛM R (x,y) Formula 1.

[0075] Where I(x,y) is the grayscale value of the pixel at the coordinate (x, y) of the infrared image, and M R(x, y) is the grayscale value of the pixel with coordinates (x, y) in the mask of the target area and the reference area, Λ is the bitwise AND expression symbol, and R(x, y) is the grayscale value of the pixel with coordinates (x, y) in the mask image of the target area or the mask image of the reference area.

[0076] S433, putting the mask image of the target area or the mask image of the reference area into the initial image database, and returning to select a mask of the target area or the mask of the reference area from the initial image database until all images in the initial image database are selected, thereby obtaining multiple mask images of the target area and multiple mask images of the reference area.

[0077] In this embodiment, the mask of the target area or the mask of the reference area obtained in the previous embodiment is processed by masking so that the image except the target area and the reference area becomes black, thereby retaining only the target area and the reference area in the infrared image.

[0078] like Figure 3 As shown, in one embodiment of the present application, the S440 includes:

[0079] S441, selecting a mask image of a target area and a mask image of a reference area from the initial image data.

[0080] S442, converting the mask image of the target area and the mask image of the reference area from the grayscale space to the YCbCr color space.

[0081] Specifically, YCbCr is a color space that is usually used in continuous image processing in movies or digital photography systems. CB and CR are the concentration offset components of blue and red, and Y is the so-called luminance, which represents the concentration of light and is nonlinear, using gamma correction encoding processing.

[0082] S443, obtaining the emissivity of the material of the component corresponding to the target area, and performing normalization adjustment on the target area using Formula 2.

[0083]

[0084] Among them, I' R1 is the normalized mask image of the target area, I R1 is the mask image of the target area after conversion to the YCbCr color space, ε R1 is the emissivity of the target area, and R1 is the reference symbol of the target area.

[0085] Specifically, tanh represents a hyperbolic tangent function, which is a type of hyperbolic function.

[0086] Like trigonometric functions, hyperbolic functions are also divided into six types: hyperbolic sine, hyperbolic cosine, hyperbolic tangent, hyperbolic cotangent, hyperbolic secant, and hyperbolic cosecant. Hyperbolic tangent function is one of them.

[0087] Similar to the tangent function, the hyperbolic tangent function is computationally equal to the ratio of the hyperbolic sine to the hyperbolic cosine, i.e., tanh(x)=sinh(x) / cosh(x). In this embodiment,

[0088] S444, obtaining the emissivity of the material of the component corresponding to the reference area, and performing normalization adjustment on the reference area using Formula 3.

[0089]

[0090] Among them, I' R2 is the normalized mask image of the reference area, I R2 is the mask image of the reference area after conversion to the YCbCr color space, ε R2 is the emissivity of the reference area, and R2 is the reference symbol of the reference area.

[0091] Specifically, if there are multiple materials, each material can be normalized by a formula similar to Formula 2 or Formula 3. R1 or ε R1 Just replace it with the emissivity of the corresponding material.

[0092] S445, downsampling the normalized mask image of the target area and the normalized mask image of the reference area four times respectively, extracting multi-scale features of the target area and multi-scale features of the reference area. The multi-scale features of the target area include feature map Q, and the multi-scale features of the reference area include feature map K and feature map V.

[0093] Specifically, the number of downsampling is set to 4 because the size of the downsampled image is smaller, thereby reducing the consumption of computing resources. The scaling ratio of 4 downsampling is more in line with the design requirements of the convolutional neural network, allowing the network to efficiently process smaller feature maps while maintaining sufficient feature information.

[0094] Multi-scale features refer to feature information extracted from different scales, which can be spatial scales, temporal scales, or other types of scales. By considering information at different scales, the data can be described more comprehensively, thereby improving the performance of the model. For example, in image processing, multi-scale features can help identify objects of different sizes, while in natural language processing, multi-scale features can capture information of different granularities in text.

[0095] S446, calculating the correlation between the target region and the reference region by using Formula 4 based on the multi-scale features of the target region and the multi-scale features of the reference region.

[0096]

[0097] in, is the correlation between the target area and the reference area, Q is the feature map of the target area, K and V are both feature maps of the reference area, is the scaling factor and T represents the transposed matrix.

[0098] Specifically, the softmax function is a normalized exponential function, which is a generalization of the logic function. It can "compress" a K-dimensional vector z containing any real number into another K-dimensional real vector σ(z), so that each element ranges between (0,1) and the sum of all elements is 1. This function is often used in multi-classification problems.

[0099] In this embodiment, since the reference area and the target area are made of different materials, the emissivity of the reference area and the emissivity of the target area are different, and the reference area and the target area can be normalized by the emissivity of different areas.

[0100] After normalization, the normalized image is downsampled 4 times to gradually reduce the resolution of the feature map. Specifically, downsampling the normalized image of the target area can reduce the resolution of the feature map Q, and downsampling the normalized image of the reference area can reduce the resolution of the feature map K and feature map V. During downsampling, the multi-scale features of the normalized image can be extracted, and the computational complexity of the attention module is reduced, reducing the amount of computation, thereby saving computing resources.

[0101] In one embodiment of the present application, the step S440 further includes:

[0102] S447, adding the correlation between the target area and the reference area as a convolution parameter to the convolution layer used for each convolution in the convolutional neural network, performing N consecutive convolutions according to Formula 5, and obtaining a feature map output by the convolutional neural network after each convolution.

[0103]

[0104] Among them, ReLU is the function calculation symbol of the rectified linear unit function, X is the input of the first convolutional layer, Y i is the output of the i-th convolution, Y j is the output of the jth convolution, W i is the weight of the i-th convolution, W j is the weight of the jth convolution, b1 is the bias of the first convolution layer, and b i is the bias of the i-th convolution, b j is the bias of the jth convolution, Concat is the symbol of the concatenation operation, and N is the total number of convolutions.

[0105] Specifically, the convolutional neural network in the present application includes seven convolutional layers. Each of the first six convolutional layers consists of 32 convolutional kernels, and the last convolutional layer consists of 8 convolutional kernels. The size of all convolutional kernels is 3×3, the step size is 1, and each convolution is followed by a ReLU activation function. The network retains and fuses information at different levels through jump connections, which helps to alleviate the problems of gradient disappearance and feature loss. Before the convolution of the fifth convolutional layer, the feature map of the fourth convolutional layer and the feature map of the third convolutional layer are spliced. Before the convolution of the sixth convolutional layer, the feature map of the fifth convolutional layer and the feature map of the second convolutional layer are spliced. The last convolutional layer convolution outputs an 8-channel feature map.

[0106] In this embodiment, the As the input of the first convolutional layer of the convolutional neural network, that is, Thus, Y1 is obtained, and then Y i As the input of the second convolutional layer, and the input of the previous convolutional layer is used as the input of the next convolutional layer in turn. In each convolutional layer, a bias is added based on the input of the convolutional layer to obtain the output of this convolutional layer, and the output of this convolutional layer is used as the input of the next convolutional layer, until the last convolutional layer, that is, the seventh convolutional layer, is reached, and the final feature map is output.

[0107] In one embodiment of the present application, the S450 includes:

[0108] An iteration parameter is obtained based on the feature map output after each convolution.

[0109] S451, inputting the normalized mask image of the target area into a nonlinear iteration module.

[0110] S452, nonlinear iteration is performed through Formula 6 and Formula 7, and the mask image of the target infrared image is continuously iterated for P times using a nonlinear iteration module, and the image after the continuous P times of nonlinear iteration is output, and the image after the continuous P times of nonlinear iteration is output as an enhanced image.

[0111] C0(x)=I' R1 Formula 6.

[0112] C n (x) = C n-1 (x)+θ n (x)C n-1 (x)(1-C n-1 (x)) Formula 7.

[0113] Among them, I' R1 is the normalized mask image of the target area, C0(x) is the initial data used in the first iteration, and C n (x) is the output of the nth nonlinear iteration, n is the number of iterations, θ n (x) is the parameter used in the pth nonlinear iteration. When performing the pth nonlinear iteration, the θ used in the iteration process n (x) is the iterative parameter obtained based on the feature map output after the z-th convolution, z = 1, 2, ..., N.

[0114] In this embodiment, the normalized mask image of the target area is used as the input of the nonlinear iteration module, so as to perform nonlinear iteration on the grayscale value of the input image to enhance the input image.

[0115] In the process of nonlinear iteration, when the result value of each nonlinear iteration is closer to 0 or 1, the degree of adjustment of the grayscale value is reduced. When the result value of each nonlinear iteration is in the middle value of 0-1, the contrast of the grayscale value can be enhanced.

[0116] Since the parameters of each iteration of the nonlinear iteration are derived from the output of the convolutional neural network, the enhancement network can adaptively adjust the enhancement according to the local features of the input image so that the grayscale values ​​of different areas are appropriately enhanced.

[0117] like Figure 5 and Figure 6 As shown, the more times of nonlinear iterations and the larger the parameters, the more obvious the enhancement effect on the input image. The maximum number of iterations set in this embodiment is 8 times, that is, n=8, but in the actual iteration process, there is no limit on the number of iterations.

[0118] In one embodiment of the present application, before S450, the following steps are included:

[0119] K410, trains the loss function of the image enhancement network in the nonlinear iterative module.

[0120] K420, the training of the loss function of the image enhancement network in the nonlinear iterative module includes:

[0121] K430, the alignment loss is calculated by Formula 8.

[0122]

[0123] Among them, L stat is the alignment loss, μ En is the mean of the target area of ​​the enhanced image, σ En is the mean square error of the target area of ​​the enhanced image, μ Fe is the mean of the reference region of the enhanced image, σ Fe is the mean square error of the reference region of the enhanced image.

[0124] K440, the probability density distance loss is calculated by formula 9.

[0125]

[0126] Among them, L hist is the probability density distance loss, H En is the probability distribution of the enhanced image target area, H Fe is the probability distribution of the reference region of the enhanced image.

[0127] Specifically, the goal of the probability density distance loss is to make the distribution of the grayscale value of the target area of ​​the enhanced image and the grayscale value of the reference area of ​​the enhanced image as similar as possible, so the difference between the grayscale value of the target area of ​​the enhanced image and the grayscale value of the reference area of ​​the enhanced image is quantified by KL divergence. KL divergence is an asymmetric measure for measuring the difference between two probability distributions.

[0128] K450, the loss function of the image enhancement network in the nonlinear iterative module is calculated by formula 10.

[0129] L total =L stat +L hist Formula 10.

[0130] Among them, L total is the loss function of the image enhancement network in the nonlinear iterative module, L stat is the alignment loss, L hist is the probability density distance loss.

[0131] In this embodiment, the loss function of the image enhancement network in the nonlinear iterative module is obtained by combining the alignment loss and the probability density distance loss, thereby generating an enhanced image that is relatively consistent with the reference area in multiple dimensions, thereby improving the overall visual quality of the input.

[0132] When the loss function of the image enhancement network in the nonlinear iterative module is obtained by combining the alignment loss and the probability density distance loss, the difference between the mean and mean square error of the enhanced target area and the mean and variance of the reference area is calculated through statistical alignment loss. The ultimate goal is to make the grayscale value distribution of the enhanced target area as similar as possible to the grayscale value distribution of the reference area.

[0133] In one embodiment of the present application, the S700 includes:

[0134] S701, according to the time shooting sequence, the enhanced images corresponding to the infrared images are sequentially calculated using formula 11 to calculate the real temperature of the object surface corresponding to each enhanced image.

[0135]

[0136] Where ε is the surface emissivity of the object being measured, T m is the measured temperature of the object being measured, T r is the true surface temperature of the object corresponding to each enhanced image, which is equivalent to the actual temperature of the measured object, T u is the background temperature, T a is the atmospheric temperature, n is a constant related to the sensor material, τ a is the atmospheric transmittance.

[0137] Specifically, in this embodiment, n=4.09.

[0138] The atmospheric transmittance τ is calculated by formula 12 and formula 13. a .

[0139]

[0140]

[0141] Among them, τ a is the atmospheric transmittance, ω is the condensation number of water, and the values ​​of ω at different temperatures are shown in Table 1. ω is the relative humidity of the environment, k(h ω ) is the atmospheric attenuation coefficient, and d is the distance between the infrared sensor and the object.

[0142] Table-1: Temperature-water condensation number comparison table

[0143]

[0144] S702, creating a temperature rectangular coordinate system, and creating a real temperature curve of the object surface by combining the real temperature of the object surface corresponding to each enhanced image.

[0145] S703, obtaining the estimated temperature of each infrared image based on the fitting function, and creating an estimated temperature curve in combination with the estimated temperature of each infrared image.

[0146] Specifically, the S703 includes:

[0147] S703a, randomly selecting an infrared image and an enhanced image of the infrared image.

[0148] S703b, randomly select an area of ​​size P*Q in the target area of ​​the infrared image. The selected area of ​​size P*Q is the pixel value area.

[0149] S703c, obtaining the pixel value of each pixel point in the pixel value area, and calculating the average pixel value of the pixel value area based on the pixel value of each pixel point, and taking the average pixel value of the pixel value area as the pixel value of the infrared image.

[0150] S703d, taking the pixel values ​​of the infrared image as input, and calculating the estimated temperature of the infrared image through a fitting function.

[0151] S703e, returning to randomly select an infrared image and an enhanced image of the infrared image until all infrared images are selected, and obtaining the estimated temperature of each infrared image.

[0152] S703f, inputting the estimated temperature of each infrared image into a temperature rectangular coordinate system according to the shooting time of each infrared image, and connecting each estimated temperature point in sequence, thereby creating an estimated temperature curve.

[0153] S704, calculating the Euclidean distance between the actual temperature curve of the object surface and the estimated temperature curve, and judging whether the actual temperature of the object surface is normal according to the Euclidean distance.

[0154] Specifically, the S704 includes:

[0155] S704a, setting a temperature threshold.

[0156] Specifically, the temperature threshold may be set to 3°C.

[0157] S704b, randomly selecting a real temperature point Q1 (x1, y1) and an estimated temperature point Q2 (x1, y2) on the same horizontal coordinate on the real temperature curve and the estimated temperature curve of the object surface.

[0158] S704c, calculate the Euclidean distance between the actual temperature point Q1 (x1, y1) and the estimated temperature point Q2 (x1, y2) using formula 14.

[0159]

[0160] Among them, Q1 is the real temperature point on the real temperature curve of the object surface, Q2 is the estimated temperature point on the estimated temperature curve, x1 is the horizontal coordinate of the real temperature point Q1, y1 is the vertical coordinate of the real temperature point Q1, x1 is the horizontal coordinate of the estimated temperature point X2, and y2 is the vertical coordinate of the estimated temperature point X2.

[0161] S704d, return to S704b until all points on the real temperature curve and the estimated temperature curve of the object surface are selected, and the Euclidean distance between each real temperature point on the real temperature curve of the object surface and the estimated temperature point on the estimated temperature curve at the same horizontal coordinate is obtained.

[0162] S704e, the Euclidean distances of all points are added to obtain the Euclidean distance between the actual temperature curve and the estimated temperature curve of the object surface.

[0163] S704f, determining whether the Euclidean distance between the actual temperature curve of the object surface and the estimated temperature curve is greater than or equal to a temperature threshold.

[0164] S704g: If the Euclidean distance between the actual temperature curve of the object surface and the estimated temperature curve is greater than or equal to the temperature threshold, it is considered that the temperature of the object surface is abnormal.

[0165] S704h: If the Euclidean distance between the actual temperature curve of the object surface and the estimated temperature curve is less than the temperature threshold, it is considered that the surface temperature of the object is normal.

[0166] In this embodiment, in order to ensure that the temperature of the enhanced image can truly reflect the actual temperature of the object surface, the object surface temperature is first corrected by the infrared sensor. Therefore, based on the infrared thermal radiation model, the actual surface temperature T of the object corresponding to each infrared image is calculated by the formula r .

[0167] like Figure 4 As shown, after calculating the real surface temperature of the object in the enhanced image corresponding to all infrared images, a temperature rectangular coordinate system is created. The horizontal coordinate of the temperature rectangular coordinate system is the shooting time of the infrared image, and the vertical coordinate of the temperature rectangular coordinate system is the temperature. According to the shooting time of each infrared image, the real surface temperature of the object in the enhanced image corresponding to the infrared image is input into the temperature rectangular coordinate system in sequence, and each point is connected in sequence to create a real temperature curve of the object surface.

[0168] Then, the fitting function obtained in step K108 is used to calculate the estimated temperature of each infrared image. After calculating the estimated temperature of each infrared image, the estimated temperature of the infrared image is input into the temperature rectangular coordinate system in sequence according to the shooting time of each infrared image, and each point is connected in sequence to create an estimated temperature curve.

[0169] Then, the Euclidean distance between the actual temperature curve and the estimated temperature curve of the object surface is calculated, and it is determined whether the sum of the Euclidean distances exceeds the temperature threshold. If the sum of the Euclidean distances does not exceed the temperature threshold, the target temperature is considered normal and an enhanced image is output.

[0170] like Figure 2 As shown, in one embodiment of the present application, a different emissivity infrared imaging correction system based on image enhancement technology is provided, which is applied to the different emissivity infrared imaging correction method based on image enhancement technology of the aforementioned embodiment, and the different emissivity infrared imaging correction system based on image enhancement technology includes an acquisition component 100 and a processing component 200.

[0171] The infrared image of the target is collected by the collection component 100. The collection component 100 is in communication with the processing component 200, and the processing component 200 performs the infrared imaging correction method for different emissivity based on the image enhancement technology described in the above embodiment on the collected infrared image.

[0172] In this embodiment, the infrared image of the target is collected by the acquisition component 100, and then all the collected infrared images are input into the processing component 200. The processing component 200 performs the different emissivity infrared imaging correction method based on image enhancement technology as described in the previous embodiment on the input infrared image.

[0173] The technical features of the above-described embodiments may be arbitrarily combined, and the execution order of the method steps is not limited. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0174] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for correcting infrared imaging with different emissivity based on image enhancement technology, characterized in that: The different emissivity infrared imaging correction method based on image enhancement technology includes: Collect multiple target infrared images, create a temperature file and a pixel value file of each target infrared image according to the temperature of each pixel point and the pixel value of each pixel point in each target infrared image, preprocess the temperature file and the pixel value file of each target infrared image to obtain the preprocessed temperature file and the preprocessed pixel value file corresponding to each target infrared image; Based on the preprocessed temperature file and the preprocessed pixel value file, a mapping relationship between the preprocessed temperature file and the preprocessed pixel value file is established through a fitting function; Creating a calibration model; the calibration model includes a segmentation network, a cross-attention module, a convolutional neural network, and a nonlinear iteration module; Inputting multiple target infrared images into the calibration model in sequence, the calibration model enhances the image based on the input target infrared image and combines the features of cross attention, and outputs the enhanced image; Convert all enhanced images into a continuous temperature data sequence, and define the temperature data sequence as an enhanced temperature data sequence; generating an enhanced image temperature curve according to the enhanced temperature data sequence; Restoring each temperature data in the enhanced temperature data sequence to the original temperature range to generate an original image temperature curve; Calculate the Euclidean distance between the enhanced image temperature curve and the original image temperature curve, and judge whether the calibration temperature is qualified according to the Euclidean distance between the enhanced image temperature curve and the original image temperature curve; If the calibration temperature is qualified, the enhanced image is output.

2. The method for correcting infrared imaging with different emissivity based on image enhancement technology according to claim 1, characterized in that: The method sequentially inputs a plurality of target infrared images into a calibration model, wherein the calibration model enhances the image based on the input target infrared image and combines the cross-attention features to output the enhanced image, including: The mask of the target area and the mask of the reference area are extracted from the target infrared image through the segmentation network; In the target infrared image, mask processing is performed on the mask of the target area and the mask of the reference area to obtain a mask image of the target infrared image; The mask image of the target infrared image is extracted by fusing the cross-attention features of the target area and the reference area; Input the cross-attention features that fuse the target area and the reference area into the convolutional neural network, and use the convolutional neural network to perform N consecutive convolutions on the cross-attention features that fuse the target area and the reference area to obtain a feature map output by the convolutional neural network after each convolution; N is a positive integer and N is greater than 1; The mask image of the target infrared image is input into a nonlinear iteration module, and the mask image of the target infrared image is iterated for P consecutive times using the nonlinear iteration module to enhance the characteristics of the target area in the mask image of the target infrared image, and the image after P consecutive iterations is output, and the image after P consecutive iterations is output as an enhanced image.

3. The method for correcting infrared imaging with different emissivity based on image enhancement technology according to claim 2, characterized in that: The method of extracting a mask of a target area and a mask of a reference area from a target infrared image by using a segmentation network comprises: Select a target infrared image; Inputting the target infrared image into the calibration model; The segmentation network of the calibration model is used to segment and mark different components of each input target infrared image, thereby obtaining a segmentation label map of each target infrared image; Perform the first binarization process on the segmentation label map of each target infrared image, and extract the mask of the target area from the target infrared image; Perform a second binarization process on the segmentation label map of each target infrared image, and extract the mask of the reference area from the target infrared image; Putting the mask of the target area and the mask of the reference area into the initial image database; Return to the step of selecting a target infrared image until all target infrared images are processed.

4. The method for correcting infrared imaging with different emissivity based on image enhancement technology according to claim 3 is characterized in that: The step of performing mask processing on the mask of the target area and the mask of the reference area in the target infrared image to obtain a mask image of the target infrared image includes: Selecting a mask of a target area or a mask of a reference area from an initial image database; The mask of the selected target area or the mask of the reference area is masked by formula 1 to obtain a mask image of the target area or a mask image of the reference area; the mask image of the target area or the mask image of the reference area is uniformly recorded as a mask image; R(x, y) = I(x, y) Λ M R (x, y) Formula 1; Where I(x,y) is the grayscale value of the pixel at the coordinate (x, y) of the infrared image, and M R (x, y) is the grayscale value of the pixel with coordinates (x, y) in the mask of the target area and the reference area, Λ is the bitwise AND symbol, and R(x, y) is the grayscale value of the pixel with coordinates (x, y) in the mask image of the target area or the mask image of the reference area; The mask image of the target area or the mask image of the reference area is placed in the initial image database, and a mask of the target area or a mask of the reference area is selected from the initial image database until all images in the initial image database are selected, thereby obtaining multiple mask images of the target area and multiple mask images of the reference area.

5. The method for correcting infrared imaging with different emissivity based on image enhancement technology according to claim 4, characterized in that: The mask image of the extracted target infrared image incorporates cross-attention features of the target area and the reference area, including: Selecting a mask image of a target area and a mask image of a reference area from the initial image data; Convert the mask image of the target area and the mask image of the reference area from the grayscale space to the YCbCr color space; Obtain the emissivity of the material of the component corresponding to the target area, and normalize and adjust the target area using Formula 2; Among them, I' R1 is the normalized mask image of the target area, I R1 is the mask image of the target area after conversion to the YCbCr color space, ε R1 is the emissivity of the target area, R1 is the reference symbol of the target area; Obtain the emissivity of the material of the component corresponding to the reference area, and perform normalization adjustment on the reference area using Formula 3; Among them, I' R2 is the normalized mask image of the reference area, I R2 is the mask image of the reference area after conversion to the YCbCr color space, ε R2 is the emissivity of the reference area, R2 is the reference symbol of the reference area; Downsampling the normalized mask image of the target area and the normalized mask image of the reference area four times respectively, extracting multi-scale features of the target area and multi-scale features of the reference area; the multi-scale features of the target area include a feature map Q, and the multi-scale features of the reference area include a feature map K and a feature map V; Based on the multi-scale features of the target area and the multi-scale features of the reference area, the correlation between the target area and the reference area is calculated by formula 4; in, is the correlation between the target area and the reference area, Q is the feature map of the target area, K and V are both feature maps of the reference area, is the scaling factor and T represents the transposed matrix.

6. The method for correcting infrared imaging with different emissivity based on image enhancement technology according to claim 5, characterized in that: The cross-attention features of the target area and the reference area are input into the convolutional neural network, and the cross-attention features of the target area and the reference area are continuously convolved N times by the convolutional neural network to obtain a feature map output by the convolutional neural network after each convolution, including: The correlation between the target area and the reference area is added as a convolution parameter to the convolution layer used for each convolution in the convolutional neural network. N consecutive convolutions are performed using Formula 5 to obtain the feature map output by the convolutional neural network after each convolution. Among them, ReLU is the function calculation symbol of the rectified linear unit function, X is the input of the first convolutional layer, Y i is the output of the i-th convolution, Y j is the output of the jth convolution, W i is the weight of the i-th convolution, W j is the weight of the jth convolution, b1 is the bias of the first convolution layer, and b i is the bias of the i-th convolution, b j is the bias of the jth convolution, Concat is the symbol of the concatenation operation, and N is the total number of convolutions.

7. The method for correcting infrared imaging with different emissivity based on image enhancement technology according to claim 6, characterized in that: The method comprises: inputting the mask image of the target infrared image into a nonlinear iteration module, performing P consecutive iterations on the mask image of the target infrared image using the nonlinear iteration module to enhance the characteristics of the target area in the mask image of the target infrared image, outputting an image after the P consecutive iterations, and outputting the image after the P consecutive iterations as an enhanced image, including: An iteration parameter is obtained based on the feature map output after each convolution; Input the normalized mask image of the target area into the nonlinear iteration module; Perform nonlinear iteration through Formula 6 and Formula 7, use the nonlinear iteration module to perform continuous P nonlinear iterations on the mask image of the target infrared image, output the image after continuous P nonlinear iterations, and output the image after continuous P nonlinear iterations as the enhanced image; C0(x)=I' R1 Formula 6; C n f(x) = C n-1 f(x) + θ n f(x)C n-1 f(x)(1 - C n-1 f(x)) Formula 7; Among them, I' R1 is the normalized mask image of the target area, C0(x) is the initial data used in the first iteration, and C n (x) is the output of the nth nonlinear iteration, n is the number of iterations, θ n (x) is the parameter used in the pth nonlinear iteration. When performing the pth nonlinear iteration, the θ used in the iteration process n (x) is the iterative parameter obtained based on the feature map output after the z-th convolution, z = 1, 2, ..., N.

8. The method for correcting infrared imaging with different emissivity based on image enhancement technology according to claim 7, characterized in that: Before inputting the mask image of the target infrared image into the nonlinear iteration module, performing P consecutive iterations on the mask image of the target infrared image using the nonlinear iteration module to enhance the characteristics of the target area in the mask image of the target infrared image, outputting the image after the P consecutive iterations, and outputting the image after the P consecutive iterations as the enhanced image, the method further includes: Train the loss function of the image enhancement network in the nonlinear iterative module; The training of the loss function of the image enhancement network in the nonlinear iterative module includes: The alignment loss is calculated by formula 8; Among them, L stat is the alignment loss, μ En is the mean of the target area of ​​the enhanced image, σ En is the mean square error of the target area of ​​the enhanced image, μ Fe is the mean of the reference region of the enhanced image, σ Fe is the mean square error of the reference region of the enhanced image; The probability density distance loss is calculated by formula 9; Among them, L hist is the probability density distance loss, H En is the probability distribution of the enhanced image target area, H Fe is the probability distribution of the reference region of the enhanced image; The loss function of the image enhancement network in the nonlinear iterative module is calculated by formula 10; L total =L stat +L hist Formula 10; Among them, L total is the loss function of the image enhancement network in the nonlinear iterative module, L stat is the alignment loss, L hist is the probability density distance loss.

9. The method for correcting infrared imaging with different emissivity based on image enhancement technology according to claim 8, characterized in that: The step of restoring each temperature data in the enhanced temperature data sequence to the original temperature range to generate the original image temperature curve includes: According to the time shooting sequence, the enhanced images corresponding to the infrared images are sequentially calculated using formula 11 to calculate the real temperature of the object surface corresponding to each enhanced image; Where ε is the surface emissivity of the object being measured, T m is the measured temperature of the object being measured, T r is the true surface temperature of the object corresponding to each enhanced image, which is equivalent to the actual temperature of the measured object, T u is the background temperature, T a is the atmospheric temperature, n is a constant related to the sensor material, and τa is the atmospheric transmittance; Create a temperature rectangular coordinate system, and create a real temperature curve of the object surface by combining the real temperature of the object surface corresponding to each enhanced image; Obtaining an estimated temperature of each infrared image based on the fitting function, and creating an estimated temperature curve by combining the estimated temperature of each infrared image; The Euclidean distance between the actual temperature curve of the object surface and the estimated temperature curve is calculated, and whether the actual temperature of the object surface is normal is determined based on the Euclidean distance.

10. A different emissivity infrared imaging correction system based on image enhancement technology, applied to the different emissivity infrared imaging correction method based on image enhancement technology described in claims 1 to 9, characterized in that: The different emissivity infrared imaging correction system based on image enhancement technology includes: A collection component, through which a target infrared image is collected; A processing component, wherein the acquisition component is communicatively connected with the processing component, and the processing component executes the different emissivity infrared imaging correction method based on image enhancement technology as described in claims 1 to 9 on the acquired infrared image.

Images