Online compensation method and system for infrared thermal imaging temperature measurement error caused by water mist interference

Abstract

The present invention discloses a method and system for online compensation of infrared thermal imaging temperature measurement errors caused by water mist interference. The system simultaneously acquires infrared thermal images and visible light images, performs image registration based on their significant contour features and detailed edge features, and estimates the water mist transmittance in the visible light band from the registered visible light image based on the dark channel prior principle. Based on the Lambert-Beer law, the water mist transmittance in the infrared light band is determined from the water mist transmittance in the visible light band. Based on the infrared temperature measurement mechanism, a temperature compensation model is constructed to compensate for infrared temperature measurement errors caused by water mist. The actual temperature of the measured object is obtained based on the water mist transmittance in the infrared light band and the temperature compensation model. This method solves the technical problem of large infrared temperature measurement errors caused by water mist interference, accurately estimates the water mist transmittance in real time, and achieves precise, online compensation for infrared temperature measurement errors.

Description

Technical Field

[0001] The present invention mainly relates to the field of temperature measurement technology, and in particular to an online compensation method and system for infrared thermal imaging temperature measurement errors caused by water mist interference. Background Art

[0002] Infrared thermal imaging is a non-contact, online temperature measurement technology. It detects infrared radiation from the object being measured, then performs signal processing and photoelectric conversion to obtain the object's temperature distribution and provide a visual image. It offers advantages such as non-destructive testing, a wide temperature measurement range, and strong real-time performance. It is currently widely used in industries such as industry, healthcare, and construction. However, due to its non-contact nature, infrared thermal imaging's temperature measurement accuracy can be severely affected by environmental factors. In complex industrial environments, water mist in the optical path between the object being measured and the thermal imager is a common interference factor. Water mist can be considered a particle system that can independently transmit, reflect, and emit infrared radiation. The atmospheric transmittance in the presence of water mist decreases significantly compared to the atmospheric transmittance in the absence of water mist, resulting in significant errors in infrared temperature measurement.

[0003] Existing compensation methods primarily use infrared thermal images to estimate atmospheric transmittance in the presence of water mist and then compensate for temperature measurement errors. However, due to limitations in the resolution and texture detail of infrared thermal images, the accuracy of these estimated atmospheric transmittances is low, and over- or under-compensation is prone to occur. In fact, there is currently limited research on overcoming the effects of water mist on infrared temperature measurement, and the difficulty in accurately quantifying atmospheric transmittance in the presence of water mist has limited the application of infrared thermal imaging in scenes with water mist.

[0004] Compared to infrared thermal images, visible light images contain richer texture information and color features. Using existing prior knowledge, we can accurately estimate water mist transmittance in the visible light band. Under natural conditions, water mist concentration distribution does not undergo sudden changes. The thickness and concentration of water mist penetrated by adjacent visible light cameras and infrared thermal imagers can be considered identical. Therefore, the water mist transmittance estimated from visible light images follows the same trend as that in the infrared band.

[0005] Based on the above analysis, the present invention proposes an online compensation method for infrared thermal imaging temperature measurement errors caused by water mist interference. This method uses the aligned visible light image to estimate the atmospheric transmittance in the presence of water mist in real time, corrects the transmittance based on the Lambert-Beer law, and establishes an infrared temperature compensation model based on the infrared temperature measurement principle to overcome the influence of water mist. Through the real-time estimated atmospheric transmittance and temperature compensation model, online compensation for the infrared temperature measurement error caused by water mist is achieved.

[0006] Patent application number 202010102087.9 discloses a correction method and system for reducing the impact of dust in the optical path on infrared temperature measurement. This patent proposes a method for overcoming the impact of dust in the optical path on infrared temperature measurement. This method estimates the transmittance of the dust by placing a reference body next to the target being measured, accurately obtaining the transmittance of the affected atmosphere. However, this method can only be used in scenarios where the dust distribution is uniform and stable, and has high requirements for the installation location and the stability of the reference body.

[0007] The invention patent with application number 202111286569.5 proposes a method for infrared temperature measurement in a water mist environment. This method defogs the infrared thermal image by estimating the water mist transmittance to obtain the accurate temperature of the object being measured. However, this patent estimates the transmittance map of the infrared thermal image based on the dark channel prior. In fact, the infrared thermal image is a pseudo-color image generated based on the temperature value. Its color characteristics do not represent the actual situation. The dark channel prior does not hold for infrared thermal images, so it is difficult to obtain a reasonable transmittance, which affects the accuracy of infrared temperature measurement. Summary of the Invention

[0008] The present invention provides an online compensation method and system for infrared thermal imaging temperature measurement errors caused by water mist interference, which solves the technical problem of large infrared temperature measurement errors caused by water mist interference.

[0009] To solve the above technical problems, the present invention proposes an online compensation method for infrared thermal imaging temperature measurement errors caused by water mist interference, which includes:

[0010] Acquire infrared thermal images and visible light images simultaneously, and perform image registration based on their significant contour features and detailed edge features;

[0011] Based on the dark channel prior principle, the water mist transmittance in the visible light band is estimated according to the visible light image after image registration;

[0012] A multi-band water mist transmittance mapping model based on the Lambert-Beer law is established to map the water mist transmittance in the visible light band to the water mist transmittance in the infrared light band.

[0013] Based on the principle of infrared radiation temperature measurement, a temperature compensation model is constructed to compensate for the infrared temperature measurement error caused by water mist.

[0014] Based on the water mist transmittance in the infrared light band and the temperature compensation model, the actual temperature of the object being measured is obtained.

[0015] Furthermore, image registration based on the significant contour features and detailed edge features of the infrared thermal image and the visible light image includes:

[0016] Extract significant contour features and detailed edge features of infrared thermal images and visible light images;

[0017] Obtaining contour centroid point pairs matching the infrared thermal image and the visible light image;

[0018] An optimization objective function is constructed to construct an affine transformation matrix based on the coordinates of the matched contour centroid points and the detail edge features;

[0019] Based on the optimization objective function, a multi-stage genetic algorithm is used for optimization;

[0020] The optimized affine transformation matrix is ​​used to transform the visible light image to obtain the visible light image registered with the infrared thermal image.

[0021] Furthermore, obtaining a pair of contour centroid points for matching the infrared thermal image and the visible light image includes:

[0022] The centroid distance is used to construct the contour descriptors of infrared thermal images and visible light images;

[0023] Obtaining matching contour pairs according to the Euclidean distance between the contour descriptors of the infrared thermal image and the visible light image;

[0024] Calculating scale features, rotation features, and position features between matching contour pairs, and screening matching contour pairs based on the scale features, rotation features, and position features between the matching contour pairs;

[0025] According to the preset matching judgment parameters, the screened matching contour pairs are matched and judged. If the match is successful, the screened matching contour pairs are used as the contour centroid point pair for matching the infrared thermal image and the visible light image. Otherwise, the matching judgment parameters are set to 0.

[0026] Furthermore, the specific formula for calculating the scale feature, rotation feature, and position feature between the matching contour pairs is:

[0027]

[0028] Among them, k ri 、k θi and k ci are the scale feature, rotation feature and position feature of the i-th matching contour pair in the infrared thermal image T, r 0,Ti ,θ Ti and (x c,Ti ,y c,Ti ) are the maximum centroid distance, the angle between the connecting line and the positive direction of the horizontal axis, and the centroid coordinates of the contour corresponding to the i-th matching contour pair in the infrared thermal image T, r 0,Ij ,θ Ti and (x c,Ij ,y c,Ij) are respectively the maximum centroid distance, the angle between the connecting line and the positive direction of the horizontal axis, and the centroid coordinates of the contour with sequence number j that matches the i-th matching contour pair in the visible light image I and the infrared thermal image T, where the connecting line of the contour is specifically the connecting line formed by connecting the contour point corresponding to the maximum centroid distance of the contour and the centroid;

[0029] The specific calculation formula for screening matching contour pairs based on the scale features, rotation features, and position features between the matching contour pairs is:

[0030]

[0031] in, For all k ri The arithmetic mean, σ r For all k ri The standard deviation of For all k θi The arithmetic mean, σ θ For all k θi The standard deviation of For all k ci The arithmetic mean, σ c For all k ci The standard deviation of .

[0032] Furthermore, according to the preset matching judgment parameters, the specific formula for matching judgment on the screened matching contour pairs is:

[0033] v=v r ·v θ ·v c

[0034]

[0035] Among them, v is the matching judgment parameter, v r 、v θ and v c are scale judgment parameter, rotation judgment parameter and position judgment parameter respectively, and th1 and th2 are preset judgment parameter thresholds.

[0036] Furthermore, the specific formula for the optimization objective function of constructing the affine transformation matrix based on the coordinates of the matched contour centroid point pairs and the detail edge features is:

[0037]

[0038]

[0039]

[0040] Among them, affine is the affine transformation matrix, v is the matching judgment parameter, N is the total number of matching contour centroid point pairs in the infrared thermal image T, ω i is the weight of the i-th pair of contour centroid points, d aff,i is the Euclidean distance between the i-th pair of contour centroids after affine transformation, N edge is the total number of edge points in the detail edge feature map of the infrared thermal image, TP is the number of edge pixels in both the detail edge feature map of the infrared thermal image and the edge feature map of the visible light image after affine transformation, and k ri is the scale feature of the i-th matching contour pair in the infrared thermal image T, For all k ri The arithmetic mean of (x c,Ti ,y c,Ti ) is the centroid coordinate corresponding to the i-th matching contour pair in the infrared thermal image T, (x c,Ij ,y c,Ij ) is the centroid coordinate of the contour with serial number j in the visible light image I that matches the i-th matching contour pair in the infrared thermal image T.

[0041] Furthermore, based on the dark channel prior principle, the specific calculation formula for estimating the water mist transmittance in the visible light band according to the visible light image after image registration is:

[0042]

[0043] Where t(x) represents the water mist transmittance in the visible light band at coordinate x, k represents the image coordinate in the neighborhood Ω(x) of coordinate x, c represents one of the three color channels r, g, and b, and I c A represents the foggy image of the c color channel. c Represents the global atmospheric light A of the c color channel, where A is the average value of the 0.1% of pixels with the highest brightness in the foggy image.

[0044] Furthermore, a multi-band water mist transmittance mapping model based on the Lambert-Beer law is established, and the specific calculation formula for mapping the water mist transmittance in the visible light band to the water mist transmittance in the infrared light band is:

[0045] τ w (x) = t(x) a ,

[0046] Among them, τ w (x) is the water mist transmittance in the infrared light band at coordinate x, a is the calibrated mapping parameter, and t(x) is the water mist transmittance in the visible light band at coordinate x.

[0047] Furthermore, based on the water mist transmittance in the infrared light band and the temperature compensation model, the specific calculation formula for obtaining the actual temperature of the measured object is:

[0048]

[0049] Among them, T0 represents the actual temperature of the object being measured, t is the water mist transmittance in the visible light band, ε0 represents the emissivity of the object being measured, τ a is the atmospheric transmittance, T rd 、T u 、T w and T a are the detection temperature, ambient temperature, water mist temperature, and atmospheric temperature of the object being measured, respectively; n is the fitting coefficient related to the detector material.

[0050] The infrared thermal imaging temperature measurement error online compensation system for water mist interference provided by the present invention includes:

[0051] A memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the method for online compensation of infrared thermal imaging temperature measurement error caused by water mist interference provided by the present invention are implemented.

[0052] Compared with the prior art, the advantages of the present invention are:

[0053] The present invention provides an online compensation method and system for infrared thermal imaging temperature measurement errors caused by water mist interference. The method and system synchronously acquire infrared thermal images and visible light images, perform image registration based on the significant contour features and detail edge features of the infrared thermal images and visible light images, estimate the water mist transmittance in the visible light band based on the visible light image after image registration based on the dark channel prior principle, determine the water mist transmittance in the infrared light band based on the water mist transmittance in the visible light band based on the Lambert-Beer law, and construct a temperature compensation model for compensating for the infrared temperature measurement error caused by water mist and a water mist transmittance and temperature compensation model based on the infrared light band based on the infrared radiation temperature measurement principle to obtain the actual temperature of the object being measured. The method solves the technical problem of large infrared temperature measurement error under water mist interference, can accurately estimate the water mist transmittance in real time, and realize precise and online compensation of infrared temperature measurement errors.

[0054] Purpose of the present invention:

[0055] The present invention aims to propose an online compensation method for infrared thermal imaging temperature measurement errors caused by water mist interference. In order to reasonably compensate for the infrared temperature measurement errors caused by water mist, the present invention treats water mist as a particle system, analyzes the influence of water mist on infrared temperature measurement, and establishes a temperature compensation model based on the principle of infrared temperature measurement. In order to eliminate the difference between infrared thermal images and visible light images, a method for infrared-visible light heterogeneous visual image registration that combines significant contour features with detailed edge features is proposed, which reduces the possibility of mismatching. In order to accurately estimate the transmittance of water mist, the transmittance of water mist is first estimated through visible light images based on dark channel priors, and then the transmittance of water mist is corrected based on the Lambert-Beer law. Compared with existing temperature compensation methods, the temperature compensation method proposed in the present invention can more accurately estimate the transmittance of water mist in real time, thereby achieving precise and online compensation for infrared temperature measurement errors.

[0056] The key points of the present invention are:

[0057] (1) The present invention proposes for the first time an online compensation method for infrared thermal imaging temperature measurement errors caused by water mist interference. It also introduces visible light information into the infrared error compensation modeling for the first time. By online estimating the transmittance of water mist, it realizes online compensation for the infrared temperature measurement error caused by water mist.

[0058] (2) Taking water mist as a particle system that can transmit, reflect and emit infrared radiation on its own, the influence of water mist on infrared temperature measurement is analyzed, and a temperature compensation model is established based on the principle of infrared temperature measurement to compensate for the infrared temperature measurement error caused by water mist.

[0059] (3) A method for infrared-visible light heterogeneous visual image registration that combines significant contour features with edge detail features is proposed. The success of contour matching is judged by constraint conditions. An optimization objective function is constructed and optimized through a multi-stage genetic algorithm to achieve heterogeneous image registration.

[0060] (4) Based on the Lambert-Beer law, an online estimation method for water mist transmittance in the infrared light band combined with visible light image information is proposed. The water mist transmittance in the visible light band is corrected to the water mist transmittance in the infrared light band, realizing the online estimation of water mist transmittance. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 This is a flow chart of the infrared temperature measurement error compensation method according to the second embodiment of the present invention;

[0062] Figure 2 This is a flow chart of the image registration algorithm according to the second embodiment of the present invention;

[0063] Figure 3 This is a diagram showing the infrared temperature measurement error compensation result of the third embodiment of the present invention;

[0064] Figure 4This is a structural block diagram of an online compensation system for infrared thermal imaging temperature measurement errors against water mist interference according to an embodiment of the present invention.

[0065] Reference numerals:

[0066] 10. Memory; 20. Processor. DETAILED DESCRIPTION

[0067] To facilitate understanding of the present invention, the present invention will be described in more comprehensive and detailed form below in conjunction with the accompanying drawings and preferred embodiments. However, the protection scope of the present invention is not limited to the following specific embodiments.

[0068] The embodiments of the present invention are described in detail below with reference to the accompanying drawings. However, the present invention can be implemented in many different ways as defined and covered by the claims.

[0069] Example 1

[0070] The first embodiment of the present invention provides an online compensation method for infrared thermal imaging temperature measurement errors caused by water mist interference, including:

[0071] Step S101, synchronously acquiring an infrared thermal image and a visible light image, and performing image registration based on significant contour features and detailed edge features of the infrared thermal image and the visible light image;

[0072] Step S102 , estimating the water mist transmittance in the visible light band based on the registered visible light image based on the dark channel prior principle;

[0073] Step S103, based on the water mist transmittance in the visible light band, determine the water mist transmittance in the infrared light band according to the Lambert-Beer law;

[0074] Step S104: constructing a temperature compensation model for compensating for infrared temperature measurement errors caused by water mist based on the infrared radiation temperature measurement principle;

[0075] Step S105 : obtaining the actual temperature of the object to be measured based on the water mist transmittance in the infrared light band and the temperature compensation model.

[0076] The embodiment of the present invention provides an online compensation method for infrared thermal imaging temperature measurement errors caused by water mist interference. The method synchronously acquires infrared thermal images and visible light images, and performs image registration based on the significant contour features and detailed edge features of the infrared thermal images and the visible light images. Based on the dark channel prior principle, the water mist transmittance in the visible light band is estimated according to the visible light image after image registration. Based on the Lambert-Beer law, the water mist transmittance in the infrared light band is determined according to the water mist transmittance in the visible light band. Based on the infrared radiation temperature measurement principle, a temperature compensation model for compensating for the infrared temperature measurement error caused by water mist and a water mist transmittance and temperature compensation model based on the infrared light band are constructed to obtain the actual temperature of the object being measured. The method solves the technical problem of large infrared temperature measurement error under water mist interference, can accurately estimate the transmittance of water mist in real time, and realizes precise and online compensation of infrared temperature measurement errors.

[0077] Specifically, in order to reasonably compensate for the infrared temperature measurement error caused by water mist, the embodiment of the present invention takes water mist as a particle system, analyzes the influence of water mist on infrared temperature measurement, and establishes a temperature compensation model based on the principle of infrared temperature measurement. In order to eliminate the difference between infrared thermal images and visible light images, this embodiment proposes an infrared-visible light heterogeneous visual image registration method that combines significant contour features with detailed edge features, thereby reducing the possibility of mismatching. In order to accurately estimate the transmittance of water mist, the transmittance of water mist is first estimated through visible light images based on the dark channel prior, and then the transmittance of water mist is corrected based on the Lambert-Beer law. Compared with the existing temperature compensation method, the temperature compensation method proposed in the embodiment of the present invention can more accurately estimate the transmittance of water mist in real time, and can achieve precise and online compensation for infrared temperature measurement errors.

[0078] Example 2

[0079] The embodiment of the present invention proposes an online compensation method for infrared thermal imaging temperature measurement error caused by water mist interference. Figure 1 FIG. 1 is a diagram of the implementation steps of the infrared temperature measurement error compensation method proposed by the present invention, which specifically includes the following steps:

[0080] (1) The influence of water mist on infrared temperature measurement is analyzed, and based on the principle of infrared radiation temperature measurement, a temperature compensation model is constructed to compensate for the infrared temperature measurement error caused by water mist.

[0081] (2) An imaging system consisting of an infrared thermal imager and a visible light camera is used to synchronously acquire infrared thermal images and visible light images, and image registration is performed based on the significant contour features and detailed edge features of the heterogeneous images.

[0082] (3) Considering that visible light images have richer texture information, the water mist transmittance in the visible light band is first estimated through visible light images based on the dark channel prior principle, and then the water mist transmittance applicable to the infrared light band is determined according to the Lambert-Beer law.

[0083] (4) Substitute the measured temperature with error and the water mist transmittance under water mist interference into the temperature compensation model to obtain the compensated temperature of the measured object.

[0084] The specific implementation plan is as follows:

[0085] (1) Establishment of infrared temperature measurement error compensation model:

[0086] Infrared thermography is a non-contact temperature measurement technology. However, its temperature measurement results can be affected by water mist in the optical path, resulting in temperature measurement errors. Therefore, this paper analyzes the impact of water mist on infrared temperature measurement and establishes a temperature compensation model based on this analysis.

[0087] Consider water mist as a particle system located on the optical path between the object being measured and the thermal imager. This particle system can transmit, emit, and reflect infrared radiation. Then, when water mist interferes, the infrared radiation received by the infrared thermal imager can be expressed as:

[0088] W rd =ε0τ a τ w W0+ρ0τ a τ w W u +ε w τ a W w +ρ w τ a W u +ε a W a (1)

[0089] Where W rd Indicates the infrared radiation received by the infrared detector, ε0τ a τ w W0 represents the infrared radiation emitted by the object being measured, ρ0τ a τ w W u Indicates the infrared radiation reflected by the measured object from the surrounding environment, ε w τ a W w represents the infrared radiation emitted by water mist, ρ w τ a W u Indicates the infrared radiation reflected by the surrounding environment, ε a W a Indicates the infrared radiation emitted by the atmosphere. ε0, ε a and ε w They represent the emissivity of the object being measured, the emissivity of the atmosphere, and the emissivity of water mist, ρ0 and ρ w Represent the reflectivity of the measured object and the reflectivity of water mist, τ aand τ w represent the atmospheric transmittance and water mist transmittance respectively.

[0090] When the object being measured is an opaque object, its transmittance is 0, and the reflectivity of the atmosphere can also be considered to be 0. Then, according to Kirchhoff's law, equation (1) can be expressed as:

[0091]

[0092] Here, it is assumed that the temperature of the water mist on the optical path is the same as the ambient temperature, so equation (2) can be simplified to:

[0093]

[0094] Let s(T) represent the detector response caused by the radiation intensity of a black body with a temperature of T, then equation (3) can be expressed as:

[0095]

[0096] According to Planck's radiation law, when the infrared detector does not consider the change of detector response with wavelength, the relationship between s(T) and temperature T can be approximated as:

[0097] s(T)=CT n (5)

[0098] Where C and n are fitting coefficients related to the detector material. Substituting equation (5) into equation (4) yields the temperature compensation model for compensating the infrared temperature measurement error caused by water mist:

[0099]

[0100] According to the Lambert-Beer law, the transmittance of water mist τ w It can be expressed as:

[0101] τ w (λ)=exp[-k(λ)·c w ·l w ] (7)

[0102] Where k(λ) represents the mass extinction coefficient, c w Indicates water mist concentration, l wRepresents the thickness of the water mist. Compared to natural haze weather, the distribution of water mist in industrial sites is usually uneven and changes dynamically. From Equation (7), it can be seen that this causes the water mist transmittance to also change dynamically. However, other influencing factors such as emissivity, ambient temperature, and atmospheric temperature in Equation (6) remain basically stable during temperature measurement and can be set as constants through prior knowledge or contact temperature measurement. Therefore, real-time and accurate estimation of the water mist transmittance parameter is the key to temperature compensation through Equation (6). However, it is difficult to accurately estimate the water mist transmittance using only infrared thermal images. Therefore, an imaging system consisting of an infrared thermal imager and a visible light camera is built, and visible light images with richer color information are introduced to estimate the water mist transmittance online, thereby achieving online compensation of temperature measurement errors.

[0103] (2) Registration of heterogeneous images:

[0104] Due to factors such as differences in infrared thermal imager parameters, camera spatial locations, and non-parallel optical axes, the position and scale of the temperature measurement object in the infrared and visible light images differ. Therefore, before estimating water mist transmittance from visible light images, the image must be translated, rotated, and scaled using an affine transformation matrix to achieve alignment between the image coordinates.

[0105] Since infrared thermal images are imaged based on the infrared radiation intensity of the temperature-measured object, they are far less rich in texture details than three-channel visible light images. Image matching methods based on feature points are prone to mismatching. Although infrared thermal images have clear contour features, they also contain temperature change information that cannot be observed in visible light images. Therefore, matching based solely on contour edge features may also cause additional errors. Based on the above analysis, the present invention proposes an infrared-visible light heterogeneous visual image registration method that combines significant contour features with detailed edge features. The registration method flow chart is shown in the figure below. Figure 2 As shown in the figure, the specific registration steps are as follows:

[0106] Step 1: Extract edge detail maps and salient contour maps from heterogeneous visual images. Use the Canny operator to extract edge details from the original infrared thermal image and visible light image, respectively, to obtain edge detail maps. Downsample the original image and segment the downsampled image using FCM clustering. Extract closed contours from the segmented image to obtain salient contour maps.

[0107] Step 2: Obtain matching contour centroid pairs in heterogeneous visual images. To match contours in heterogeneous images, a contour descriptor needs to be constructed to accurately describe contour features. Centroid distance can simultaneously reflect both local and global contour features, so it is used to construct the contour descriptor.

[0108] First, the mean of the contour point coordinates is taken as the contour centroid, and the distance from the contour edge point to the centroid is calculated using formula (8).

[0109]

[0110] Where r is the distance between the centroids, (x c ,y c ) are the coordinates of the center of mass.

[0111] To ensure the rotation invariance of the contour descriptor, the line connecting the contour points corresponding to the centroid and the maximum centroid distance is set to 0°. Then, a contour point is selected every 30° clockwise, for a total of 12 contour points. The centroid distances corresponding to these contour points are combined into a 12-dimensional vector in the order of selection. This vector is normalized to ensure scale invariance. The normalized vector is used as the descriptor of the contour, specifically expressed by Equation (9).

[0112]

[0113] Where R is the contour descriptor and r0 is the maximum centroid distance. After obtaining the contour descriptor, the Euclidean distance between the contour descriptors is calculated to measure the similarity between the visible light contour and the infrared light contour, as shown in formula (10).

[0114] d ij =||R Ti -R Ij ||2 (10)

[0115] Where R Ti and R Ij are the contour descriptors on infrared thermal images and visible light images, d ij is the similarity between the two descriptors.

[0116] Due to interference from factors such as image distortion and noise, the contour pair with the highest similarity is not necessarily the matching contour pair. Therefore, the nearest neighbor distance ratio is used for judgment, specifically:

[0117]

[0118] Where NNDR is the nearest neighbor distance ratio, d imin and d inmin and R Ti The Euclidean distance corresponding to the closest and second closest visible light contour descriptors. When the NNDR is less than the threshold, the pair of contours with the highest similarity is considered to be a matching contour pair.

[0119] Formula (11) is a commonly used constraint standard for feature matching of homologous images. However, infrared thermal images contain temperature change information that is not present in visible light images, which can easily lead to unmatched contours and mismatches. Therefore, a stricter constraint standard is needed to further determine whether a mismatch occurs.

[0120] Use the maximum centroid distance ratio k r , the maximum center of mass distance angle ratio k θ and the ratio of the center of mass position k c To further describe the scale feature, rotation feature and position feature of the contour pair, as shown in formula (12).

[0121]

[0122] Where k ri is the maximum centroid distance ratio of the i-th contour in the infrared thermal image T, r 0,Ti is the maximum centroid distance of the i-th contour of the infrared thermal image, r 0,Ij k is the maximum centroid distance of the visible light image contour that matches the contour. Connect the contour point and the centroid corresponding to the maximum centroid distance of the contour. The angle between the connecting line segment and the positive direction of the horizontal axis is the maximum centroid distance angle θ. θi and k ci The variable subscripts in the calculation formula and k ri have the same meaning.

[0123] Generally speaking, matching contours at different positions and scales in an image should have similar transformation relationships, so it is assumed that k r 、k θ 、k c The normal distribution is followed by the 3σ principle. If any of the three parameters does not meet the 3σ principle, the matching contour pair is eliminated. Formula (13) indicates whether the scale characteristics of the i-th pair of contours meet the standard according to the 3σ principle.

[0124]

[0125] In the formula For all k r The arithmetic mean, σ r For all k r The standard deviation of .

[0126] In this way, the contour matching results are screened by equations (11) and (13), and multiple pairs of contour centroid coordinates matching in the infrared thermal image and the visible light image are obtained.

[0127] In addition, considering the possibility that most contour mismatches may cause the contour feature ratio to not conform to the normal distribution, the parameter v is set to judge whether the overall contour matching is successful. The calculation of the parameter v is shown in formula (14).

[0128] v=v r ·v θ ·v c

[0129]

[0130] Since the distance between the infrared thermal imager and the visible light camera is small, the coordinate difference between the images is small. or When the thresholds th1 and th2 are exceeded, the contour matching is considered to have failed and v is set to 0.

[0131] Step 3: Construct the optimization objective function based on the matched contour centroid coordinates and edge detail map. The objective function is shown in formula (15).

[0132]

[0133] Where affine is the affine transformation matrix, ω i is the weight of the i-th pair of contour centroids, d aff,i is the Euclidean distance between the centroids of the i-th pair of contours after affine transformation, TP is the number of pixels that are edges in both the infrared edge detail image and the visible edge detail image after affine transformation, and N edge is the total number of edge points in the infrared edge detail image.

[0134] The first term in Equation (15) evaluates whether the contour centroid pair is successfully registered by calculating the Euclidean distance of the contour centroid pair, where the contour weight ω i The distance d from the center of mass aff,i Calculated by formula (16).

[0135]

[0136]

[0137] ω i K ri and The difference between ri and The smaller the difference, the greater the weight, and the sigmoid function is used to convert ω i It is limited to 0-1. When v is 0, that is, the overall contour matching fails, the first item is set to 0.

[0138] The second term in formula (15) is the ratio of the number of unmatched edge points to the number of matched edge points, which is used to evaluate the matching degree between the visible light edge detail map and the infrared light edge detail map after affine transformation.

[0139] Step 4: Use a multi-stage genetic algorithm for optimization. In the genetic algorithm optimization process, smaller step sizes and search boundaries may cause the optimization results to fall into local optimal solutions, while large step sizes make it difficult to obtain high-precision optimization results. Therefore, the present invention divides the traditional genetic algorithm into multiple stages, and gradually reduces the step size and search boundary in each stage. The first stage in the algorithm flow is the same as the normal genetic algorithm. When the optimal value does not change after 10 iterations, this stage is terminated and the edge map is transformed according to the optimized affine transformation matrix. The transformed edge map is used as the input of the next stage. In the next stage, the step size and search boundary are reduced and the population is reinitialized for a new round of optimization. After four stages, the final affine transformation matrix is ​​output.

[0140] Step 5: Use the optimized affine transformation matrix to transform the visible light image to obtain a visible light image that is spatially aligned with the infrared thermal image.

[0141] (3) Estimation and correction of water mist transmittance:

[0142] Compared to infrared thermal images, visible light images have richer color features. Based on the dark channel prior principle and atmospheric scattering models, we can obtain atmospheric transmittance with higher accuracy. Therefore, this paper uses visible light image information to estimate atmospheric transmittance, improving the accuracy of temperature compensation.

[0143] The dark channel prior means that in most non-sky areas of a fog-free image, the pixel values ​​of at least one color channel in the image block will be close to 0, and the dark channel image will be dark overall. It can be specifically expressed by formula (17).

[0144]

[0145] Where J dark represents the dark channel image, J represents the haze-free image, x represents the pixel coordinate, and Ω(x) represents the neighborhood near x.

[0146] However, in foggy images, the dark channel image appears grayish white as a whole, indicating that the overall brightness of the image will increase when affected by water mist. This phenomenon can be explained by the atmospheric scattering model, which is shown in Equation (10).

[0147] I(x)=t(x)·J(x)+(1-t(x))A (18)

[0148] Where I represents the foggy image, A represents the global atmospheric light, and t represents the water mist transmittance. From Equation (18), we can see that the pixels of the fog-free image will be affected by the atmospheric light scattered by the water mist, and the lower the original brightness, the more serious the influence of atmospheric light. Therefore, the extracted dark channel image is brighter overall.

[0149] Based on the dark channel prior and the atmospheric scattering model, the specific atmospheric transmittance estimation method is as follows. First, assuming that the global atmospheric light is a constant for the entire image, then the lowest pixel value of the three color channels of J in the Ω(x) region will also correspond to the lowest pixel value of the three color channels of I. Therefore, Equation (18) can be expressed as:

[0150]

[0151] Substituting formula (17) into formula (19) yields:

[0152]

[0153] Generally speaking, the brightest area in the image is considered to be the sky area or the light source area. Therefore, the average value of the 0.1% pixels with the highest brightness in the entire image is taken as the global atmospheric light A. In this way, the atmospheric transmittance corresponding to all pixels in the visible light image can be obtained through formula (20).

[0154] When the distance is close, the atmospheric transmittance without fog can be regarded as 1. At this time, the atmospheric transmittance affected by water fog obtained by formula (20) can be regarded as water fog transmittance. However, as shown in formula (7), the transmittance is affected by wavelength. The transmittance at visible light wavelength is different from that at infrared wavelength. Therefore, it is necessary to correct the water fog transmittance t in the visible light band estimated from the visible light image to the water fog transmittance τ in the infrared band. w , the corrected t can be used as a parameter of the temperature compensation model.

[0155] Since the infrared thermal imager and visible light camera are located close to each other during image acquisition, the water mist concentration and thickness corresponding to the same pixel can be considered equal. Taking the logarithm of both sides of equation (7) yields equation (21).

[0156]

[0157] Where a is a constant, and its specific value is obtained through radiation calibration experiments. Exponentiation of formula (21) yields:

[0158] τ w (x) = t(x) a (twenty two)

[0159] Formula (22) is used to correct the water mist transmittance in the visible light band to the water mist transmittance in the infrared light band.

[0160] In this way, after the imaging system synchronously obtains the infrared thermal image and the visible light image, the water mist transmittance in the visible light band is first obtained according to formula (20) through the visible light image after image registration, and then the water mist transmittance in the visible light band is corrected to the water mist transmittance in the infrared light band through formula (22). Finally, the corrected water mist transmittance and the measured temperature are substituted into formula (6) to realize the online compensation of the infrared temperature measurement error caused by water mist.

[0161] The embodiment of the present invention regards water mist as a particle system to analyze its influence on infrared temperature measurement, and establishes a temperature compensation model for compensating for the infrared temperature measurement error caused by water mist. In order to solve the problem of pixel mismatch between visible light images and infrared thermal images, a method for infrared-visible light heterogeneous visual image registration combining significant contour features with edge detail features is proposed, which reduces the possibility of contour mismatching and can perform registration based on edge detail features when contour matching fails, thereby improving the accuracy of heterogeneous image registration. In order to solve the problem that the transmittance of water mist is difficult to determine online, the present invention first estimates the transmittance of water mist through visible light images based on dark channel priors, and then corrects the transmittance of water mist based on the Lambert-Beer law, thereby realizing online estimation of the transmittance of water mist in the infrared light band. Finally, the corrected transmittance and detected temperature are substituted into the temperature compensation model to obtain the accurate surface temperature of the object being measured under water mist interference.

[0162] The embodiment of the present invention introduces visible light information into infrared error compensation modeling for the first time, solving the problem that the influence of water mist is difficult to accurately quantify during infrared temperature measurement, and realizing online accurate compensation for infrared temperature measurement errors caused by water mist.

[0163] Example 3

[0164] The following examples are used to illustrate the present invention but are not intended to limit the scope of the present invention.

[0165] This embodiment uses a high-temperature blackbody furnace as the temperature measurement object. An imaging system consisting of an infrared thermal imager and a visible light camera is placed in front of the blackbody furnace. A water mist generator is placed between the imaging system and the blackbody furnace. The generated water mist is distributed on the optical path between the imaging system and the blackbody furnace. In this way, infrared thermal images and visible light images affected by the water mist can be obtained. The temperature points on a horizontal line at the blackbody furnace mouth are tested. The experimental results are as follows: Figure 3As shown in the figure. The true temperature is the temperature of the corresponding temperature point in the fog-free thermal image. It can be seen that, affected by water mist, the detected temperature is lower than the true temperature and is distributed in an arc shape, with an average temperature error of -11.3843°C. The temperature after temperature compensation using the uncorrected water mist transmittance is significantly higher, with an average temperature error of 5.8165°C. The temperature after temperature compensation using the corrected water mist transmittance is more uniform and fluctuates around the true temperature, with an average temperature error of 0.8959°C. By comparing the detected temperature, the compensated temperature before correcting the water mist transmittance, and the compensated temperature after correcting the water mist transmittance, it is shown that the present invention can effectively compensate for the infrared temperature measurement error caused by water mist.

[0166] Reference Figure 4 The embodiment of the present invention proposes an online compensation system for infrared thermal imaging temperature measurement errors against water mist interference, comprising:

[0167] A memory 10, a processor 20, and a computer program stored in the memory 10 and executable on the processor 20, wherein the processor 20 implements the steps of the online compensation method for infrared thermal imaging temperature measurement error due to water mist interference proposed in this embodiment when executing the computer program.

[0168] The specific working process and working principle of the infrared thermal imaging temperature measurement error online compensation system for water mist interference in this embodiment can refer to the working process and working principle of the infrared thermal imaging temperature measurement error online compensation method for water mist interference in this embodiment.

[0169] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.

Claims

1. An online compensation method for infrared thermal imaging temperature measurement error caused by water mist interference, characterized in that: The method comprises: Acquire infrared thermal images and visible light images simultaneously, and perform image registration based on their significant contour features and detailed edge features; Based on the dark channel prior principle, the water mist transmittance in the visible light band is estimated according to the visible light image after image registration; A multi-band water mist transmittance mapping model based on the Lambert-Beer law is established to map the water mist transmittance in the visible light band to the water mist transmittance in the infrared light band. Based on the principle of infrared radiation temperature measurement, a temperature compensation model is constructed to compensate for the infrared temperature measurement error caused by water mist. Based on the water mist transmittance in the infrared light band and the temperature compensation model, the actual temperature of the object being measured is obtained.

2. The online compensation method for infrared thermal imaging temperature measurement error against water mist interference according to claim 1 is characterized in that: Image registration based on the significant contour features and detailed edge features of infrared thermal images and visible light images includes: Extract significant contour features and detailed edge features of infrared thermal images and visible light images; Obtaining contour centroid point pairs matching the infrared thermal image and the visible light image; An optimization objective function is constructed to construct an affine transformation matrix based on the coordinates of the matched contour centroid points and the detail edge features; Based on the optimization objective function, a multi-stage genetic algorithm is used for optimization; The optimized affine transformation matrix is ​​used to transform the visible light image to obtain the visible light image registered with the infrared thermal image.

3. The online compensation method for infrared thermal imaging temperature measurement error against water mist interference according to claim 2 is characterized in that: The contour centroid point pairs obtained by matching the infrared thermal image with the visible light image include: The centroid distance is used to construct the contour descriptors of infrared thermal images and visible light images; Obtaining matching contour pairs according to the Euclidean distance between the contour descriptors of the infrared thermal image and the visible light image; Calculating scale features, rotation features, and position features between matching contour pairs, and screening matching contour pairs based on the scale features, rotation features, and position features between the matching contour pairs; According to the preset matching judgment parameters, the screened matching contour pairs are matched and judged. If the match is successful, the screened matching contour pairs are used as the contour centroid point pair for matching the infrared thermal image and the visible light image. Otherwise, the matching judgment parameters are set to 0.

4. The online compensation method for infrared thermal imaging temperature measurement error against water mist interference according to claim 3 is characterized in that: The specific formula for calculating the scale feature, rotation feature, and position feature between matching contour pairs is: Among them, k ri 、k θi and k ci are the scale feature, rotation feature and position feature of the i-th matching contour pair in the infrared thermal image T, r 0,Ti ,θ Ti and (x c,Ti ,y c,Ti ) are the maximum centroid distance, the angle between the connecting line and the positive direction of the horizontal axis, and the centroid coordinates of the contour corresponding to the i-th matching contour pair in the infrared thermal image T, r 0,Ij ,θ Ti and (x c,Ij ,y c,Ij ) are respectively the maximum centroid distance, the angle between the connecting line and the positive direction of the horizontal axis, and the centroid coordinates of the contour with sequence number j that matches the i-th matching contour pair in the visible light image I and the infrared thermal image T, where the connecting line of the contour is specifically the connecting line formed by connecting the contour point corresponding to the maximum centroid distance of the contour and the centroid; The specific calculation formula for screening matching contour pairs based on the scale features, rotation features, and position features between the matching contour pairs is: in, For all k ri The arithmetic mean, σ r For all k ri The standard deviation of For all k θi The arithmetic mean, σ θ For all k θi The standard deviation of For all k ci The arithmetic mean, σ c For all k ci The standard deviation of .

5. The method for online compensation of infrared thermal imaging temperature measurement error against water mist interference according to claim 4 is characterized in that: According to the preset matching judgment parameters, the specific formula for matching judgment of the screened matching contour pairs is: v=v r ·v θ ·v c Among them, v is the matching judgment parameter, v r 、v θ and v c are scale judgment parameter, rotation judgment parameter and position judgment parameter respectively, and th1 and th2 are preset judgment parameter thresholds.

6. The method for online compensation of infrared thermal imaging temperature measurement error against water mist interference according to claim 5 is characterized in that: The specific formula of the optimization objective function for constructing the affine transformation matrix based on the coordinates of the matched contour centroid point pair and the detail edge features is: Among them, affine is the affine transformation matrix, v is the matching judgment parameter, N is the total number of matching contour centroid point pairs in the infrared thermal image T, ω i is the weight of the i-th pair of contour centroid points, d aff,i is the Euclidean distance between the i-th pair of contour centroids after affine transformation, N edge is the total number of edge points in the detail edge feature map of the infrared thermal image, TP is the number of edge pixels in both the detail edge feature map of the infrared thermal image and the edge feature map of the visible light image after affine transformation, and k ri is the scale feature of the i-th matching contour pair in the infrared thermal image T, For all k ri The arithmetic mean of (x c,Ti ,y c,Ti ) is the centroid coordinate corresponding to the i-th matching contour pair in the infrared thermal image T, (x c,Ij ,y c,Ij ) is the centroid coordinate of the contour with serial number j in the visible light image I that matches the i-th matching contour pair in the infrared thermal image T.

7. The method for online compensation of infrared thermal imaging temperature measurement error against water mist interference according to claim 6 is characterized in that: Based on the dark channel prior principle, the specific calculation formula for estimating the water mist transmittance in the visible light band according to the visible light image after image registration is: Among them, t(x) represents the water mist transmittance in the visible light band at coordinate x, and k represents the neighborhood Ω( x ), c represents one of the three color channels r, g, and b, I c A represents the foggy image of the c color channel. c Represents the global atmospheric light A of the c color channel, where A is the average value of the 0.1% of pixels with the highest brightness in the foggy image.

8. The method for online compensation of infrared thermal imaging temperature measurement error against water mist interference according to claim 1 or 7, characterized in that: A multi-band water mist transmittance mapping model based on the Lambert-Beer law is established. The specific calculation formula for mapping the water mist transmittance in the visible light band to the water mist transmittance in the infrared light band is: τ w (x)=t(x) a , Among them, τ w (x) is the water mist transmittance in the infrared light band at coordinate x, a is the calibrated mapping parameter, and t(x) is the water mist transmittance in the visible light band at coordinate x.

9. The method for online compensation of infrared thermal imaging temperature measurement error against water mist interference according to claim 8, characterized in that: Based on the water mist transmittance and temperature compensation model in the infrared light band, the specific calculation formula for obtaining the actual temperature of the measured object is: Among them, T0 represents the actual temperature of the object being measured, t is the water mist transmittance in the visible light band, ε0 represents the emissivity of the object being measured, τ a is the atmospheric transmittance, T rd 、T u 、T w and T a are the detection temperature, ambient temperature, water mist temperature, and atmospheric temperature of the object being measured, respectively; n is the fitting coefficient related to the detector material.

10. An online compensation system for infrared thermal imaging temperature measurement errors caused by water mist interference, the system comprising: A memory (10), a processor (20), and a computer program stored in the memory (10) and executable on the processor (20), wherein the processor (20) implements the steps of the method according to any one of claims 1 to 9 when executing the computer program.

Images