A multi-source visual temperature measurement method for different emissivity targets

Abstract

The application discloses a multi-source visual temperature measurement method for different emissivity targets, and the method comprises the following steps: a bimodal semantic segmentation model taking a visible light image and an infrared image of a scene as input is established, target regions with different emissivities are identified, and an emissivity region segmentation map and a classification confidence map are output; a material emissivity priori library is established, a pixel-by-pixel emissivity map is generated in combination with the emissivity region segmentation map and the classification confidence map, and a corresponding uncertainty map is obtained through a comprehensive uncertainty quantification method; a different emissivity target temperature compensation model that fuses a neural network and a thermal radiation mechanism constraint is constructed, the weight of data driving and physical driving is dynamically balanced according to the pixel-by-pixel emissivity map, the uncertainty map and infrared radiation brightness, and a compensated temperature value is output. The temperature measurement method disclosed by the application can adapt to a complex scene composed of multiple materials, and has the advantages of uncertainty quantification, high temperature measurement precision and strong robustness.

Description

Technical Field

[0001] This invention belongs to the field of infrared thermometry technology, specifically a multi-source visual thermometry method for targets with different emissivity. Background Technology

[0002] In fields such as metallurgical manufacturing, equipment inspection, and environmental monitoring, temperature information is a crucial parameter reflecting the operating status, heat distribution characteristics, and abnormal changes of a target. Its measurement accuracy is essential for fault diagnosis, condition assessment, process control, and safety early warning. Infrared thermal imaging technology, as a typical non-contact temperature measurement method, is widely used in these fields due to its advantages such as fast response speed and wide temperature measurement range.

[0003] In infrared thermometry, the measurement results are not only related to the actual temperature of the target but also closely related to the radiation characteristics of the target surface, with emissivity being a crucial factor affecting measurement accuracy. According to the Stefan-Boltzmann law, the radiative exitance of an object's surface is simultaneously affected by the coupling effect of temperature and emissivity. When the emissivity parameter deviates from its true value, it will directly lead to a systematic deviation in the measurement results. However, in actual temperature measurement, the measured scene is usually composed of multiple materials, and the surface emissivity of different material regions varies significantly, resulting in different infrared radiation characteristics. This makes infrared thermometry results in complex scenes susceptible to the influence of uneven spatial distribution of emissivity.

[0004] To address the challenge of accurate temperature measurement under spatially non-uniform emissivity distribution, existing methods primarily include: calculating the temperature of the entire field of view using a uniform emissivity parameter, manually setting emissivity parameters for different regions based on operator experience, or adaptive emissivity estimation using algorithms based on a single infrared image. The uniform emissivity parameter method fails to reflect material differences, resulting in systematic biases in the temperature measurement results. Manually setting regional parameters relies on operational experience and is inefficient, making it difficult to adapt to dynamic changes in the scene. Furthermore, relying solely on infrared images for emissivity estimation is limited in accuracy because the infrared radiation signal is simultaneously affected by temperature and emissivity, making it difficult to distinguish between radiation differences caused by temperature variations and those caused by material emissivity differences. Visible light images, however, have a natural advantage in characterizing material texture and surface structure features, providing complementary information that infrared images lack for distinguishing regions with different emissivity.

[0005] In the prior art, for example, patent application number CN202510307632.0 discloses a dual-band infrared thermometry method. This method achieves dual-band continuous infrared thermometry by changing the radiation constant in Planck's formula to a parameter that varies with factors such as the material and properties of the object being measured, and selecting the optimal temperature measurement wavelength based on the radiation intensity-wavelength relationship curve. However, the parameter solution of this method depends on the accurate measurement and calibration of multi-wavelength radiation intensity at a specific temperature point. The calibration process is complex, and long-term stable operation is difficult. Furthermore, it does not provide an adaptive compensation mechanism for the problem of emissivity varying with space in complex scenarios.

[0006] For example, patent application number CN202511484539.3 discloses a method for calculating the temperature field of a bridge section by integrating infrared imaging and finite element analysis. This method uses the surface temperature measured by infrared thermal imaging as a boundary condition to introduce into the finite element model to reconstruct the temperature field of the structural section. However, its temperature measurement results depend on the preset values ​​of parameters such as emissivity. In cases where the emissivity space is non-uniform or the surface state changes, the error is easily amplified, and there is a lack of effective error constraint mechanism, resulting in insufficient stability of temperature estimation.

[0007] Visible light images have a natural advantage in characterizing material texture and surface structure features, providing complementary information that infrared images lack, which is needed to distinguish regions with different emissivity. Therefore, how to utilize the complementary information from infrared-visible multi-source images and the radiative transfer mechanism to address the insufficient accuracy of infrared thermometry under non-uniform emissivity conditions and achieve accurate online detection of surface temperature in complex scenes is a pressing technical problem in this field. Summary of the Invention

[0008] To address the aforementioned technical challenges, this invention proposes a multi-source visual temperature measurement method for targets with varying emissivity. This invention utilizes an infrared thermal imager and a visible light camera to simultaneously acquire infrared and visible light images of the target scene, providing a data foundation for subsequent multimodal fusion analysis. To address the issues of significant emissivity differences across different material regions and the potential for systematic temperature measurement errors introduced by unifying emissivity parameters, this invention models the emissivity region segmentation problem as a bimodal semantic segmentation task, achieving pixel-level automatic emissivity region segmentation and identification through a cross-modal fusion mechanism. To address the multi-source uncertainty in emissivity estimation, this invention employs a probabilistic modeling method to model emissivity, generating an emissivity estimation map and an uncertainty map by integrating multi-source uncertainties. To address the issue that emissivity estimation deviations can amplify temperature measurement errors, this invention constructs a temperature compensation model that integrates data-driven and radiation mechanism constraints. Through an uncertainty-guided dynamic equilibrium compensation strategy, combined with an emissivity back-calculation verification and correction mechanism, it achieves accurate temperature detection for targets with multiple emissivity. The temperature measurement method and system proposed in this invention can adapt to complex scenes composed of various materials, possessing advantages such as quantifiable uncertainty, high temperature measurement accuracy, and strong robustness.

[0009] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0010] A multi-source visual thermometry method for targets with different emissivity, the method includes:

[0011] S1. Establish a bimodal semantic segmentation model with visible light and infrared images of the scene as input, identify target regions with different emissivity, and output emissivity region segmentation map and classification confidence map;

[0012] S2. Establish a material emissivity prior library, generate a pixel-by-pixel emissivity map by combining the emissivity region segmentation map and the classification confidence map, and obtain the corresponding uncertainty map by the comprehensive uncertainty quantification method.

[0013] S3. Construct a temperature compensation model for different emissivity targets by integrating neural network compensation and thermal radiation mechanism constraints. Based on the pixel-by-pixel emissivity map, uncertainty map and infrared radiation brightness, dynamically balance the weights of data-driven and physical-driven approaches, and output the compensated temperature value.

[0014] Further, step S1 specifically includes:

[0015] S11. A dual-branch encoder-decoder structure is used to construct a dual-modal semantic segmentation model, using visible light. and infrared images As input, output a multi-scale feature set of visible light. and infrared multi-scale feature sets ;

[0016] S12. Adaptive weighted fusion of visible light multi-scale features and infrared multi-scale features is performed through a cross-modal guided fusion module.

[0017] S13, Combine the multi-scale features The data is fed into the decoder for upsampling and feature integration. The decoder consists of four deconvolutional layer modules. The layer decoding process is as follows:

[0018]

[0019] In the formula, Indicates the deconvolution operation;

[0020] The decoder ultimately outputs a material segmentation map. And classification confidence plot For pixel position Its prediction category is

[0021]

[0022] In the formula, Represents pixels Belongs to the The confidence probability of the emission region satisfies the normalization condition:

[0023]

[0024] Segmentation results Divide the scene into Different emissivity regions .

[0025] Further, step S2 specifically includes:

[0026] S21. For each material appearing in the scene. to its emission rate Modeled as a normal distribution, it is represented as follows:

[0027]

[0028] In the formula, Indicates material The average emissivity, Represents the emission rate variance. Emission prior a priori parameters. The emissivity can be obtained through three methods: first, by measuring the infrared emissivity at a known experimental temperature; second, by consulting publicly available standard material emissivity data tables; and third, by referring to measurement results from published studies.

[0029] S22. Based on the obtained segmentation results and the established emission prior library, each pixel... Classified into the corresponding material category And assign the material a priori mean emissivity. Generate pixel-by-pixel emissivity map :

[0030]

[0031] S23. By integrating uncertainty quantification methods, pixels Emission rate uncertainty at Represented as:

[0032]

[0033] In the formula, , , These are the weighting coefficients for each uncertainty component; To segment uncertainty; Due to boundary uncertainty; The inherent variance of the material;

[0034] Among them, segmentation uncertainty Represented as:

[0035]

[0036] In the formula, This is the proportionality coefficient. For pixels The classification confidence score corresponding to the predicted category;

[0037] Boundary uncertainty Represented as:

[0038]

[0039] In the formula, This is the proportionality coefficient. For pixels The distance to the nearest material boundary, τ is the attenuation constant;

[0040] Material inherent variance Variance parameters derived from the launch test library:

[0041]

[0042] In the formula, For pixels Material The standard deviation of emissivity.

[0043] Furthermore, the specific method for step S3 is as follows:

[0044] For a hybrid pixel at a material boundary, the infrared radiation it receives is a linear superposition of the contributions from the radiation of multiple materials. According to the principle of radiation superposition, the equivalent emissivity of the hybrid pixel is... Represented as:

[0045]

[0046] In the formula, The number of material types in the scene;

[0047] For a single-material pixel in a non-boundary region, its emissivity uncertainty is expressed as: According to error propagation theory, the emissivity uncertainty of mixed pixels Represented as:

[0048]

[0049] In the formula, For the first The prior uncertainty of the emissivity of the material, The uncertainty of the segmentation probability;

[0050] The radiance received by an infrared thermal imager consists of two parts: the target's own radiation and the radiation reflected from the environment. According to Kirchhoff's laws, under thermal equilibrium conditions, the absorptivity of an object equals its emissivity, and the apparent radiance is... Represented as:

[0051]

[0052] In the formula, For temperature Blackbody radiation brightness under the following conditions To achieve the target true temperature, Ambient temperature;

[0053] According to Planck's radiation law, the spectral radiation intensity of a blackbody can be expressed as:

[0054]

[0055] In the formula, The first radiation constant, The second radiation constant, This is the operating wavelength of the infrared thermal imager;

[0056] In temperature measurement applications, the inverse operation of the Planck function is approximated by the Sakuma-Hattori equation, therefore the relationship between radiance and temperature can be simplified as follows:

[0057]

[0058] In the formula, , , These are the fitting coefficients related to the infrared thermal imager;

[0059] Target radiance Represented as

[0060]

[0061] Substituting the above equation into the inverse form of the Sakuma-Hattori equation, the initial temperature is derived. :

[0062]

[0063] A convolutional neural network with an encoder-decoder structure is used to learn the temperature correction. The network takes apparent radiance, emissivity map, emissivity uncertainty map, and ambient temperature as input, and outputs a temperature correction, expressed as:

[0064]

[0065] In the formula, This represents the mapping function of a convolutional neural network. These are the network's learnable parameters;

[0066] Considering the uncertainty in emissivity estimation, a dual-gating mechanism is designed to weight the temperature correction. First, the emissivity uncertainty is normalized:

[0067]

[0068] In the formula, and These are the minimum and maximum values ​​of the uncertainty, respectively.

[0069] Then the gate function Defined as the product of uncertainty gating and learning gating:

[0070]

[0071] In the formula, For adaptive gating weights in network learning;

[0072] Final compensation temperature The initial temperature derived from the physical model is obtained by weighted fusion with the neural network correction.

[0073] .

[0074] Furthermore, in the dual-branch encoder-decoder structure, the visible light image branch uses a Swin-Transformer as the encoder to extract visible light multi-scale features; the infrared image branch uses a ConvNext as the encoder to extract infrared multi-scale features.

[0075] Furthermore, the specific method of step S12 is as follows:

[0076] Construct a pixel-level modal consistency guide map, obtained using normalized dot product correlation:

[0077]

[0078] In the formula, Indicates that along the channel is Normalization, F VIS F represents the characteristics of visible light. IR Indicates visible infrared characteristics;

[0079] Then, spatial gating weights are generated based on the statistical responses of the two modal features, the modal difference term, and the aforementioned correlation-guided information. Channel gating weights are generated based on the channel descriptors obtained from global average pooling and their differences and interaction terms. Multiplying the two together yields the joint gating weight. Based on this, cross-modal adaptive weighted fusion is achieved:

[0080]

[0081] In the formula, This represents element-wise multiplication;

[0082] Simultaneously, a learnable residual enhancement branch is introduced to further mine the nonlinear complementary information between the two modes, thereby obtaining residual features. The final fusion output is:

[0083]

[0084] in, This is the learnable scaling factor.

[0085] Furthermore, the final compensated temperature is obtained in step S3. Subsequently, to ensure that the temperature compensation results conform to the physical laws of infrared radiation, a physical consistency constraint is introduced, which is applied to the compensation temperature. Substitute into the forward radiation model to reconstruct the radiance. Represented as:

[0086]

[0087] Loss of physical consistency To reconstruct the mean square error between the radiation and the measured radiation:

[0088]

[0089] In the formula, and These are the height and width of the image, respectively;

[0090] To enhance the spatial continuity of the temperature field within the same material region, while avoiding excessive smoothing at the boundaries between different materials, a material-perceived smoothing loss is defined. for:

[0091]

[0092] In the formula, and These are the gradient operators for the horizontal and vertical directions, respectively. and The boundary-aware weights are represented as

[0093]

[0094] Let be the boundary sensitivity coefficient. Considering all the above constraints, the total loss function for training the neural network can be expressed as:

[0095]

[0096] In the formula, For sparse truth points, supervise loss. For the correction of the regularization term, , , , These are the weighting coefficients for each loss term;

[0097] To verify the rationality of the emissivity estimate, the compensated temperature was used. Back-calculation verification of emission rate ,get:

[0098]

[0099] Will With input emissivity By comparison, the relative deviation is defined:

[0100]

[0101] like This triggers the emissivity correction mechanism, which then adopts... Update the emissivity estimate and re-execute the temperature compensation process to improve the reliability of the temperature measurement results.

[0102] Compared with the prior art, the present invention has the following beneficial effects:

[0103] 1. The multi-source visual temperature measurement method for targets with different emissivity proposed in this invention covers key aspects such as emissivity region segmentation, uncertainty quantification and temperature adaptive compensation, and realizes online accurate detection of the temperature of multiple emissivity targets;

[0104] 2. This invention uses visible light images and infrared images as inputs, and achieves adaptive weighted fusion of dual-modal features through a cross-modal guided fusion module. It intelligently identifies and segments target regions with different emissivity, solving the problem that a single infrared image is difficult to distinguish the coupled effects of temperature and emissivity.

[0105] 3. A pixel-level emissivity a priori model considering segmentation error, boundary mixing effect and inherent material fluctuation was established, achieving accurate estimation of pixel-by-pixel emissivity and its uncertainty, providing a reliable basis for uncertainty quantification for subsequent temperature compensation;

[0106] 4. This invention integrates neural network compensation and physical constraint layer, and achieves accurate compensation of target temperature with different emissivity through uncertainty gating mechanism adaptive compensation weight, and can still obtain high-precision temperature measurement results in the absence of real temperature calibration.

[0107] 5. This invention introduces an emissivity back-calculation verification and correction mechanism, which further improves the reliability and robustness of the temperature measurement results. Attached Figure Description

[0108] Figure 1 Flowchart of the method of this invention;

[0109] Figure 2 The segmentation result image obtained in the embodiments of the present invention;

[0110] Figure 3 A temperature measurement result image obtained by direct reading from an infrared thermal imager before compensation in an embodiment of the present invention;

[0111] Figure 4 The temperature measurement results after multi-source visual temperature compensation in this embodiment of the invention. Detailed Implementation

[0112] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments:

[0113] Example:

[0114] This embodiment provides a multi-source visual thermometry method for targets with different emissivity, the method including:

[0115] S1. Establish a bimodal semantic segmentation model using visible light and infrared images of the scene as input, identify target regions with different emissivity, and output emissivity region segmentation maps and classification confidence maps, as detailed below:

[0116] S11. A dual-branch encoder-decoder structure is used to construct a dual-modal semantic segmentation model, using visible light. and infrared images As input, output a multi-scale feature set of visible light. and infrared multi-scale feature sets ;

[0117] In the dual-branch encoder-decoder structure, the visible light image branch uses a Swing-Transformer as the encoder to extract visible light multi-scale features; the infrared image branch uses a ConvNext as the encoder to extract infrared multi-scale features.

[0118] S12. Adaptive weighted fusion of visible light multi-scale features and infrared multi-scale features is performed through a cross-modal guided fusion module.

[0119] Construct a pixel-level modal consistency guide map, obtained using normalized dot product correlation:

[0120]

[0121] In the formula, Indicates that along the channel is Normalization, F VIS F represents the characteristics of visible light. IR Indicates visible infrared characteristics;

[0122] Then, spatial gating weights are generated based on the statistical responses of the two modal features, the modal difference term, and the aforementioned correlation-guided information. Channel gating weights are generated based on the channel descriptors obtained from global average pooling and their differences and interaction terms. Multiplying the two together yields the joint gating weight. Based on this, cross-modal adaptive weighted fusion is achieved:

[0123]

[0124] In the formula, This represents element-wise multiplication;

[0125] Simultaneously, a learnable residual enhancement branch is introduced to further mine the nonlinear complementary information between the two modes, thereby obtaining residual features. The final fusion output is:

[0126]

[0127] in, The learnable scaling factor;

[0128] S13, Combine the multi-scale features The data is fed into the decoder for upsampling and feature integration. The decoder consists of four deconvolutional layer modules. The layer decoding process is as follows:

[0129]

[0130] In the formula, Indicates the deconvolution operation;

[0131] The decoder ultimately outputs a material segmentation map. And classification confidence plot For pixel position Its prediction category is

[0132]

[0133] In the formula, Represents pixels Belongs to the The confidence probability of the emission region satisfies the normalization condition:

[0134]

[0135] Segmentation results Divide the scene into Different emissivity regions .

[0136] S2. Establish a material emissivity prior library, generate a pixel-by-pixel emissivity map by combining the emissivity region segmentation map and the classification confidence map, and obtain the corresponding uncertainty map through a comprehensive uncertainty quantification method, as follows:

[0137] S21. For each material appearing in the scene. to its emission rate Modeled as a normal distribution, it is represented as follows:

[0138]

[0139] In the formula, Indicates material The average emissivity, Represents the emission rate variance. Emission prior a priori parameters. The emissivity can be obtained through three methods: first, by measuring the infrared emissivity at a known experimental temperature; second, by consulting publicly available standard material emissivity data tables; and third, by referring to measurement results from published studies.

[0140] S22. Based on the obtained segmentation results and the established emission prior library, each pixel... Classified into the corresponding material category And assign the material a priori mean emissivity. Generate pixel-by-pixel emissivity map :

[0141]

[0142] S23. By integrating uncertainty quantification methods, pixels Emission rate uncertainty at Represented as:

[0143]

[0144] In the formula, , , These are the weighting coefficients for each uncertainty component; To segment uncertainty; Due to boundary uncertainty; The inherent variance of the material;

[0145] Among them, segmentation uncertainty Represented as:

[0146]

[0147] In the formula, This is the proportionality coefficient. For pixels The classification confidence score corresponding to the predicted category;

[0148] Boundary uncertainty Represented as:

[0149]

[0150] In the formula, This is the proportionality coefficient. For pixels The distance to the nearest material boundary, τ is the attenuation constant;

[0151] Material inherent variance Variance parameters derived from the launch test library:

[0152]

[0153] In the formula, For pixels Material The standard deviation of emissivity.

[0154] S3. Construct a temperature compensation model for targets with different emissivity that integrates neural network compensation and thermal radiation mechanism constraints. Based on the pixel-by-pixel emissivity map, uncertainty map, and infrared radiance, dynamically balance the weights of data-driven and physical-driven approaches, and output the compensated temperature value. The specific method is as follows:

[0155] For a hybrid pixel at a material boundary, the infrared radiation it receives is a linear superposition of the contributions from the radiation of multiple materials. According to the principle of radiation superposition, the equivalent emissivity of the hybrid pixel is... Represented as:

[0156]

[0157] In the formula, The number of material types in the scene;

[0158] For a single-material pixel in a non-boundary region, its emissivity uncertainty is expressed as: According to error propagation theory, the emissivity uncertainty of mixed pixels Represented as:

[0159]

[0160] In the formula, For the first The prior uncertainty of the emissivity of the material, The uncertainty of the segmentation probability;

[0161] The radiance received by an infrared thermal imager consists of two parts: the target's own radiation and the radiation reflected from the environment. According to Kirchhoff's laws, under thermal equilibrium conditions, the absorptivity of an object equals its emissivity, and the apparent radiance is... Represented as:

[0162]

[0163] In the formula, For temperature Blackbody radiation brightness under the following conditions To achieve the target true temperature, Ambient temperature;

[0164] According to Planck's radiation law, the spectral radiation intensity of a blackbody can be expressed as:

[0165]

[0166] In the formula, The first radiation constant, The second radiation constant, This is the operating wavelength of the infrared thermal imager;

[0167] In temperature measurement applications, the inverse operation of the Planck function is approximated by the Sakuma-Hattori equation, therefore the relationship between radiance and temperature can be simplified as follows:

[0168]

[0169] In the formula, , , These are the fitting coefficients related to the infrared thermal imager;

[0170] Target radiance Represented as

[0171]

[0172] Substituting the above equation into the inverse form of the Sakuma-Hattori equation, the initial temperature is derived. :

[0173]

[0174] A convolutional neural network with an encoder-decoder structure is used to learn the temperature correction. The network takes apparent radiance, emissivity map, emissivity uncertainty map, and ambient temperature as input, and outputs a temperature correction, expressed as:

[0175]

[0176] In the formula, This represents the mapping function of a convolutional neural network. These are the network's learnable parameters;

[0177] Considering the uncertainty in emissivity estimation, a dual-gating mechanism is designed to weight the temperature correction. First, the emissivity uncertainty is normalized:

[0178]

[0179] In the formula, and These are the minimum and maximum values ​​of the uncertainty, respectively.

[0180] Then the gate function Defined as the product of uncertainty gating and learning gating:

[0181]

[0182] In the formula, For adaptive gating weights in network learning;

[0183] Final compensation temperature The initial temperature derived from the physical model is obtained by weighted fusion with the neural network correction.

[0184] .

[0185] To ensure that the temperature compensation results conform to the physical laws of infrared radiation, a physical consistency constraint is introduced, which will affect the compensation temperature. Substitute into the forward radiation model to reconstruct the radiance. Represented as:

[0186]

[0187] Loss of physical consistency To reconstruct the mean square error between the radiation and the measured radiation:

[0188]

[0189] In the formula, and These are the height and width of the image, respectively;

[0190] To enhance the spatial continuity of the temperature field within the same material region, while avoiding excessive smoothing at the boundaries between different materials, a material-perceived smoothing loss is defined. for:

[0191]

[0192] In the formula, and These are the gradient operators for the horizontal and vertical directions, respectively. and The boundary-aware weights are represented as

[0193]

[0194] Let be the boundary sensitivity coefficient. Considering all the above constraints, the total loss function for training the neural network can be expressed as:

[0195] ;

[0196] In the formula, For sparse truth points, supervise loss. For the correction of the regularization term, , , , These are the weighting coefficients for each loss term;

[0197] To verify the rationality of the emissivity estimate, the compensated temperature was used. Back-calculation verification of emission rate ,get:

[0198]

[0199] Will With input emissivity By comparison, the relative deviation is defined:

[0200]

[0201] like This triggers the emissivity correction mechanism, which then adopts... Update the emissivity estimate and re-execute the temperature compensation process to improve the reliability of the temperature measurement results.

[0202] Taking a material plate with interchangeable targets of different emissivity as the research object, the multi-source visual thermometry method for targets with different emissivity proposed in the above embodiments is applied to the temperature detection of the material plate. The material plate contains a variety of metallic and non-metallic materials with an emissivity ranging from 0.1 to 0.9. Under multiple temperature conditions, dual-modal images of the material plate are simultaneously acquired using an infrared-visible integrated thermal imager.

[0203] like Figure 2 The diagram shows the segmentation results of target regions with different emissivity in a material plate according to the present invention. The present invention can accurately identify and segment target regions of different materials, with clear segmentation boundaries, an average intersection-over-union ratio of 92%, and a pixel accuracy of 96%. To verify the effectiveness of the temperature measurement method of the present invention, the set temperature of the material plate was used as the actual temperature reference value. The temperature measurement results of the present invention were compared and analyzed with the temperature directly read by an infrared thermal imager and the temperature measurement results obtained solely by physical inference methods. Figure 3 To compensate for the temperature measurement results directly read from the front infrared thermal imager, Figure 4 This is the temperature measurement result after multi-source visual temperature compensation according to the present invention. The camera direct-reading temperature does not consider the influence of emissivity differences, and its average absolute error is 18.594. After compensation using only physical back-calculation methods, the mean absolute error was reduced to 15.123. After compensation using the method of this invention, the mean absolute error is further reduced to 4.552. This represents a 10.570 improvement over the physical reverse engineering method. The above results demonstrate that the present invention can eliminate temperature measurement deviations caused by non-uniform emissivity and significantly improve the temperature measurement accuracy of multi-emissivity targets.

[0204] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any other way. Any modifications or equivalent changes made based on the technical essence of the present invention shall still fall within the scope of protection claimed by the present invention.

Claims

1. A multi-source visual thermometry method for targets with different emissivity, characterized in that, The methods include: S1. Establish a bimodal semantic segmentation model with visible light and infrared images of the scene as input, identify target regions with different emissivity, and output emissivity region segmentation map and classification confidence map; S2. Establish a material emissivity prior library, generate a pixel-by-pixel emissivity map by combining the emissivity region segmentation map and the classification confidence map, and obtain the corresponding uncertainty map by the comprehensive uncertainty quantification method. S3. Construct a temperature compensation model for different emissivity targets by integrating neural network compensation and thermal radiation mechanism constraints. Based on the pixel-by-pixel emissivity map, uncertainty map and infrared radiation brightness, dynamically balance the weights of data-driven and physical-driven approaches, and output the compensated temperature value.

2. The multi-source visual thermometry method for targets with different emissivity according to claim 1, characterized in that, Step S1 specifically includes: S11. A dual-branch encoder-decoder structure is used to construct a dual-modal semantic segmentation model, using visible light. and infrared images As input, output a multi-scale feature set of visible light. and infrared multi-scale feature sets ; S12. Adaptive weighted fusion of visible light multi-scale features and infrared multi-scale features is performed through a cross-modal guided fusion module. S13, Combine the multi-scale features The data is fed into the decoder for upsampling and feature integration. The decoder consists of four deconvolutional layer modules. The layer decoding process is as follows: ； In the formula, Indicates the deconvolution operation; The decoder ultimately outputs a material segmentation map. And classification confidence plot For pixel position Its prediction category is ； In the formula, Represents pixels Belongs to the The confidence probability of the emission region satisfies the normalization condition: ； Segmentation results Divide the scene into Different emissivity regions .

3. The multi-source visual thermometry method for targets with different emissivity according to claim 2, characterized in that, Step S2 specifically includes: S21. For each material appearing in the scene. to its emission rate Modeled as a normal distribution, it is represented as follows: ； In the formula, Indicates material The average emissivity, Represents the emission rate variance, and the emission prior a priori parameter. The emissivity can be obtained through three methods: first, by measuring the infrared emissivity at a known experimental temperature; second, by consulting publicly available standard material emissivity data tables; and third, by referring to measurement results from published studies. S22. Based on the obtained segmentation results and the established emission prior library, each pixel... Classified into the corresponding material category And assign the material a priori mean emissivity. Generate pixel-by-pixel emissivity map : ； S23. By integrating uncertainty quantification methods, pixels Emission rate uncertainty at Represented as: ； In the formula, , , These are the weighting coefficients for each uncertainty component; To segment uncertainty; Due to boundary uncertainty; The inherent variance of the material; Among them, segmentation uncertainty Represented as: ； In the formula, This is the proportionality coefficient. For pixels The classification confidence score corresponding to the predicted category; Boundary uncertainty Represented as: ； In the formula, This is the proportionality coefficient. For pixels The distance to the nearest material boundary, τ is the attenuation constant; Material inherent variance Variance parameters derived from the launch test library: ； In the formula, For pixels Material The standard deviation of emissivity.

4. The multi-source visual thermometry method for targets with different emissivity according to claim 3, characterized in that, The specific method for step S3 is as follows: For a hybrid pixel at a material boundary, the infrared radiation it receives is a linear superposition of the contributions from the radiation of multiple materials. According to the principle of radiation superposition, the equivalent emissivity of the hybrid pixel is... Represented as: ； In the formula, The number of material types in the scene; For a single-material pixel in a non-boundary region, its emissivity uncertainty is expressed as: According to error propagation theory, the emissivity uncertainty of mixed pixels Represented as: ； In the formula, For the first The prior uncertainty of the emissivity of the material, The uncertainty of the segmentation probability; The radiance received by an infrared thermal imager consists of two parts: the target's own radiation and the radiation reflected from the environment. According to Kirchhoff's laws, under thermal equilibrium conditions, the absorptivity of an object equals its emissivity, and the apparent radiance is... Represented as: ； In the formula, For temperature Blackbody radiation brightness under the following conditions To achieve the target true temperature, Ambient temperature; According to Planck's radiation law, the spectral radiation intensity of a blackbody can be expressed as: ； In the formula, The first radiation constant, The second radiation constant, This is the operating wavelength of the infrared thermal imager; In temperature measurement applications, the inverse operation of the Planck function is approximated by the Sakuma-Hattori equation, therefore the relationship between radiance and temperature can be simplified as follows: ； In the formula, , , These are the fitting coefficients related to the infrared thermal imager; Target radiance Represented as ； Substituting the above equation into the inverse form of the Sakuma-Hattori equation, the initial temperature is derived. : ； A convolutional neural network with an encoder-decoder structure is used to learn the temperature correction. The network takes apparent radiance, emissivity map, emissivity uncertainty map, and ambient temperature as input, and outputs a temperature correction, expressed as: ； In the formula, This represents the mapping function of a convolutional neural network. These are the network's learnable parameters; Considering the uncertainty in emissivity estimation, a dual-gating mechanism is designed to weight the temperature correction; firstly, the emissivity uncertainty is normalized: ； In the formula, and These are the minimum and maximum values ​​of the uncertainty, respectively. Then the gate function Defined as the product of uncertainty gating and learning gating: ； In the formula, For adaptive gating weights in network learning; Final compensation temperature The initial temperature derived from the physical model is obtained by weighted fusion with the neural network correction. 。 5. A multi-source visual thermometry method for targets with different emissivity according to claim 2, characterized in that, In the dual-branch encoder-decoder structure, the visible light image branch uses a Swin-Transformer as the encoder to extract visible light multi-scale features; the infrared image branch uses a ConvNext as the encoder to extract infrared multi-scale features.

6. A multi-source visual thermometry method for targets with different emissivity according to claim 2, characterized in that, The specific method for step S12 is as follows: Construct a pixel-level modal consistency guide map, obtained using normalized dot product correlation: ； In the formula, Indicates that along the channel is Normalization, F VIS F represents the characteristics of visible light. IR Indicates visible infrared characteristics; Then, spatial gating weights are generated based on the statistical responses of the two modal features, the modal difference term, and the aforementioned correlation-guided information. ; Channel gating weights are generated based on the channel descriptors obtained from global average pooling and their differences and interaction terms. Multiplying the two together yields the joint gating weight. Based on this, cross-modal adaptive weighted fusion is achieved: ； In the formula, This represents element-wise multiplication; Simultaneously, a learnable residual enhancement branch is introduced to further mine the nonlinear complementary information between the two modes, thereby obtaining residual features. The final fusion output is: ； in, This is the learnable scaling factor.

7. A multi-source visual thermometry method for targets with different emissivity according to claim 4, characterized in that, The final compensated temperature is obtained in step S3. Subsequently, to ensure that the temperature compensation results conform to the physical laws of infrared radiation, a physical consistency constraint is introduced, which is applied to the compensation temperature. Substitute into the forward radiation model to reconstruct the radiance. Represented as: ； Loss of physical consistency To reconstruct the mean square error between the radiation and the measured radiation: ； In the formula, and These are the height and width of the image, respectively; To enhance the spatial continuity of the temperature field within the same material region, while avoiding excessive smoothing at the boundaries between different materials, a material-perceived smoothing loss is defined. for: ； In the formula, and These are the gradient operators for the horizontal and vertical directions, respectively. and The boundary-aware weights are represented as ； Given the boundary sensitivity coefficient and considering the above constraints, the total loss function for training the neural network can be expressed as: ； In the formula, For sparse truth points, supervise loss. For the correction of the regularization term, , , , These are the weighting coefficients for each loss term; To verify the rationality of the emissivity estimate, the compensated temperature was used. Back-calculation verification of emission rate ,get: ； Will With input emissivity By comparison, the relative deviation is defined: ； like This triggers the emissivity correction mechanism, which then adopts... Update the emissivity estimate and re-execute the temperature compensation process to improve the reliability of the temperature measurement results.

Images