Infrared and visible light dynamic temperature measurement method based on multispectral fusion

Abstract

The invention discloses an infrared and visible light dynamic temperature measurement method based on multispectral fusion. The method specifically comprises the following steps: S1, synchronously acquiring infrared thermal radiation data and visible light image data through a multi-modal data synchronous acquisition module; s2, performing feature matching and region segmentation on the acquired data based on a multispectral data fusion module; s3, environment interference and heat conduction errors are corrected through a dynamic compensation algorithm module; s4, a visual temperature distribution diagram is generated based on the temperature field reconstruction module, and abnormal temperature early warning is triggered, the method is based on a transfer learning method, on the basis of an original task domain model, small sample transfer to a new task domain is achieved, the limitation of sample dependence is relieved, and the universal generalization of the model is improved.

Description

Technical Field

[0001] The present invention relates to the field of dynamic temperature measurement in the field of industrial monitoring combining infrared thermal imaging with visible light image analysis, and specifically to a dynamic temperature measurement method of infrared and visible light based on multi-spectral fusion. Background Art

[0002] Multimodal data fusion:

[0003] Synchronously collect infrared thermal radiation data and visible light image data;

[0004] Establish a feature matching model based on deep learning (such as YOLO+U-Net hybrid network);

[0005] Use visible light images to assist in locating key temperature measurement areas;

[0006] Motion compensation algorithm:

[0007] Develop environmental parameter compensation module (temperature and humidity / distance / reflectivity);

[0008] Construct a heat conduction model to correct the difference between surface temperature and internal temperature;

[0009] Hardware innovation:

[0010] Dual-path coaxial optical system design (shared objective lens + beam splitter prism);

[0011] Integrates a high-precision infrared sensor (8-14μm band) and a 4K visible light camera. Summary of the Invention

[0012] In view of the limitations of existing infrared temperature measurement technology: easily affected by ambient temperature, humidity, and distance; difficulty in achieving accurate target positioning with a single infrared spectrum; inability to simultaneously obtain the morphological characteristics of the measured object; shortcomings of visible light temperature measurement technology: reliance on the reflectivity of the object surface; inability to directly obtain thermal radiation information; defects of existing solutions: large temperature measurement errors with a single sensor; and immature multi-sensor data fusion algorithms. This patent proposes a dynamic temperature measurement method based on infrared and visible light multi-spectral fusion, which specifically includes the following steps:

[0013] S1. Synchronously acquire infrared thermal radiation data and visible light image data through a multimodal data synchronization acquisition module;

[0014] S2. Perform feature matching and region segmentation on the collected data based on the multispectral data fusion module;

[0015] S3. Correct environmental interference and heat conduction errors through the dynamic compensation algorithm module;

[0016] S4. Generate a visual temperature distribution map based on the temperature field reconstruction module and trigger an abnormal temperature warning.

[0017] Preferably, the specific steps of the multimodal data synchronous acquisition module in step S1 are:

[0018] S11. Start the dual-path coaxial optical system and initialize the infrared sensor and visible light camera;

[0019] S12. Synchronously collect infrared thermal radiation data (8-14μm band) and visible light images (RGB + depth information).

[0020] Preferably, the preprocessing step of the infrared thermal radiation data in step S12 is:

[0021] S121. Eliminate infrared sensor pixel response differences based on the non-uniformity correction algorithm (NUC). The formula is as follows:

[0022]

[0023] Among them, B(i,j) is the pixel bias and G(i,j) is the pixel gain;

[0024] S122. Use wavelet denoising algorithm (Daubechies-4 basis function) to suppress thermal radiation noise;

[0025] S123. Convert radiation intensity to temperature based on Planck's blackbody radiation formula:

[0026]

[0027] Where c1 and c2 are radiation constants, λ is the wavelength, and L is the radiation intensity;

[0028] S13. Align and encapsulate the collected data with the timestamp, including:

[0029] S131. Ensure synchronization of infrared and visible light acquisition frame rates (≥30fps) through hardware trigger signals.

[0030] S132. Use JSON format to encapsulate timestamp, environmental parameters (temperature, humidity / distance) and raw data.

[0031] Preferably, the specific steps of the multispectral data fusion module in step S2 are:

[0032] S21. Build a hybrid network based on YOLOv5+U-Net, where:

[0033] S211.YOLOv5 network extracts texture and geometric features of visible light images;

[0034] S212.U-Net network extracts temperature gradient features of infrared thermal images;

[0035] S22. Pixel-level alignment is achieved through a cross-modal spatial registration algorithm, as follows:

[0036]

[0037] Among them, the affine matrix parameters are optimized through SIFT feature matching;

[0038] S23. Use the Mask R-CNN network to segment the key temperature measurement area and dynamically mark the ROI (region of interest).

[0039] Preferably, the specific steps of the dynamic compensation algorithm module in step S3 are:

[0040] S31. Environmental parameter compensation:

[0041] S311. Correct the radiation attenuation error based on the TOF ranging data. The formula is as follows:

[0042] T corrected =T raw ·e k·d

[0043] Wherein, k is the medium attenuation coefficient, and d is the measurement distance;

[0044] S312. Compensate for the effects of ambient temperature and humidity by table lookup interpolation;

[0045] S32. Heat conduction model compensation:

[0046] S321. Establish three-dimensional transient heat conduction equation:

[0047]

[0048] S322. Use the finite element method to solve the mapping relationship from surface temperature to internal temperature.

[0049] Preferably, the specific steps of the temperature field reconstruction module in step S4 are:

[0050] S41. Generate a continuous temperature field based on the Kriging interpolation algorithm. The formula is as follows:

[0051]

[0052] Among them, the weight w i Optimization through semivariogram;

[0053] S42. Visualization output:

[0054] S421. Overlay the temperature field onto the visible light image, and use Jet color scale for pseudo color mapping.

[0055] Compared with the prior art, the present invention has the following beneficial effects:

[0056] 1. This method performs mosaic enhancement on the dataset, enriches the background information of the image, and expands the scale of the dataset.

[0057] 2. This method uses the advanced YOLOv8 target detection network as the backbone network, and adds an attention mechanism module on this basis, so that the network can learn more robust static features, improve the detection accuracy of protective equipment, and reduce the occurrence of missed detection and false detection.

[0058] This method is based on the transfer learning method. On the basis of the original task domain model, it realizes the migration of small samples to the new task domain, alleviates the limitations of sample dependence, and improves the generalization of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention.

[0060] In the attached figure:

[0061] Figure 1 Technology roadmap;

[0062] Figure 2 Infrared semantic segmentation to recognize portraits;

[0063] Figure 3 Visible light converter valve device;

[0064] Figure 4 Converter valve device under infrared perspective;

[0065] Figure 5 Infrared-saturated reactor;

[0066] Figure 6 IR-insulator string;

[0067] Figure 7 IR-damping capacitor. DETAILED DESCRIPTION

[0068] The preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0069] Example: Figure 1-7 As shown, the present invention provides a technical solution, a dynamic temperature measurement method of infrared and visible light based on multi-spectral fusion, which specifically includes the following steps:

[0070] S1. Synchronously acquire infrared thermal radiation data and visible light image data through a multimodal data synchronization acquisition module;

[0071] S2. Perform feature matching and region segmentation on the collected data based on the multispectral data fusion module;

[0072] S3. Correct environmental interference and heat conduction errors through the dynamic compensation algorithm module;

[0073] S4. Generate a visual temperature distribution map based on the temperature field reconstruction module and trigger an abnormal temperature warning.

[0074] Preferably, the specific steps of the multimodal data synchronous acquisition module in step S1 are:

[0075] S11. Start the dual-path coaxial optical system and initialize the infrared sensor and visible light camera;

[0076] S12. Synchronously collect infrared thermal radiation data (8-14μm band) and visible light images (RGB + depth information).

[0077] Preferably, the pre-processing step of the infrared thermal radiation data in step S12 is:

[0078] S121. Eliminate infrared sensor pixel response differences based on the non-uniformity correction algorithm (NUC). The formula is as follows:

[0079]

[0080] Among them, B(i,j) is the pixel bias and G(i,j) is the pixel gain;

[0081] S122. Use wavelet denoising algorithm (Daubechies-4 basis function) to suppress thermal radiation noise;

[0082] S123. Convert radiation intensity to temperature based on Planck's blackbody radiation formula:

[0083]

[0084] Where c1 and c2 are radiation constants, λ is the wavelength, and L is the radiation intensity;

[0085] S13. Align and encapsulate the collected data with the timestamp, including:

[0086] S131. Ensure synchronization of infrared and visible light acquisition frame rates (≥30fps) through hardware trigger signals.

[0087] S132. Use JSON format to encapsulate timestamp, environmental parameters (temperature, humidity / distance) and raw data.

[0088] Preferably, the specific steps of the multispectral data fusion module in step S2 are:

[0089] S21. Build a hybrid network based on YOLOv5+U-Net, where:

[0090] S211.YOLOv5 network extracts texture and geometric features of visible light images;

[0091] S212.U-Net network extracts temperature gradient features of infrared thermal images;

[0092] S22. Pixel-level alignment is achieved through a cross-modal spatial registration algorithm, as follows:

[0093]

[0094] Among them, the affine matrix parameters are optimized through SIFT feature matching;

[0095] S23. Use the Mask R-CNN network to segment the key temperature measurement area and dynamically mark the ROI (region of interest).

[0096] Preferably, the specific steps of the dynamic compensation algorithm module in step S3 are:

[0097] S31. Environmental parameter compensation:

[0098] S311. Correct the radiation attenuation error based on the TOF ranging data. The formula is as follows:

[0099] T corrected =T raw ·e k·d

[0100] Wherein, k is the medium attenuation coefficient, and d is the measurement distance;

[0101] S312. Compensate for the effects of ambient temperature and humidity by table lookup interpolation;

[0102] S32. Heat conduction model compensation:

[0103] S321. Establish three-dimensional transient heat conduction equation:

[0104]

[0105] S322. Use the finite element method to solve the mapping relationship from surface temperature to internal temperature.

[0106] Preferably, the specific steps of the temperature field reconstruction module in step S4 are:

[0107] S41. Generate a continuous temperature field based on the Kriging interpolation algorithm. The formula is as follows:

[0108]

[0109] Among them, the weight w i Optimization through semivariogram;

[0110] S42. Visualization output:

[0111] S421. Overlay the temperature field onto the visible light image, and use Jet color scale for pseudo color mapping.

[0112] Finally, it should be noted that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art will be able to modify the technical solutions described in the aforementioned embodiments or substitute equivalents for some of the technical features. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.

Claims

1. A dynamic temperature measurement method based on infrared and visible light multi-spectral fusion, characterized in that: The specific steps include: S1. Synchronously acquire infrared thermal radiation data and visible light image data through a multimodal data synchronization acquisition module; S2. Perform feature matching and region segmentation on the collected data based on the multispectral data fusion module; S3. Correct environmental interference and heat conduction errors through the dynamic compensation algorithm module; S4. Generate a visual temperature distribution map based on the temperature field reconstruction module and trigger an abnormal temperature warning.

2. The method for dynamic temperature measurement based on infrared and visible light multi-spectral fusion according to claim 1, characterized in that: The specific steps of the multimodal data synchronous acquisition module in step S1 are: S11. Start the dual-path coaxial optical system and initialize the infrared sensor and visible light camera; S12. Synchronously collect infrared thermal radiation data (8-14μm band) and visible light images (RGB + depth information).

3. The method for dynamic temperature measurement based on infrared and visible light multi-spectral fusion according to claim 2, characterized in that: The pre-processing steps of the infrared thermal radiation data in step S12 are: S121. Eliminate infrared sensor pixel response differences based on the non-uniformity correction algorithm (NUC). The formula is as follows: Among them, B(i,j) is the pixel bias and G(i,j) is the pixel gain; S122. Use wavelet denoising algorithm (Daubechies-4 basis function) to suppress thermal radiation noise; S123. Convert radiation intensity to temperature based on Planck's blackbody radiation formula: Where c1 and c2 are radiation constants, λ is the wavelength, and L is the radiation intensity; S13. Align and encapsulate the collected data with the timestamp, including: S131. Ensure synchronization of infrared and visible light acquisition frame rates (≥30fps) through hardware trigger signals. S132. Use JSON format to encapsulate timestamp, environmental parameters (temperature, humidity / distance) and raw data.

4. The method for dynamic temperature measurement based on infrared and visible light multi-spectral fusion according to claim 1, characterized in that: The specific steps of the multispectral data fusion module in step S2 are: S21. Build a hybrid network based on YOLOv5+U-Net, where: S211.YOLOv5 network extracts texture and geometric features of visible light images; S212.U-Net network extracts temperature gradient features of infrared thermal images; S22. Pixel-level alignment is achieved through a cross-modal spatial registration algorithm, as follows: Among them, the affine matrix parameters are optimized through SIFT feature matching; S23. Use the Mask R-CNN network to segment the key temperature measurement area and dynamically mark the ROI (region of interest).

5. The method for dynamic temperature measurement based on infrared and visible light multi-spectral fusion according to claim 1, characterized in that: The specific steps of the dynamic compensation algorithm module in step S3 are: S31. Environmental parameter compensation: S311. Correct the radiation attenuation error based on the TOF ranging data. The formula is as follows: T corrected =T raw ·e k·d Wherein, k is the medium attenuation coefficient, and d is the measurement distance; S312. Compensate for the effects of ambient temperature and humidity by table lookup interpolation; S32. Heat conduction model compensation: S321. Establish three-dimensional transient heat conduction equation: S322. Use the finite element method to solve the mapping relationship from surface temperature to internal temperature.

6. The method for dynamic temperature measurement based on infrared and visible light multi-spectral fusion according to claim 1, characterized in that: The specific steps of the temperature field reconstruction module in step S4 are: S41. Generate a continuous temperature field based on the Kriging interpolation algorithm. The formula is as follows: Among them, the weight w i Optimization through semivariogram; S42. Visualization output: S421. Overlay the temperature field onto the visible light image, and use Jet color scale for pseudo color mapping.

Images