A visible light camera temperature measurement image denoising method based on deep learning
By using an improved DnCNN network model and data augmentation techniques, the problems of temperature measurement accuracy and image noise when using visible light cameras to measure the temperature of high-temperature objects were solved, and accurate measurement of the surface temperature of high-temperature objects was achieved.
Patent Information
- Application Number
- CN202410802282.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-06-20
- Publication Date
- 2026-08-25
- Estimated Expiration
- 2044-06-20
AI Technical Summary
When existing visible light cameras measure the temperature of high-temperature objects, the radiation is concentrated in the visible light and near-infrared bands, which leads to a decrease in temperature measurement accuracy, and image noise affects the image quality and is difficult to remove effectively.
Using a visible light camera with near-infrared radiative response, an improved DnCNN network model was constructed. Image denoising was performed through sparse convolution and dual attention mechanisms. Combined with data augmentation and temperature-grayscale relationship fitting, the surface temperature of high-temperature objects was measured.
It improves the accuracy and stability of visible light cameras in measuring the temperature of high-temperature objects, and significantly enhances image noise reduction and the accuracy of temperature measurement.
Smart Images
Figure CN118822883B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of visible light camera temperature measurement technology, and specifically to a method for noise reduction of visible light camera temperature measurement images based on deep learning. Background Technology
[0002] Visible light cameras use CMOS or CCD sensors to convert light signals emitted by objects into electrical signals for processing, ultimately transforming them into digital images. Due to their high resolution, fast response speed, low cost, and stable operation, they are widely used in various scenarios. For example, in industrial production, they are used for quality control and inspection, automated production, and production data analysis; in the medical field, visible light cameras are used for medical imaging and disease diagnosis; and in daily life, they play an indispensable role in security deployment, automotive transportation, and digital equipment. Beyond these applications, visible light cameras can also penetrate various other fields and play a crucial role.
[0003] Visible light cameras also have significant applications in temperature measurement, offering clear advantages over commonly used infrared thermal imagers. Traditional infrared thermal imagers are generally used for temperature measurements in environments below 500°C. However, because infrared imagers receive mid-wave infrared radiation, their measurement effectiveness is poor at higher temperatures (above 500°C). As temperature increases further, the radiation emitted by an object increases significantly and concentrates in the near-infrared and visible light bands, making the object appear reddish to the naked eye. This mismatch between the object's radiation and the primary radiation range received by the infrared imager leads to a decrease in measurement accuracy. Therefore, effectively utilizing near-infrared and visible light radiation information is highly valuable for measuring high-temperature objects.
[0004] With advancements in technology, modern visible light cameras can now utilize sensors with broader radiation response bands, or achieve a wider range of radiation responses by embedding two or more sensors with different radiation responses. In the field of visible light camera temperature measurement, one of these methods can be used to achieve responses to both visible and near-infrared radiation, significantly improving the measurement range and performance. Depending on the temperature measurement scenario, visible light cameras with sensors featuring different radiation response characteristics can be selected for temperature measurement. Finally, by acquiring grayscale values from images under specific conditions and using the temperature-grayscale curve obtained after system calibration, the object's temperature can be calculated. Therefore, image quality is a crucial factor limiting the temperature measurement capabilities of visible light cameras, and noise reduction is key to improving image quality. Considering the factors affecting the accuracy of visible light camera temperature measurement, it is necessary to develop a method for noise suppression in images acquired by visible light cameras. Summary of the Invention
[0005] To suppress noise in images captured by a visible light camera during temperature measurement, this invention provides a deep learning-based method for denoising visible light camera temperature measurement images.
[0006] The technical problem to be solved by this invention is achieved by the following technical solution:
[0007] A deep learning-based method for denoising visible light camera thermometric images includes the following steps:
[0008] (1) Blackbody furnace image data were acquired using a visible light camera with near-infrared band radiation response. Gaussian white noise was added to the acquired images. The noisy image data and the corresponding original image data were saved, and the images were augmented and the dataset was divided.
[0009] (2) Construct an improved DnCNN network model that incorporates sparse convolution and dual attention mechanism and verify the noise reduction performance of the network model. Use the training set and test set obtained in step (1) to train the network model and obtain the trained improved DnCNN network model.
[0010] (3) Take images of the blackbody furnace opening at different temperatures using a visible light camera, and use the improved DnCNN network model trained in step (2) to denoise the blackbody furnace images after adding noise, and fit the temperature-grayscale relationship at different temperatures.
[0011] (4) In the actual temperature measurement process, the high-temperature object under test is photographed by a visible light camera, the gray value of the object under test in the acquired image is read, and the surface temperature of the object is calculated according to the relationship obtained by fitting in step (3).
[0012] In a further technical solution, step (1) specifically includes the following steps:
[0013] (1.1) Within the calibrated temperature range T min ~T max Select n temperature calibration points T1, T2, T3...T n The blackbody furnace temperature was set to various temperature calibration points in sequence. After the temperature inside the blackbody furnace stabilized, a set of blackbody furnace images (Frames_Raw) were collected by a visible light camera at different temperatures in the blackbody furnace.
[0014] (1.2) Add Gaussian white noise of different noise levels to the blackbody furnace image Frames_Raw obtained in step (1.1). Since the main noise in the actual measurement system is thermal noise, which is a typical Gaussian white noise, the noise reduction simulation in the actual measurement process is carried out by adding Gaussian white noise to the blackbody furnace image. The image group after adding noise is named Frames_Train.
[0015] (1.3) The noisy blackbody furnace image obtained in step (1.2) is simultaneously subjected to horizontal or vertical flipping, rotation, random step-size translation and cropping to achieve image data enhancement. The noisy image data and the corresponding original image data are saved respectively, and the dataset images are divided into training set and validation set according to the proportion.
[0016] In a further technical solution, step (2) includes the following steps:
[0017] (2.1) An improved DnCNN network model is constructed by introducing sparse convolution and dual attention mechanism. The convolutional layers in the original DnCNN network model are replaced with sparse convolution, and channel attention and spatial attention modules are added after the other sparse convolutional layers except the first and last sparse convolutional layers.
[0018] (2.2) The denoising performance of the improved DnCNN network model was verified using the set12 image set. Data augmentation operations such as cropping, scaling, and rotation were performed on the images in the image set, and Gaussian white noise with different noise levels was added. The original images and the noisy images were divided into training set and validation set according to the ratio. The network model was trained using the training set images, and then the denoising performance of the network model was verified and compared with other denoising methods.
[0019] (2.3) Use the training set obtained in step (1.3) to train the network and obtain the trained improved DnCNN network model.
[0020] In a further technical solution, step (3) includes the following steps:
[0021] (3.1) Set the blackbody furnace temperature to each temperature calibration point in sequence, and take pictures of the blackbody furnace opening at each temperature calibration point after the internal temperature of the blackbody furnace stabilizes.
[0022] (3.2) The improved DnCNN network model trained in step (2.3) is used to denoise the image, and the gray value and the corresponding temperature value at the furnace mouth of the blackbody furnace image after denoising are recorded.
[0023] (3.3) The grayscale values at the furnace mouth of the blackbody furnace image obtained in step (3.2) are fitted with the corresponding temperature values. The fitting formula is shown in equation (1):
[0024]
[0025] Where G is the gray value at the furnace mouth position in the blackbody furnace image, T is the temperature value obtained after calculation by the fitting formula, and T0, A1, A2, t1, and t2 are constants.
[0026] In a further technical solution, the actual temperature measurement step in step (4) is to take a picture of the high-temperature object being measured using a visible light camera with the same settings as in step (3). The settings include exposure time, aperture number, filter, etc. The surface temperature of the high-temperature object being measured is calculated using the fitting relationship obtained in step (3.3), thereby realizing the surface temperature measurement of the high-temperature object.
[0027] The beneficial effects of this invention are as follows: Compared with the prior art, this invention solves the problem of decreased temperature measurement accuracy caused by the concentration of radiation emitted by high-temperature objects in the visible light and near-infrared bands by using a visible light camera with near-infrared radiation response for temperature measurement. Furthermore, by modifying and adjusting the DnCNN network structure, an improved DnCNN network is constructed to suppress noise in the images captured by the visible light camera, further improving the measurement accuracy and stability of the visible light camera in the process of measuring the surface temperature of high-temperature objects. Attached Figure Description
[0028] Figure 1 This is a flowchart illustrating the process of image denoising and temperature measurement based on deep learning.
[0029] Figure 2 Diagram of the improved DnCNN network architecture;
[0030] Figure 3 A comparison chart showing the image noise reduction effects of different methods on the test image;
[0031] Figure 4 The images show the image denoising performance of the improved DnCNN network at different noise levels.
[0032] Figure 5 Comparison of PSNR values (a) and SSIM values (b) for denoising blackbody furnace images using different methods;
[0033] Figure 6 A schematic diagram showing the actual calibration of blackbody furnace grayscale values and corresponding blackbody furnace temperatures.
[0034] Figure 7 The graph shows the fitting curves between the grayscale of the blackbody furnace and the temperature of the blackbody furnace.
[0035] Figure 8 This is a comparison chart showing the difference between the actual blackbody furnace temperature and the fitted calculated temperature before and after using the improved DnCNN network for noise reduction. Detailed Implementation
[0036] To make the technical means, creative features, objectives and effects of this invention easier to understand, the invention will be further described below with reference to specific embodiments and illustrations.
[0037] Example 1
[0038] This embodiment provides a method for denoising visible light camera temperature measurement images based on deep learning, specifically including the following steps:
[0039] S1: The visible light camera used in this embodiment is the MER-507-23GM NIR camera from Daheng Imaging, which has a near-infrared radiation response and can capture 12-bit grayscale images. Furthermore, based on the experimental environment and temperature requirements, a filter with a center wavelength of 865nm and a half-width at half-maximum (FWHM) of 10nm was selected. This filter, combined with the visible light camera, effectively filters out background radiation and reduces interference. A basic visible light camera temperature measurement system for calibration was then constructed based on the visible light camera, filter, and blackbody furnace.
[0040] S2: After completing the equipment selection, begin preparing for the calibration experiment. The overall process is as follows: Figure 1 As shown. In this embodiment, the blackbody furnace temperature is set to 800–1200°C for grayscale-temperature response calibration. The blackbody furnace temperature is set to the highest temperature T in the calibration range. max That is, set the temperature to 1200℃ and wait for 20 minutes to allow the temperature inside the furnace to stabilize.
[0041] S3: Place a visible light camera with a filter at a distance of 1 meter from the blackbody furnace opening. Adjust the camera's exposure time and gain so that the grayscale value at the center of the furnace opening in the image captured by the camera is close to but does not exceed 4096. Record the temperature measurement system settings at this time, including camera exposure time and gain, lens aperture and focal length, and distance between the camera and the furnace opening.
[0042] S4: Within the calibrated temperature range, select a calibration temperature point every 50°C, for a total of 8 calibration temperature points. Set the blackbody furnace temperature to each calibration temperature point and wait 20 minutes for the furnace temperature to stabilize. Then, take images of the blackbody furnace at that calibration temperature, for a total of 9 sets, named Frames_Raw.
[0043] S5: Add Gaussian white noise of different levels to each image in the multiple sets of blackbody furnace images Frames_Raw obtained in step S4. Simultaneously perform the same degree of vertical or horizontal flipping, rotation, random step-size translation, and cropping operations on both the original image and the image with added noise to expand the sample size and achieve data augmentation, resulting in the noisy image set Frames_Train. Then, save the original blackbody furnace image and the corresponding noisy image for subsequent deep learning denoising processing.
[0044] S6: An improved DnCNN network model is constructed by introducing a dual attention mechanism and sparse convolution into the original DnCNN network. The noisy image is taken as input, processed by the improved DnCNN network to obtain a residual image. The denoised image is obtained by subtracting the noisy image from the residual image. The improved DnCNN network structure is as follows: Figure 2 As shown.
[0045] Specifically, the model modification involves first replacing the standard convolutional layers in the original DnCNN network with sparse convolutional layers. Then, channel attention modules and spatial attention modules are added after the sparse convolutional layers (excluding the first and last layers). During the forward propagation, the input feature map X first undergoes convolution operations in the sparse convolutional layers, followed by global average pooling in the channel attention module of the dual attention mechanism. The dependencies between channels are learned through fully connected layers, and the sigmoid activation function is applied to obtain channel attention weights. These weights are then multiplied element-wise with the input feature map to obtain the channel-weighted feature map X_c. The feature map X_c is then input into the spatial attention module. After channel pooling, concatenation, and convolution, the sigmoid activation function is applied again to obtain spatial attention weights, which are then multiplied with the input feature map X_c to obtain the spatially weighted feature map X_cs. The final output feature map is the channel-weighted and spatially weighted feature map X_cs.
[0046] Sparse convolution reduces model parameters and computational cost, while the introduction of dual attention mechanism helps the network better focus on important features and suppress the influence of noise and irrelevant features, thereby improving the model's noise reduction performance.
[0047] S7: The denoising performance of the improved DnCNN network model was validated using the set12 graph dataset, with Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index (SSIM) as evaluation metrics. A higher PSNR value and an SSIM value closer to 1 indicate a more significant denoising effect.
[0048] Figure 3 Table 1 shows the results of testing a specific image from the set12 dataset using different methods at a noise level of σ = 25. The average noise reduction performance of the test image from the set12 dataset under each method at this noise level is shown in Table 1.
[0049] Table 1 shows the comparison of noise reduction performance of each method on the set12 dataset.
[0050]
[0051] As shown in Table 1, the method proposed in this invention significantly improves the peak signal-to-noise ratio (PSNR) and structural similarity metrics compared to denoising methods such as bilateral filtering, Gaussian filtering, wavelet transform, NLM, and BM3D. Furthermore, compared to the original DnCNN network, this method improves the PSNR by 1.81 dB (approximately 6.92%) and structural similarity by 0.0387 (approximately 5.23%). In addition, this method also outperforms the original network in training efficiency. For example, with 100 iterations, the original DnCNN network takes 18 minutes and 47 seconds to train, while the improved DnCNN network proposed in this invention takes only 8 minutes and 17 seconds for the same number of iterations—less than half the time of the original method. Moreover, it maintains the advantage of improved denoising performance while reducing the time required, greatly enhancing the efficiency of network training and subsequent denoising processing.
[0052] S8: After verifying the denoising performance of the improved network in step S6, a denoising experiment was conducted on the blackbody furnace image. The blackbody furnace temperature was set to a value different from the calibration point temperature in the above steps, and an image of the blackbody furnace was captured at a certain temperature. In this embodiment, the blackbody furnace temperature was set to 1075℃. At this temperature, the grayscale value obtained from the image would not be too small or too large, otherwise, the denoising effect would not be significantly different. Similarly, the peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM) were used to verify the denoising effect of the improved network in the blackbody furnace image denoising process. The denoising effect at this temperature and after adding different noise levels was obtained as follows: Figure 4 As shown. The denoising effect of the method proposed in this invention compared with other denoising methods at different noise levels is shown in the figure. Figure 5 As shown. Compared to the original DnCNN network denoising method, the improved method proposed in this invention not only improves denoising performance but also significantly increases training speed. Taking 100 iterations as an example, the original DnCNN network training takes 32 minutes and 49 seconds, while the method proposed in this invention takes only 17 minutes and 9 seconds.
[0053] S9: If the denoising verification shows good results, the average grayscale value within the center range of the furnace opening in the image obtained after adding and reducing noise is taken as the grayscale value corresponding to that temperature. This yields the corresponding data between the blackbody furnace temperature and the grayscale of the blackbody furnace image, as follows: Figure 6 As shown.
[0054] However, in the actual temperature measurement process, the measured data is grayscale value. Therefore, it is necessary to fit a fitting curve with grayscale value as the independent variable and temperature as the dependent variable. The fitting formula is shown in equation (1):
[0055]
[0056] Where G is the gray value at the furnace mouth position in the blackbody furnace image, T is the temperature value obtained after calculation by the fitting formula, and T0, A1, A2, t1, and t2 are constants.
[0057] The final fitted curve obtained based on the calibration is as follows: Figure 7 As shown, the detailed parameters of the formula are shown in Table 2.
[0058] Table 2. Parameters of Fitted Curve
[0059]
[0060]
[0061] S10: Verify the fitting results. First, fit the grayscale data of the original blackbody furnace image without noise reduction to the corresponding temperature data using the same formula, following the method described above. Then, take a set of blackbody furnace images (Frames_Test) at different temperature points than those selected in step S4.
[0062] In this embodiment, temperatures of 825℃, 875℃, 925℃...1175℃ were selected, with each temperature point representing a 50℃ interval. The blackbody furnace was set to these temperatures and allowed to stabilize for 20 minutes. Images of the blackbody furnace at these temperatures were then captured. Noise was added to and reduced in these images. The grayscale values at the center of the furnace opening were then substituted into the images before and after noise reduction to calculate the corresponding temperature values. The deviation between these calculated and actual temperature values was analyzed. Finally, a comparison was obtained between the actual temperature and the calculated temperature before and after noise reduction, and the difference between the actual and actual temperatures. Figure 8 As shown in Table 3, detailed comparison data before and after noise reduction are presented.
[0063] Table 3. Validation data of fitting effect
[0064]
[0065] As shown in Table 3, the temperature measurement accuracy of the visible light camera was significantly improved after denoising using the method proposed in this invention. The average accuracy improved by 2.91% in the verification phase. Furthermore, without denoising using this method, the standard deviation of the difference between the fitted and actual temperatures was 4.61℃, while after denoising using this method, the standard deviation was only 2.06℃, indicating a certain improvement in data processing stability. Considering the advantages of this method over other denoising methods, especially the original DnCNN network denoising method, in terms of network training, denoising, and improved temperature measurement accuracy, it can be concluded that the method proposed in this invention is feasible and reliable, and has a positive effect on improving the temperature measurement accuracy of visible light cameras.
[0066] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of this invention is defined by the appended claims and their equivalents.
Claims
1. A method for denoising visible light camera thermometric images based on deep learning, characterized in that, Includes the following steps: (1) Use a visible light camera with near-infrared band radiation response to acquire blackbody furnace image data, add Gaussian white noise to the acquired images, save the noisy image data and the corresponding original image data respectively, and perform data enhancement and dataset partitioning on the images. (2) Construct an improved DnCNN network model that incorporates sparse convolution and dual attention mechanism and verify the noise reduction performance of the network model. Train the network model using the training set and test set obtained in step (1) to obtain the trained improved DnCNN network model. The improved DnCNN network model replaces the convolutional layers in the original DnCNN network model with sparse convolution, and adds channel attention modules and spatial attention modules after the other sparse convolutional layers except for the first and last sparse convolutional layers. (3) Take images of the blackbody furnace opening at different temperatures using a visible light camera, and use the improved DnCNN network model trained in step (2) to denoise the blackbody furnace images after adding noise, and fit the temperature-grayscale relationship at different temperatures. (4) In the actual temperature measurement process, the high-temperature object under test is photographed by a visible light camera, the gray value of the object under test in the acquired image is read, and the surface temperature of the object is calculated according to the relationship obtained by fitting in step (3); the temperature of the high-temperature object is 800~1200℃.
2. The method for denoising visible light camera temperature measurement images based on deep learning according to claim 1, characterized in that, Step (1) specifically includes the following steps: (1.1) Within the calibrated temperature range T min ~T max Select n temperature calibration points T1, T2, T3...T n The blackbody furnace temperature was set to each temperature calibration point in sequence. After the temperature inside the blackbody furnace stabilized, a set of blackbody furnace images (Frames_Raw) were collected by a visible light camera at different temperatures in the blackbody furnace. (1.2) Add Gaussian white noise of different noise levels to the blackbody furnace image Frames_Raw obtained in step (1.1), and name the image group after adding noise as Frames_Train; (1.3) Simultaneously perform data augmentation on the noisy blackbody furnace image obtained in step (1.2) and the original blackbody furnace image, save the noisy image data and the corresponding original image data respectively, and divide the dataset images into training set and validation set according to the ratio.
3. The method for denoising visible light camera temperature measurement images based on deep learning according to claim 2, characterized in that, Step (2) includes the following steps: (2.1) An improved DnCNN network model is constructed by introducing sparse convolution and dual attention mechanism. The convolutional layers in the original DnCNN network model are replaced with sparse convolution, and channel attention module and spatial attention module are added after the other sparse convolutional layers except the first and last sparse convolutional layers respectively. (2.2) The denoising performance of the improved DnCNN network model was verified using the set12 image set. Data augmentation was performed on the images in the image set, and Gaussian white noise with different noise levels was added. The original images and the noisy images were divided into training set and validation set according to the ratio. The network model was trained using the training set images. Then the denoising performance of the network model was verified, and the denoising effect was compared with other denoising methods. (2.3) Use the training set obtained in step (1.3) to train the network and obtain the trained improved DnCNN network model.
4. The method for denoising visible light camera thermometry images based on deep learning according to claim 3, characterized in that, Step (3) includes the following steps: (3.1) Set the blackbody furnace temperature to each temperature calibration point in sequence, and take pictures of the blackbody furnace opening at each temperature calibration point after the internal temperature of the blackbody furnace stabilizes. (3.2) The improved DnCNN network model trained in step (2.3) is used to denoise the image, and the gray value and the corresponding temperature value at the furnace mouth of the blackbody furnace image after denoising are recorded. (3.3) The grayscale values at the furnace mouth of the blackbody furnace image obtained in step (3.2) are fitted with the corresponding temperature values. The fitting formula is shown in equation (1): (1) Where G is the gray value at the furnace mouth position in the blackbody furnace image, T is the temperature value obtained after calculation by the fitting formula, and T0, A1, A2, t1, and t2 are constants.
5. The method for denoising visible light camera temperature measurement images based on deep learning according to claim 4, characterized in that, The actual temperature measurement step in step (4) involves taking a picture of the high-temperature object being measured using a visible light camera with the same settings as in step (3), and calculating the surface temperature of the high-temperature object using the fitting formula obtained in step (3.3), thereby realizing the surface temperature measurement of the high-temperature object.