A Terahertz Image Optimization Method Based on Deep Learning
By constructing a 20-layer neural network for terahertz image optimization, combining batch normalization and residual network, the problems of low-quality terahertz image processing efficiency and subjective evaluation are solved, and efficient and objective image optimization effect is achieved.
Patent Information
- Application Number
- CN202211522224.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-11-30
- Publication Date
- 2025-07-25
- Estimated Expiration
- 2042-11-30
AI Technical Summary
The prior art is inefficient and lacks objective evaluation when processing low-quality terahertz images, and the traditional digital image processing method takes a long time and has strong subjective effects.
A 20-layer neural network is adopted, including input layer, convolutional layer, hidden layer and output layer, combined with batch normalization and residual network, image optimization is used to use deep learning, and objective evaluation is performed through PSNR and SSIM.
Efficient image optimization is achieved, with a single image optimization time of only 1.79 seconds. After optimization, the image PSNR reaches 29.55 and the SSIM reaches 0.70, and the effect is objective and quantifiable.
Smart Images
Figure CN115984123B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of terahertz image optimization, and particularly relates to a terahertz image optimization method based on deep learning. Background Art
[0002] Terahertz waves are electromagnetic waves with frequencies between microwaves and infrared rays. Their wavebands can cover the characteristic spectra of various materials and have good penetration. Terahertz imaging technology analyzes and processes by recording the two-dimensional information of the amplitude and phase of the transmission spectrum, absorption spectrum, or reflection spectrum of a sample, and uses the difference in transmission, absorption, and reflection coefficients of different parts of the sample for imaging.
[0003] Since terahertz attenuates rapidly in air, at the same time, there are many microwave devices in laboratory equipment, which easily affect the transmission of terahertz and are likely to cause environmental noise. And the terahertz imaging system includes equipment such as a stepping machine and a chopper, which are prone to slight vibrations during operation, and there will also be electrical noise between the equipment, thus causing system noise. Under the influence of the above various noises, the imaging quality of the terahertz imaging system is easily degraded, resulting in problems such as blurred images and low signal-to-noise ratio.
[0004] The traditional method to solve this problem is to denoise low-quality terahertz images one by one through digital image processing. However, this method has a long processing time and low work efficiency. When faced with a large number of low-quality terahertz images that need to be processed, it is difficult to solve this problem. At the same time, the terahertz images processed by digital image processing methods are subjective, and there are no objective indicators to evaluate the processed images.
[0005] Based on the above technical problems, there is an urgent need for an efficient image optimization method for low-quality terahertz images, and an objective evaluation of the optimized image quality through quantitative indicators. Summary of the Invention
[0006] Aiming at the problems and deficiencies existing in the above-mentioned prior art, the purpose of the present invention is to provide a terahertz image optimization method based on deep learning, which performs blind denoising optimization through neural network learning and has a good optimization effect on low-quality terahertz images. Technical Solution
[0007] To achieve the above invention purpose, the technical solution provided by the present invention is a terahertz image optimization method based on deep learning, including the following steps:
[0008] S1, construct a neural network with 20 layers including 3 different types of network layers.
[0009] S2, preprocess the terahertz image.
[0010] S3, output the predicted image and determine the optimization quality.
[0011] Further, the steps of constructing a neural network with 20 layers including 3 different types of network layers in S1 are as follows:
[0012] S1.1, construct the input layer and the convolutional layer: Input the training image data into the convolutional layer, use 64 convolutional kernels of 3×3×c to extract the original terahertz image, generate 64 feature maps, and the corresponding output feature value is y. c represents the number of image channels. Use the rectified linear unit ReLU(y) for non-linear processing. The ReLU(y) = max(y, 0), which means comparing the output feature value y with 0 and taking the maximum value as the output;
[0013] S1.2, construct the hidden layer. Each layer uses 64 filters of size 3×3×64, and add batch normalization processing between the convolutional layer and the rectified linear unit ReLU(y). The above batch normalization processing transforms the feature values of the pictures extracted in S1.1 into numbers with a mean of 0 and a variance of 1, and enters the next layer of the network to alleviate the problem of internal covariate shift; This network uses a residual network to solve the problem of network depth: Add the input function a before the rectified linear unit ReLU(y) [l] , and the formula of the residual network is: a [L] = a [l] + F(a [l] , W), where a [L] is the input value of the L-th layer network, W is the matching convolutional operation for dimensionality increase or decrease, and F(a [l] , W) is the residual part; The above residual network is used in combination with the above batch normalization to optimize the network efficiency. The network solves and evaluates the model by minimizing the loss function, and the training image data where, x i is the i-th noise-free image, y i is the i-th original training image, N is the total number of training images, and the loss function l(θ) of network training is defined as: where, is the square of the F-norm of the difference between the residual mapping noise and the true noise of the training network, R(y i ; θ) is the residual mapping noise, ||()|| F represents the F-norm of (), θ is the network parameter, and the mean square error MSE is used as the loss function to constrain between the true residual picture and the network output. The Adam algorithm is used as the gradient optimization, and the formula of the Adam algorithm is:
[0014] m t := beta1 * m t-1 + (1 - beta1) * g
[0015] v t := beta2 * v t-1+(1-beta2)*g*g
[0016]
[0017] where m t is the first-order exponential smoothing value of the historical gradient, v t is the first-order exponential smoothing value of the square of the historical gradient, is the learning rate, set to 0.001, beta1 is the exponential decay rate of the first-moment estimate, beta2 is the exponential decay rate of the second-moment estimate, ε is a constant added to maintain numerical stability, g is the gradient at step t, variable is the formula for updating the Adam algorithm parameters, and the := operator means assigning the right-hand side of the equation to the left-hand side.
[0018] S1.3, construct the output layer: Finally, use a filter of size 3×3×64 to reconstruct the output residual image.
[0019] Furthermore, in S2, preprocess the terahertz image. The steps are: convert the color RGB image into a grayscale image. The conversion formula: P Grey = 0.299*P R + 0.587*P G + 0.114*P B where P Grey is the grayscale pixel of the converted image, P R is the red pixel of the image before conversion, P G is the green pixel of the image before conversion, P B is the blue pixel of the image before conversion.
[0020] Furthermore, in S3, output the predicted image and determine the optimization quality. The steps are: call the neural network to output the optimized image, which is obtained by subtracting the original image from the network output residual image. The formula is:
[0021] x′ = -v′, where x′ is the image optimized by the network, v′ is the network output residual network image, and z is the original image. Compare the PSNR and SSIM between the optimized image and the original image to judge the optimization result of the terahertz image network. The judgment basis includes two parameters. The first parameter is the peak signal-to-noise ratio PSNR: where b is the color depth, taking 8, and MSE is the mean square error; the second parameter is the structural similarity SSIM: where is the image optimized by the neural network, μ z is the average value of z, μ d is the average value of d, is the variance of z, is the variance of d, σ zd is the covariance of z and d,
[0022] C1 = (0.001) 2 , C2 = (0.003) 2 , where L is the pixel dynamic range.
[0023] Beneficial effects
[0024] The present invention has a very good optimization effect on terahertz images with serious noise and blurred images, and has high operating efficiency. Under the condition of a trained network model, it only takes 1.79 seconds to optimize a single terahertz image, and multiple terahertz images can be optimized simultaneously. The peak signal-to-noise ratio of the optimized image reaches 29.55, and the similarity structure reaches 0.70. Description of the drawings
[0025] Figure 1 is the flowchart of the solution of the present invention;
[0026] Figure 2 is a comparison diagram before and after optimization using the present invention, where Figure 2(a) is an unoptimized terahertz image, and Figure 2(b) is a terahertz image optimized by the present invention. Detailed implementation manners
[0027] The present invention will be further clarified below in conjunction with the drawings and specific embodiments. However, it should be understood that these embodiments are only used to illustrate the present invention in more detail and specifically, and should not be construed as limiting the present invention in any form. After reading the present invention, various equivalent modifications made by those skilled in the art to the present invention fall within the scope defined by the appended claims of this application.
[0028] A terahertz image processing method, the processing flow of which is as Figure 1 shown. The specific steps are as follows:
[0029] A terahertz image optimization method based on deep learning, the processing flow includes the following steps:
[0030] S1. Construct a neural network with 20 layers, including 3 different types of network layers.
[0031] The step of constructing a neural network with 20 layers, including 3 different types of network layers in S1, includes:
[0032] S1.1. Construct an input layer and a convolutional layer: Input the training image data into the convolutional layer: Use 64 convolutional kernels of 3×3×c to extract the original terahertz image, generate 64 feature maps, and the corresponding output feature value is y. c represents the number of image channels. Use the rectified linear unit ReLU(y) for non-linear processing. The ReLU(y) = max(y, 0), which means comparing the output feature value y with 0 and taking the maximum value as the output; in the formula, c takes 1, representing the selection of grayscale images to reduce the amount of training data;
[0033] S1.2. Construct a hidden layer, with 64 filters of size 3×3×64 in each layer, and add batch normalization between the convolutional layer and the rectified linear unit ReLU(y). The above batch normalization converts the feature values of the pictures extracted in S1.1 into numbers with a mean of 0 and a variance of 1, and enters the next layer of the network to alleviate the problem of internal covariate shift. This network uses a residual network to solve the problem of network depth: add an input function a before the rectified linear unit ReLU(y). [l] , and the formula of the residual network is: a [L] = a [l] + F(a [l] , W), where a [L] is the input value of the L-th layer network, W is the matching convolutional operation for dimensionality increase or decrease, and F(a [l] , W) is the residual part. The above residual network is used in combination with the above batch normalization to optimize the network efficiency. The network solves and evaluates the model by minimizing the loss function, and trains the image data where x i is the i-th noise-free image, y i is the i-th original training image, N is the total number of training images, and the loss function l(θ) for network training is defined as: where is the square of the F-norm of the difference between the residual mapping noise and the true noise in the training network, R(y i ; θ) is the residual mapping noise, ||()|| F represents the F-norm of (), θ is the network parameter, and the mean squared error MSE is used as the loss function for constraint between the true residual picture and the network output. The Adam algorithm is used as the gradient optimization, and the formula of the Adam algorithm is:
[0034] m t := beta1 * m t - 1 + (1 - beta1) * g
[0035] v t := beta2 * v t-1 + (1 - beta2) * g * g
[0036]
[0037] where m t is the first-order exponential smoothing value of the historical gradient, initially 0, v t is the first-order exponential smoothing value of the square of the historical gradient, initially 0. is the learning rate, set to 0.001, beta1 is the exponential decay rate of the first moment estimate, usually taken as 0.9, beta2 is the exponential decay rate of the second moment estimate, usually taken as 0.999, ε is a constant added to maintain numerical stability, g is the gradient at step t, variable is the Adam algorithm parameter update formula, and the := operator means assigning the right - hand side of the equation to the left - hand side.
[0038] S1.3, construct the output layer: Finally, use a filter of size 3×3×64 to reconstruct the output residual image. After construction, train the network. According to the decreasing speed of the loss function loss value during training, adjust network parameters such as the learning rate. The SSID dataset can be selected. Train the noisy pictures and noise - free pictures in one - to - one correspondence. It is recommended that the number of training iterations epoch be more than 300 times. If the SIDD dataset is not used, noise can also be added to clean images and train them in one - to - one correspondence.
[0039] S2, preprocess the terahertz image.
[0040] In S2, when preprocessing the terahertz image, the steps are as follows: Convert the color RGB image into a grayscale image. The conversion formula is: P Grey = 0.299 * P R + 0.587 * P G + 0.114 * P B In the formula, P Grey is the grayscale pixel of the converted image, P R is the red pixel of the original image, P G is the green pixel of the original image, P B is the blue pixel of the original image.
[0041] S3, output the predicted image and determine the optimization quality. In S3, output the predicted image and determine the optimization quality. The steps are as follows: Call the neural network to output the optimized image, which is obtained by subtracting the original image from the network - output residual image. The formula is: x′ = -v′, where x′ is the network - optimized image, v′ is the network - output residual network image, and z is the original image. Compare the optimized image with the original image in terms of PSNR and SSIM to judge the network optimization result of the terahertz image. The judgment basis includes two parameters. The first parameter is the peak signal - to - noise ratio PSNR: In the formula, b is the color depth, taken as 8, and MSE is the mean square error; The second parameter is the structural similarity SSiM: In the formula, is the image optimized by the neural network, μ z is the average value of z, μ d is the average value of d, is the variance of y, is the variance of d, σ zdis the covariance of z and d, C1 = (0.001) 2 , C2 = (0.003) 2 , where L is the pixel dynamic range. The optimization results show that after optimization by this method, compared with the unoptimized Figure 2(a), the PSNR of Figure 2(b) can reach 29.55 and the SSIM can reach 0.7.
Claims
1. A terahertz image optimization method based on deep learning, characterized in that, Including the following steps: S1. Construct a 20-layer neural network including three different types of network layers; S2. Preprocess the terahertz image; S3. Output the predicted image and determine the optimization quality; In step S1, the following steps are included: S1.
1. Construct the input layer and the convolutional layer: Input the training image data into the convolutional layer, use 64 convolutional kernels of 3×3×c to extract the original terahertz image, generate 64 feature maps, and correspondingly output the feature value y. c represents the number of image channels, and use the rectified linear unit ReLU(y) for non-linear processing; The ReLU(y) = max(y, 0), which means comparing the output feature value y with 0 and taking the maximum value as the output; S1.
2. Build a hidden layer, with 64 filters of size 3×3×64 for each layer. Add batch normalization between the convolutional layer and the rectified linear unit ReLU(y). The batch normalization converts the feature values of the image extracted in step S1.1 into numbers with a mean of 0 and a variance of 1, and then enters the next layer of the network to alleviate the problem of internal covariate shift. This network uses a residual network to solve the problem of network depth: add an input function a before the rectified linear unit ReLU(y). [l] , and the formula of the residual network is: a [L] = a [l] + F(a [l] , W), where a [L] is the input value of the L-th layer network, W is the matching convolutional operation for dimension elevation or reduction, and F(a [l] , W) is the residual part. The residual network is used in combination with the batch normalization to optimize the network efficiency. The network solves and evaluates the model by minimizing the loss function, training the image data where x i is the i-th noise-free image, y i is the i-th original training image, N is the total number of training images, and the loss function l(θ) of the training network is defined as: where is the square of the F-norm of the difference between the residual mapping noise and the true noise of the training network, R(y i ; θ) is the residual mapping noise, ‖()‖ F represents the F-norm of (), θ is the network parameter, the mean squared error MSE is used as the loss function between the true residual image and the network output, and the Adam algorithm is used as the gradient optimization. The Adam algorithm formula is: m t := beta1 * m t-1 + (1 - beta1) * g v t := beta2 * v t-1 + (1 - beta2) * g * g where m t is the first-order exponential smoothing value of the historical gradient, v t is the first-order exponential smoothing value of the square of the historical gradient, l rt is the learning rate, set to 0.001, beta1 is the exponential decay rate of the first-order moment estimate, beta2 is the exponential decay rate of the second-order moment estimate, ε is a constant added to maintain numerical stability, g is the gradient at step t, wariable is the formula for updating the Adam algorithm parameters, and := is the assignment operator representing assigning the right-hand side of the equation to the left-hand side; S1.
3. Construct the output layer: Finally, use a filter with a size of 3×3×64 to reconstruct the output residual image.
2. The terahertz image optimization method based on deep learning according to claim 1, characterized in that: In the step S2, the following steps are included: converting the color RGB image into a grayscale image, and the conversion formula: P Grey = 0.299 * P R + 0.587 * P G + 0.114 * P B , where P Grey is the grayscale pixel of the converted image, P R is the red pixel of the image before conversion, P G is the green pixel of the image before conversion, and P B is the blue pixel of the image before conversion.
3. A terahertz image optimization method based on deep learning according to claim 2, characterized in that: In the step S3, the following steps are included: calling the neural network to output the optimized image, which is obtained by subtracting the original image from the network output residual image. The formula is: x′ = z - v′, where x′ is the network-optimized image, v′ is the network output residual network image, and z is the original image; judging the optimization result of the terahertz image network according to the peak signal-to-noise ratio PSNR and the similarity structure SSIM between the optimized image and the original image. The judgment basis includes two parameters: the first parameter is the peak signal-to-noise ratio PSNR: where b is the color depth, taking 8, and MSE is the mean square error; the second parameter is the similarity structure SSIM: where d is the image optimized by the neural network, μ z is the average value of z, μ d is the average value of d, is the variance of z, is the variance of d, σ zd is the covariance of z and d, C1 = (0.001L) 2 and C2 = (0.003L) 2 where L is the pixel dynamic range.