Terahertz image denoising method based on multi-scale expansion convolution residual network

Abstract

The invention discloses a terahertz image denoising method based on a multi-scale expansion convolution residual network, and relates to the technical field of image processing. The method comprises the steps that training test verification data are acquired, and the training test verification data comprise a training set, a verification set and a test set; constructing a terahertz image noise model based on a deep denoising cyclic convolutional network DN-CNN in combination with U-Net; training the terahertz image noise model based on a preset loss function by using the training set and the verification set to determine the terahertz image noise model; and testing the determined terahertz image noise model by using the test set so as to evaluate the denoising performance of the terahertz image noise model. According to the invention, the denoising effect of the terahertz picture is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a terahertz image denoising method based on a multi-scale dilated convolution residual network. BACKGROUND

[0002] Terahertz waves are electromagnetic waves with a frequency range between microwaves and infrared, and the oscillation frequency is usually defined as 0.1-10 THz (terahertz). Due to the good penetration of terahertz waves for non-polar materials, the fingerprint spectrum characteristics of materials in the terahertz wave band, and the low photon energy characteristics, terahertz waves are suitable for application in material identification, security and defense, non-contact non-destructive testing, etc. Terahertz imaging is just the complementary information formed by the unique properties of terahertz waves and the imaging of other bands in the electromagnetic spectrum (such as X-ray, ultraviolet, visible light, infrared, microwave). Compared with higher frequency band imaging such as infrared, terahertz waves can penetrate non-polar materials such as plastic, rubber, and paperboard for tomographic imaging to detect internal substances. Compared with low frequency band imaging such as microwave and millimeter wave, terahertz waves have shorter wavelengths, so the spatial resolution is higher.

[0003] By wideband terahertz pulse imaging, terahertz frequency domain spectral data can be obtained, which has three-dimensional spectral information in addition to two-dimensional spatial information. Using the spectral data of terahertz waves can identify and classify materials, which has great application potential. Terahertz spectral imaging can be divided into time domain information imaging and frequency domain spectral imaging. Generally, the limit resolution of terahertz spectral imaging is reflected in the image formed by the spectrum of the higher frequency region of the terahertz spectrum. Due to the influence of hardware such as femtosecond lasers, choppers, and lock-in amplifiers in the system for exciting terahertz waves, there is a problem of low energy and high noise in the higher frequency band of the terahertz frequency domain, which directly leads to blurred images and high noise interference in the spectral imaging of the higher frequency band of terahertz, affecting the resolution and contrast of the image. Optimizing the system hardware to reduce noise in terahertz imaging is costly and has limited effect. Therefore, it is particularly important to use image denoising algorithms to solve the noise pollution problem of terahertz spectral images and improve the edge resolution and overall quality of the images. SUMMARY

[0004] The present application provides a terahertz image denoising method based on a multi-scale dilated convolution residual network, a system, a device, a storage medium, and a program product to solve the problems of low resolution and low contrast in current THz image processing technology, and improve the denoising effect of terahertz images.

[0005] The present application is implemented by the following technical solutions.

[0006] In a first aspect, a terahertz image denoising method based on a multi-scale dilated convolution residual network is provided, and the method comprises:

[0007] obtaining training test validation data, wherein the test validation data comprises: a training set, a validation set, a test set;

[0008] constructing a terahertz image noise model based on a deep denoising cyclic convolutional network (DN-CNN) combined with a U-Net;

[0009] training the terahertz image noise model based on a preset loss function using the training set and the validation set to determine the terahertz image noise model;

[0010] testing the determined terahertz image noise model using the test set to evaluate the denoising performance of the terahertz image noise model.

[0011] In some embodiments, the terahertz image noise model is constructed based on a DN-CNN combined with a U-Net, comprising:

[0012] constructing a terahertz image noise framework, wherein the framework comprises: at least three groups of encoding layers and pooling layers, at least three groups of transpose convolutional layers and decoding layers, a bottleneck layer, and a last convolutional layer, the at least three groups of encoding layers comprise a first group of encoding layers and pooling layers, a second group of encoding layers and pooling layers, and a third group of encoding layers and pooling layers, the encoding layers and the pooling layers in each group are connected, the pooling layer of the first group is connected to the encoding layer of the second group, the pooling layer of the second group is connected to the encoding layer of the third group, and the pooling layer of the third group is connected to the bottleneck layer; the at least three groups of transpose convolutional layers and decoding layers comprise a first group of transpose convolutional layers and decoding layers, a second group of transpose convolutional layers and decoding layers, and a third group of transpose convolutional layers and decoding layers, the decoding layers and the transpose convolutional layers in each group are connected, the transpose convolutional layer of the first group is connected to the bottleneck layer, the decoding layer of the first group is connected to the transpose convolutional layer of the second group, the decoding layer of the second group is connected to the transpose convolutional layer of the third group, and the decoding layer of the third group is connected to the last convolutional layer, and the encoding layers in the at least three groups of encoding layers and pooling layers are connected to the decoding layers in the at least three groups of transpose convolutional layers and decoding layers one by one through a skip connection;

[0013] wherein each encoding layer in the at least three groups of encoding layers and pooling layers and each decoding layer in the at least three groups of transpose convolutional layers and decoding layers are subjected to first convolutional processing using a 3*3 convolutional kernel with different expansion rates, and the bottleneck layer is subjected to residual convolutional processing using a 3*3 convolutional kernel with different expansion rates.

[0014] In some embodiments, after each encoding layer in the at least three groups of encoding layers and pooling layers and each decoding layer in the at least three groups of transpose convolutional layers and decoding layers are subjected to convolutional processing using a 3*3 convolutional kernel with different expansion rates,

[0015] The first splicing is performed on the results of the first convolutional processing of each expansion rate of each encoding layer or each decoding layer.

[0016] The first expansion rate is used for batch normalization processing and GELU activation function processing on the results of the first splicing of each encoding layer or each decoding layer on different expansion rates, and the processed results are input into the pooling layer after each encoding layer or the convolutional layer after each decoding layer.

[0017] In some embodiments, the bottleneck layer uses a 3*3 convolution kernel with different expansion rates for residual convolution processing, including:

[0018] The output results of the third group of pooling layers are subjected to second convolution processing, batch normalization processing and GELU activation function processing using a 3*3 convolution kernel.

[0019] The results of the second convolution processing, batch normalization processing and GELU activation function processing are subjected to first residual processing, wherein the first residual processing includes:

[0020] The results of the second convolution processing, batch normalization processing and GELU activation function processing are divided into multiple parts, and each part is input into a first dilated convolution layer with different expansion rates.

[0021] Each first dilated convolution layer is subjected to convolution processing, batch normalization processing and GELU activation function processing using a 3*3 convolution kernel, and the outputs of each first dilated convolution layer are spliced.

[0022] The spliced results are subjected to convolution processing, batch normalization processing and GELU activation function processing using a 1*1 convolution kernel with a predetermined expansion rate.

[0023] The results of the convolution processing, batch normalization processing and GELU activation function processing using a 1*1 convolution kernel are subjected to convolution processing, batch normalization processing using a 3*3 convolution kernel with the predetermined expansion rate, and the processed results are connected to the results of the second convolution processing, batch normalization processing and GELU activation function processing.

[0024] The results of the first residual processing are subjected to second residual processing, wherein the second residual processing includes:

[0025] The results of the first summation are divided into multiple parts, and each part is input into a second dilated convolution layer with different expansion rates.

[0026] Each second dilated convolution layer is subjected to convolution processing, batch normalization processing and GELU activation function processing using a 3*3 convolution kernel, and the outputs of each second dilated convolution layer are spliced.

[0027] The results are processed by convolution with a 1*1 convolution kernel, batch normalization and GELU activation function at a predetermined inflation rate;

[0028] The results processed by convolution with a 1*1 convolution kernel, batch normalization and GELU activation function are processed by convolution with a 3*3 convolution kernel, batch normalization at the predetermined inflation rate, and the processed results are secondly connected with the results of the first residual processing;

[0029] The results after the second residual processing are processed by convolution with a 3*3 convolution kernel, batch normalization and GELU activation function.

[0030] In some embodiments, the loss function is:

[0031] CombinedLoss=0.6×MSE+0.3×L1+0.1×GradientLoss;

[0032] ;

[0033] ;

[0034] ;

[0035] ;

[0036] ;

[0037] wherein CombinedLoss represents the combined loss; MSE represents the mean square error; L1 represents the linear error; GradientLoss represents the gradient loss; predNoise represents the predicted noise, trueNoise represents the true noise, and predNoise and trueNoise are respectively and the predicted noise and the true noise of the i-th pixel value of predNoise represents the number of pixels; predNoise and trueNoise represent the gradient of the predicted noise and the true noise in the x direction respectively; predNoise and trueNoise represent the gradient of the predicted noise and the true noise in the y direction respectively.

[0038] In some embodiments, the training test verification data is obtained, comprising:

[0039] A plurality of original gray pictures are obtained, and a plurality of geometric transformations are performed on the plurality of original gray pictures;

[0040] randomly crop the plurality of original grayscale pictures after the plurality of geometric transformations into smaller M*M image blocks to form a clean image block set, M being a positive integer;

[0041] extract a terahertz spectrum residual image from the real terahertz spectrum image through high-pass filtering;

[0042] superimpose the terahertz spectrum residual image on the clean image block set and add Gaussian white noise of different intensities.

[0043] In a second aspect, a terahertz image denoising system based on a multi-scale dilated convolution residual network is provided, and the system comprises:

[0044] a data acquisition module configured to acquire training test validation data, wherein the training test validation data comprises a training set, a validation set and a test set;

[0045] a terahertz image noise model construction module configured to construct a terahertz image noise model based on a deep denoising recurrent convolutional network (DN-CNN) combined with a U-Net;

[0046] a terahertz image noise model determination module configured to train the terahertz image noise model based on a preset loss function using the training set and the validation set to determine the terahertz image noise model;

[0047] a terahertz image noise model evaluation module configured to test the determined terahertz image noise model using the test set to evaluate the denoising performance of the terahertz image noise model.

[0048] In a third aspect, a terahertz image denoising device based on a multi-scale dilated convolution residual network is provided, and the device comprises:

[0049] at least one processor;

[0050] at least one memory coupled to the at least one processor and storing instructions for execution by the at least one processor, the instructions, when executed by the at least one processor, implementing the method described above.

[0051] In a fourth aspect, a computer-readable storage medium is provided, and the computer-readable storage medium stores instructions, which, when executed by a computer, cause the computer to perform the method described above.

[0052] In a fifth aspect, a computer program product is provided, and the computer program product comprises instructions, which, when executed by a computer, cause the computer to perform the method described above.

[0053] Compared with the prior art, the present application has the following advantages and beneficial effects: taking the DN-CNN structure as a basic model architecture, combining the advantages of the U-net structure; introducing multi-dilation feature extraction, adding a skip connection and a residual connection operation, introducing GELU as an activation function, accelerating the convergence of the model and reducing the risk of overfitting; referring to the TCN network design in the denoising aspect, increasing the signal-to-noise ratio; introducing a CombinedLoss loss function, which has high efficiency, stability and flexibility in the denoising task. BRIEF DESCRIPTION OF DRAWINGS

[0054] In order to more clearly illustrate the technical solutions of the example embodiments of the present application, the drawings needed in the examples will be briefly introduced as follows, and it should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as a limitation on the scope, and for those skilled in the art, other related drawings can also be obtained without creative labor. In the drawings:

[0055] Figure 1 A flowchart of a terahertz image denoising method based on a multi-scale dilated convolution residual network according to an embodiment of the present application.

[0056] Figure 2 A schematic diagram of a terahertz image noise model architecture according to an embodiment of the present application.

[0057] Figure 3 A schematic diagram of the structure of a decoder or encoder according to an embodiment of the present application.

[0058] Figure 4 A schematic diagram of the structure of a bottleneck layer according to an embodiment of the present application.

[0059] Figure 5 A schematic diagram of the structure of a residual block 1 according to an embodiment of the present application.

[0060] Figure 6 A schematic diagram of the structure of a residual block 2 according to an embodiment of the present application.

[0061] Figure 7 A block diagram of a terahertz image denoising system based on a multi-scale dilated convolution residual network according to an embodiment of the present application.

[0062] Figure 8 A schematic diagram of a terahertz image denoising device based on a multi-scale dilated convolution residual network according to an embodiment of the present application. DETAILED DESCRIPTION

[0063] In order to make the objects, technical solutions and advantages of the present application clearer, further detailed description will be given below in combination with embodiments and drawings. The illustrative embodiments and their descriptions are only used to explain the present application and not to limit the present application.

[0064] The present application applies deep learning combined with terahertz technology to the field of image denoising, thereby performing automatic feature extraction and multi-scale local feature extraction on THz data, learning the distribution and features of noise in the terahertz noise picture through a convolutional neural network (CNN), and extracting the local detail features of the data through different inflation rate convolutions, thereby extracting terahertz noise features using the automatic learning ability of the convolutional neural network and improving the stability of feature extraction. The convolutional neural network structure designed in the present application performs high-pass filtering on the terahertz picture, thereby extracting a terahertz spectrum residual image, and fusing the terahertz spectrum residual image with a natural grayscale picture and Gaussian noise to obtain a large number of terahertz noise-containing pictures, and introducing convolution kernels with different dilation rates for feature extraction. And adding operations such as residual connection to avoid the loss of global features of the data during the learning process of the network model, and enhance the sensitivity of the model to small spectral differences.

[0065] In one aspect, the present application provides a terahertz image denoising method based on a multi-scale dilated convolution residual network. Figure 1 A flowchart of a terahertz image denoising method based on a multi-scale dilated convolution residual network according to an embodiment of the present application. Referring to Figure 1 The method comprises S10 to S40.

[0066] In S10, training test validation data is obtained, wherein the training test validation data comprises a training set, a validation set and a test set.

[0067] In some embodiments, obtaining the training validation data comprises: obtaining a plurality of original grayscale pictures, and performing a plurality of geometric transformations on the plurality of original grayscale pictures; randomly cropping the plurality of original grayscale pictures after the plurality of geometric transformations into smaller M*M image blocks to form a clean image block set, M being a positive integer; extracting a terahertz spectrum residual image from a real terahertz spectrum image through high-pass filtering; superimposing the terahertz spectrum residual image on the clean image block set and adding Gaussian white noise with different intensities. For example, 70% of the data can be selected as the training set, 20% of the data can be selected as the validation set, and 10% of the data can be selected as the test set.

[0068] Exemplarily, using four hundred gray scale pictures, six geometric transformations (original picture, horizontal flip, upside-down flip, 90° clockwise rotation, 180°, 270°) are performed on the 400 original images, all images are randomly cropped into smaller 128*128 image blocks to form a final clean image block set, from the real terahertz spectrum images collected by the experimental system, the "terahertz spectrum residual image" (i.e. the real noise component) is extracted by high-pass filtering, and then it is superimposed on the clean image block, on the basis of the above, different intensity of Gaussian white noise is added, and finally the noise standard deviation [0, 55] of the added noise data.

[0069] In S20, a terahertz image noise model is constructed based on a deep denoising convolutional neural network (DN-CNN) combined with a U-Net.

[0070] In some embodiments, the terahertz image noise model is constructed based on the DN-CNN combined with the U-Net, including: constructing a terahertz image noise framework. Figure 2 The schematic diagram of the terahertz image noise model architecture according to the embodiment of the present application is shown in FIG. 1. Referring to FIG. 1, Figure 2 The framework includes at least three groups of encoding layers and pooling layers, at least three groups of transposed convolution layers and decoding layers, a bottleneck layer, and a last convolution layer. The at least three groups of encoding layers include a first group of encoding layers and pooling layers, a second group of encoding layers and pooling layers, and a third group of encoding layers and pooling layers. The encoding layers and the pooling layers in each group are connected. The pooling layer of the first group is connected to the encoding layer of the second group. The pooling layer of the second group is connected to the encoding layer of the third group. The pooling layer of the third group is connected to the bottleneck layer. The at least three groups of transposed convolution layers and decoding layers include a first group of transposed convolution layers and decoding layers, a second group of transposed convolution layers and decoding layers, and a third group of transposed convolution layers and decoding layers. The decoding layers and the transposed convolution layers in each group are connected. The transposed convolution layer of the first group is connected to the bottleneck layer. The decoding layer of the first group is connected to the transposed convolution layer of the second group. The decoding layer of the second group is connected to the transposed convolution layer of the third group. The decoding layer of the third group is connected to the last convolution layer. The encoding layers in the at least three groups of encoding layers and pooling layers are connected to the decoding layers in the at least three groups of transposed convolution layers and decoding layers through a one-to-one correspondence connection by a jump connection.

[0071] Each encoding layer in the at least three groups of encoding layers and pooling layers and each decoding layer in the at least three groups of transposed convolution layers and decoding layers are respectively subjected to first convolution processing by a 3*3 convolution kernel with different expansion rates. The bottleneck layer is subjected to residual convolution processing by a 3*3 convolution kernel with different expansion rates. The last convolution layer is a terminal feature extraction layer of the CNN, and its core function is to map high-level abstract features to a classification space.

[0072] In the present application, three sets of encoding layers and pooling layers, three sets of transpose convolution layers and decoding layers are taken as examples for illustration, but the present application is not limited thereto. The present application is based on the encoder and decoder architecture of U-Net, combined with the convolution processing of DN-CNN, to form the above-mentioned framework structure.

[0073] In some embodiments, after each encoding layer in the at least three sets of encoding layers and pooling layers and each decoding layer in the at least three sets of transpose convolution layers and decoding layers are respectively subjected to convolution processing by 3*3 convolution kernels with different dilation rates, the results of the first convolution processing of each encoding layer or each decoding layer are subjected to batch normalization processing and GELU activation function processing, and then first splicing is respectively performed; the results of the first splicing of each encoding layer or each decoding layer are subjected to batch normalization processing and GELU activation function processing with a first dilation rate, and are input into the pooling layer after each encoding layer or the convolution layer after each decoding layer.

[0074] Figure 3 The structure diagram of the decoder or the encoder according to the embodiments of the present application is shown in FIG. 2. For example, referring to FIG. 2, the encoder and the decoder are taken as examples for illustration. Figure 3 For each encoding layer (or encoder) and decoding layer (or decoder), 3*3 convolution kernels (Conv3*3) and dilation rates d = 1, 2 or 3 are used for convolution processing, and then batch normalization processing and GELU activation function are performed after the convolution kernel Conv3*3 and the dilation rate d = 1, and finally the results are output to the subsequent pooling layer or convolution layer.

[0075] In some embodiments, the bottleneck layer uses 3*3 convolution kernels with different dilation rates for residual convolution processing, including: using 3*3 convolution kernels for second convolution processing, batch normalization processing and GELU activation function processing on the output results of the third set of pooling layers; performing first residual processing on the results after the second convolution processing, batch normalization processing and GELU activation function processing; performing second residual processing on the results after the first residual processing; and performing convolution processing, batch normalization and GELU activation function processing on the results after the second residual processing using 3*3 convolution kernels.

[0076] Figure 4 The structure diagram of the bottleneck layer according to the embodiments of the present application is shown in FIG. 3. For example, referring to FIG. 3, the bottleneck layer is taken as an example for illustration. Figure 4 The first residual processing is performed by the residual module 1, including: dividing the results of the second convolution processing, batch normalization processing and GELU activation function processing into multiple parts, and each part enters a first dilated convolution layer with different dilation rates; for each first dilated convolution layer, convolution processing, batch normalization processing and GELU activation function processing are respectively performed using 3*3 convolution kernels, and the outputs of each first dilated convolution layer are spliced (for example, referring to FIG. 4). Figure 5The feature map data can be divided into four parts, each part entering an expansion convolution layer, each expansion convolution layer having a different expansion rate; the spliced result is subjected to convolution processing, batch normalization processing and GELU activation function processing using a 1*1 convolution kernel at a predetermined expansion rate; the result of the convolution processing, batch normalization processing and GELU activation function processing using the 1*1 convolution kernel is subjected to convolution processing, batch normalization processing using a 3*3 convolution kernel at a predetermined expansion rate, and the processing result is subjected to first residual connection with the result of the second convolution processing, batch normalization processing and GELU activation function processing.

[0077] The second residual processing includes: dividing the result of the first summation into multiple parts, each part of the multiple parts entering a second expansion convolution layer with a different expansion rate; the outputs of the second expansion convolution layers are spliced (for details, refer to Figure 6 The feature map data can be divided into four parts, each part entering an expansion convolution layer, each expansion convolution layer having a different expansion rate; the spliced result is subjected to convolution processing, batch normalization processing and GELU activation function processing using a 1*1 convolution kernel at a predetermined expansion rate; the result of the convolution processing, batch normalization processing and GELU activation function processing using the 1*1 convolution kernel is subjected to convolution processing, batch normalization processing using a 3*3 convolution kernel at a predetermined expansion rate, and the processing result is subjected to second residual connection with the result of the first residual processing.

[0078] In S30, the terahertz image denoising model is trained based on a preset loss function using the training set and the validation set to determine the terahertz image noise model. Exemplarily, the loss function is:

[0079] CombinedLoss=0.6×MSE+0.3×L1+0.1×GradientLoss;

[0080] ;

[0081] ;

[0082] ;

[0083] ;

[0084] ;

[0085] wherein, CombinedLoss represents a combined loss; MSE represents a mean square error; denotes a linear error; GradientLoss denotes a gradient loss; denotes a predicted noise, denotes a real noise, and respectively denote a predicted noise and a real noise of the i-th pixel value of the image; and respectively denote a predicted noise and a real noise of the i-th pixel value of the image; denotes a number of pixels; respectively denote a gradient of the predicted noise and the real noise in the x direction; respectively denote a gradient of the predicted noise and the real noise in the y direction.

[0086] In S40, the determined terahertz image noise model is tested by using a test set to evaluate the denoising performance of the terahertz image noise model. The test set is used to verify the true effect of the model, and the model effect is determined by calculating the psnr (peak signal-to-noise ratio) and ssim (structural similarity) values. The larger the psnr and ssim values, the better the denoising effect.

[0087] For image data, THz-TDS (Terahertz Time-Domain Spectroscopy) transmission imaging system can be used to obtain THz-TDS signal. The obtained terahertz time-domain spectroscopy signal is arranged to obtain THz data of different samples, and finally the THz data is enhanced.

[0088] The THz-TDS transmission imaging system uses a time-domain spectrometer, the effective working frequency band is 0.1~5THZ, the collection window is 100ps, the intensity uses a non-scalar unit, the frequency resolution is 10GHZ, the sample is scanned by a reflection imaging mode, the control software is developed based on LabVIEW, and the experiment measures that the number of horizontal scanning data collection points of the sample is 3750. The PC platform loads a graphics card 1650Ti, and the CPU is AMD Ryzen 7 4800H with Radeon Graphics.

[0089] The real terahertz image collected from the experimental system is extracted as a "terahertz spectral residual image" (i.e. the real noise component) through high-pass filtering, and then it is superimposed on a clean image block (a gray image is subjected to 6 kinds of geometric transformation (original image, horizontal flip, upside-down flip, 90° clockwise rotation, 180°, 270°)). On the basis of the above, different intensity Gaussian white noise is added. Finally, as a data set, 70% of the data is selected as a training set, 20% of the data is selected as a validation set, and 20% of the data is selected as a test set, and the training loss, the test loss and the signal-to-noise ratio are evaluated respectively.

[0090] Each training is performed for 100 epochs, and the data batch number of each epoch is 64. The optimization algorithm uses the AdamW algorithm. The learning rate is 0.0005, the weight decay is 1e-4, and the learning rate is automatically reduced based on the validation loss plateau (factor 0.5, patience value 10). After each training, the training loss and test loss are recorded, and a curve is drawn after the training is completed to facilitate observation.

[0091] In the present application, a THz-TDS image denoising scheme based on a denoising convolutional neural network DN-CNN is proposed. The scheme: takes the DN-CNN structure as the basic model architecture, combines the advantages of the U-net structure; introduces multi-dilation feature extraction, adds skip connection and residual connection operation, introduces GELU as the activation function, accelerates the convergence of the model and reduces the risk of overfitting; in terms of denoising, refer to the TCN network design, increase the signal-to-noise ratio; introduce CombinedLoss loss function, which has high efficiency, stability and flexibility in the denoising task.

[0092] On the other hand, the present application provides a terahertz image denoising system based on a multi-scale dilated convolution residual network. Figure 7 A structure block diagram of a terahertz image denoising system based on a multi-scale dilated convolution residual network according to an embodiment of the present application. Referring to Figure 7 The system comprises: a data acquisition module, a terahertz image noise model construction module, a terahertz image noise model determination module, and an image data denoising module.

[0093] The data acquisition module is configured to acquire training, testing and validation data, wherein the training, testing and validation data comprises a training set, a validation set and a test set.

[0094] The terahertz image noise model construction module is configured to construct a terahertz image noise model based on a deep denoising recurrent convolutional network DN-CNN combined with a U-Net.

[0095] The terahertz image noise model determination module is configured to train the terahertz image noise model based on a preset loss function using the training set and the validation set to determine the terahertz image noise model.

[0096] The terahertz image noise model evaluation module is configured to test the determined terahertz image noise model using the test set to evaluate the denoising performance of the terahertz image noise model.

[0097] The other details of the terahertz image denoising system based on the multi-scale dilated convolution residual network are described in the foregoing description of the terahertz image denoising method based on the multi-scale dilated convolution residual network, and will not be described here.

[0098] In the case of implementing the functions of the above-mentioned integrated modules in the form of hardware, the embodiments of the present application provide a structure of the terahertz image denoising device based on the multi-scale dilated convolution residual network involved in the above-mentioned embodiments. Figure 8 FIG. 1 is a structure diagram of a terahertz image denoising device based on a multi-scale dilated convolution residual network according to an embodiment of the present application. Figure 8 The terahertz image denoising device based on the multi-scale dilated convolution residual network includes at least one processor and at least one memory. The at least one memory is coupled to the at least one processor and stores instructions for execution by the at least one processor, which, when executed by the at least one processor, implement the above-mentioned method.

[0099] The processor can be a central processing unit, a general purpose processor, a digital signal processor, an application specific integrated circuit, a field programmable gate array, or other programmable logic device, transistor logic, hardware components, or any combination thereof, which can implement or execute the various exemplary logical blocks, modules, and circuits described in combination with the embodiments of the present application. The processor can be a combination of computing functions, such as a combination of one or more microprocessors, a combination of a digital signal processor and a microprocessor, and the like.

[0100] The memory can be a read only memory (ROM) or other type of static storage device that can store static information and instructions, a random access memory (RAM) or other type of dynamic storage device that can store information and instructions, an electrically erasable programmable read only memory (EEPROM), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program codes in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto.

[0101] As an implementation manner, the memory can exist independently of the processor, and the memory can be connected to the processor through a bus for storing instructions or program codes. When the processor invokes and executes the instructions or program codes stored in the memory, the method provided by the embodiments of the present application can be implemented. In another implementation manner, the memory can also be integrated with the processor.

[0102] In another aspect, the present application also provides a computer-readable storage medium (for example, a non-transitory computer-readable storage medium) having computer program instructions stored therein, which, when executed on a computer, cause the computer to perform the method of any of the above embodiments.

[0103] Exemplarily, the above computer-readable storage medium can include, but is not limited to, a magnetic storage device (for example, a hard disk, a floppy disk or a magnetic tape, etc.), an optical disc (for example, a Compact Disk (CD), a Digital Versatile Disk (DVD), etc.), a smart card and a flash memory device (for example, an Erasable Programmable Read Only Memory (EPROM), a card, a stick or a key drive, etc.). The various computer-readable storage media described in the present application can represent one or more devices and / or other machine-readable storage media for storing information. The term "machine-readable storage medium" can include, but is not limited to, a wireless channel and various other media capable of storing, containing and / or carrying instructions and / or data.

[0104] The embodiments of the present application provide a computer program which, when executed on a computer, causes the computer to perform the method of any of the above embodiments.

[0105] The embodiments of the present application provide a computer program product containing instructions which, when executed on a computer, cause the computer to perform the method of any of the above embodiments.

[0106] The above detailed description further illustrates the purpose, technical solutions and beneficial effects of the present application. It should be understood that the above detailed description is only a specific embodiment of the present application and is not used to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the protection scope of the present application.

[0107] The above detailed description further illustrates the purpose, technical solutions and beneficial effects of the present application. It should be understood that the above detailed description is only a specific embodiment of the present application and is not used to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the protection scope of the present application.

Claims

1. A terahertz image denoising method based on a multi-scale dilated convolutional residual network, characterized in that, The method includes: Obtain training, testing, and validation data, wherein the training, testing, and validation data includes: a training set, a validation set, and a test set; A terahertz image noise model is constructed based on the deep denoising recurrent convolutional network DN-CNN combined with U-Net; The terahertz image noise model is trained using the training set and the validation set based on a preset loss function to determine the terahertz image noise model; The determined terahertz image noise model was tested using the test set to evaluate its denoising performance.

2. The method according to claim 1, characterized in that, A terahertz image noise model is constructed based on DN-CNN combined with U-Net, including: A terahertz image noise framework is constructed, comprising: at least three sets of coding layers and pooling layers, at least three sets of transposed convolutional layers and decoding layers, a bottleneck layer, and a final convolutional layer. The at least three sets of coding layers include a first set of coding layers and pooling layers, a second set of coding layers and pooling layers, and a third set of coding layers and pooling layers. Within each set, the coding layers and pooling layers are connected. The pooling layers of the first set are connected to the coding layers of the second set, the pooling layers of the second set are connected to the coding layers of the third set, and the pooling layers of the third set are connected to the bottleneck layer. The at least three sets of transposed convolutional layers and decoding layers include the first set of coding layers and pooling layers, the second set of coding layers and pooling layers, and the third set of coding layers and decoding layers. The system comprises a first group of transposed convolutional layers and a decoding layer, a second group of transposed convolutional layers and a decoding layer, and a third group of transposed convolutional layers and a decoding layer. Within each group, the decoding layer is connected to the transposed convolutional layer. The transposed convolutional layer of the first group is connected to the bottleneck layer. The decoding layer of the first group is connected to the transposed convolutional layer of the second group. The decoding layer of the second group is connected to the transposed convolutional layer of the third group. The decoding layer of the third group is connected to the final convolutional layer. The encoding layers in the at least three groups of encoding layers and pooling layers are respectively connected to the decoding layers in the at least three groups of transposed convolutional layers and decoding layers through skip connections. In this process, each of the at least three sets of coding layers and pooling layers, and each of the at least three sets of transposed convolutional layers and decoding layers, performs first convolution processing using 3*3 convolutional kernels with different dilation rates. The bottleneck layer performs residual convolution processing using 3*3 convolutional kernels with different dilation rates.

3. The method according to claim 2, characterized in that, The method further includes: performing convolution processing on each of the at least three sets of coding layers and pooling layers, and on each of the at least three sets of transposed convolutional layers and decoding layers, using 3*3 convolutional kernels with different dilation rates. After batch normalization and GELU activation function processing of the first convolution results of each coding layer or each decoding layer for each dilation rate, the results are concatenated separately. For the concatenation results of each coding layer or each decoding layer with different dilation rates, batch normalization and GELU activation function processing are performed using the first dilation rate, and then input into the pooling layer after each coding layer or the convolutional layer after each decoding layer.

4. The method according to claim 2, characterized in that, The bottleneck layer uses 3x3 convolution kernels with different dilation rates for residual convolution processing, including: The output of the pooling layer in the third group is processed by a second convolution with a 3*3 convolution kernel, followed by batch normalization and GELU activation. The results of the second convolution processing, batch normalization processing, and GELU activation function processing are subjected to a first residual processing, wherein the first residual processing includes: The results of the second convolution processing, batch normalization processing, and GELU activation function processing are divided into multiple parts, and each of the multiple parts is entered into the first dilated convolutional layer with a different dilation rate. For each of the first dilated convolutional layers, convolution processing, batch normalization processing, and GELU activation function processing are performed using 3*3 convolutional kernels respectively, and the outputs of each of the first dilated convolutional layers are concatenated. For the splicing results, convolution processing is performed using a predetermined dilation rate and a 1*1 convolution kernel, followed by batch normalization and GELU activation function processing. The result of convolution processing, batch normalization processing and GELU activation function processing using a 1*1 convolution kernel is processed with a 3*3 convolution kernel using the predetermined dilation rate, and batch normalization processing is performed. The processing result is then connected with the result of the second convolution processing, batch normalization processing and GELU activation function processing using a first residual connection. The result after the first residual processing is subjected to a second residual processing, wherein the second residual processing includes: The result of the first summation is divided into multiple parts, and each of the multiple parts is entered into a second dilated convolutional layer with a different dilation rate; Each second dilated convolutional layer is processed by convolution with a 3*3 convolutional kernel, batch normalization, and GELU activation function, and the outputs of each second dilated convolutional layer are concatenated. For the splicing results, convolution processing is performed using a predetermined dilation rate and a 1*1 convolution kernel, followed by batch normalization and GELU activation function processing. The result of convolution processing, batch normalization processing and GELU activation function processing using a 1*1 convolution kernel is processed with a 3*3 convolution kernel using the predetermined dilation rate, and batch normalization processing is performed. The processing result is then connected with the result of the first residual processing using a second residual concatenation. The result after processing the second residual is processed by convolution with a 3*3 kernel, batch normalization, and GELU activation function.

5. The method according to claim 1, characterized in that, The loss function is: CombinedLoss=0.6×MSE+0.3×L1+0.1×GradientLoss; ； ； ； ； ； Wherein, CombinedLoss represents the combined loss; This represents the mean square error; Represents linear error; GradientLoss represents gradient loss. Indicates prediction noise. Represents real noise. and They are respectively and The predicted noise and the actual noise of the i-th pixel value; Indicates the number of pixels; These represent the gradients of the predicted noise and the actual noise in the x-direction, respectively. These represent the gradients of the predicted noise and the actual noise in the y-direction, respectively.

6. The method according to claim 1, characterized in that, Obtain training, testing, and validation data, including: Multiple original grayscale images are acquired, and various geometric transformations are performed on the multiple original grayscale images; Multiple original grayscale images after undergoing the aforementioned geometric transformations are randomly cropped into smaller M*M image blocks to form a clean set of image blocks, where M is a positive integer; Terahertz spectral residual images are extracted from real terahertz spectral images using high-pass filtering; The terahertz spectral residual image is superimposed onto the clean image block set, and Gaussian white noise of different intensities is added.

7. A terahertz image denoising system based on a multi-scale dilated convolutional residual network, characterized in that, The system includes: The data acquisition module is used to: acquire training, testing, and validation data, wherein the training, testing, and validation data includes: a training set, a validation set, and a test set; The terahertz image noise model construction module is used to construct a terahertz image noise model based on the deep denoising recurrent convolutional network DN-CNN combined with U-Net. A terahertz image noise model determination module is used to: train the terahertz image noise model using the training set and the validation set based on a preset loss function, so as to determine the terahertz image noise model; The terahertz image noise model evaluation module is used to: test the determined terahertz image noise model using the test set to evaluate the denoising performance of the terahertz image noise model.

8. A terahertz image denoising device based on a multi-scale dilated convolutional residual network, characterized in that, The device includes: At least one processor; At least one memory coupled to the at least one processor and storing instructions for execution by the at least one processor, the instructions implementing the method of any one of claims 1 to 6 when executed by the at least one processor.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 6.

10. A computer program product, characterized in that, The computer program product includes instructions that, when executed by a computer, cause the computer to perform the method according to any one of claims 1 to 6.

Images