A terahertz single-pixel imaging method and system

Abstract

The present invention relates to a terahertz single-pixel imaging method and system, belonging to the field of optical imaging and deep learning. It includes: S1. Selecting a sampling scheme, constructing an image reconstruction network model, and training the image reconstruction network model to obtain a trained image reconstruction network model; S2. Generating a corresponding mask pattern according to the sampling scheme; S3. Modulating the terahertz laser through the mask, the modulated terahertz laser interacts with the target, the formed modulated signal is received by the detector, and undersampled one-dimensional data is obtained; S4. Importing the undersampled one-dimensional data into the trained image reconstruction network model to obtain a reconstructed image. The present invention combines an end-to-end convolutional deep neural network with terahertz single-pixel imaging, and at the same time combines the residual dense connection and deep compression in the network model into a triangular dense block, which can reduce the algorithm redundancy of the system and can adapt to most of the single-pixel imaging algorithms currently used.

Description

Technical Field

[0001] The present application relates to the fields of optical imaging and deep learning, and in particular to a terahertz single-pixel imaging method and system. Background Art

[0002] Terahertz waves have the advantages of low photon energy and penetration of non-polar materials, and have great potential in terahertz imaging, spectral analysis, and high-speed communications. Among them, terahertz imaging has begun to be applied in fields such as national defense security and biological imaging. However, due to the lack of suitable materials, the development of terahertz pixelated detector arrays has been slow. At present, most multi-pixel terahertz detector arrays are narrow-band or need to work in a low-temperature refrigeration environment; and traditional terahertz single-point scanning imaging cannot meet the requirements of fast imaging, which greatly restricts the actual promotion of terahertz imaging technology.

[0003] Another solution to terahertz imaging is to use a terahertz single-pixel imaging system, which not only saves hardware costs compared to a planar array terahertz imaging system, but also brings new possibilities for the miniaturization and commercialization of terahertz. At present, terahertz single-pixel imaging systems are implemented through compressed sensing, Hadamard basis and Fourier basis. Among them, She et al. used a 220um silicon-based graphene modulator and Fourier fringes to achieve sub-wavelength terahertz image reconstruction, and used the sparse characteristics of the image to collect low-frequency coefficients and inverse Fourier transform to achieve image reconstruction under a 10% modulation mask; Rayko et al. used a silicon total internal reflection prism and a Hadamard mask to achieve near-real-time terahertz single-pixel video, and used the sparse characteristics of the Hadamard domain to use undersampling technology to reduce sampling time.

[0004] However, due to the particularity of the terahertz wave wavelength and the limitations of single-pixel imaging, the terahertz single-pixel imaging technology has the following problems to be solved: (1) Slow imaging speed. Since single-pixel imaging uses a modulator to send a single detector to receive the light intensity of several mask patterns, and the imaging speed depends on the modulation time, the number of projections, the projection speed and the response time of the detector, etc.; when the image resolution increases, the increase in the number of samples will also slow down the imaging speed. (2) Poor imaging quality: Terahertz waves are extremely susceptible to coherent light interference during transmission, and the detection error caused by hardware thermal noise greatly affects the imaging quality. In addition, although the undersampling method can greatly shorten the imaging time, it will cause the loss of high-frequency information of the output image and deteriorate the image; from an observable perspective, when the sampling rate is lower than a certain ratio, the reconstructed pattern will be severely distorted or even twisted, so the sampling rate is not suitable to be lower than a certain lower limit. (3) The traditional deep learning algorithm currently used in single-pixel imaging solutions cannot be lower than a certain sampling rate, especially for complex patterns. At the same time, the splicing of algorithms will also bring about algorithm redundancy, which greatly increases the requirements for the imaging system. Summary of the invention

[0005] The embodiments of the present application provide a terahertz single-pixel imaging method and system to solve the problems of slow imaging speed, poor imaging quality, and redundancy of system algorithms in the related art.

[0006] One aspect of the present invention provides a terahertz single-pixel imaging method, comprising the following steps:

[0007] S1. construct an image reconstruction network model, select a sampling scheme, and train the image reconstruction network model based on the sampling scheme;

[0008] S2. Generate a corresponding mask pattern according to the sampling scheme;

[0009] S3. The terahertz laser is modulated by the mask pattern, and the modulated terahertz laser interacts with the target object, and the resulting modulated signal is received by the detector, and undersampled one-dimensional data is obtained;

[0010] S4. Importing the undersampled one-dimensional data into the trained image reconstruction network model to obtain a reconstructed image;

[0011] The image reconstruction network model includes a fully connected block and a triangle dense block, wherein the triangle dense block includes a convolution block, a downsampling layer, an upsampling layer and a residual dense connection.

[0012] Furthermore, the convolution block includes a convolution layer with an activation layer, a common convolution layer and a residual connection.

[0013] Furthermore, the sampling scheme is Hadamard coding, Fourier coding and wavelet transform coding.

[0014] Furthermore, step S3 includes: sending the mask pattern to a digital micromirror device, irradiating the digital micromirror device with a laser, and reflecting the mask pattern onto a modulator in a target pattern area, then emitting a terahertz laser and modulating the laser intensity using a terahertz modulator, and simultaneously receiving the terahertz laser through a terahertz single-point detector to obtain under-sampled and fully sampled one-dimensional data.

[0015] Furthermore, the step S4 also includes: performing inverse transformation image processing on the fully sampled one-dimensional data, comparing it with the reconstructed image, and performing image quality evaluation.

[0016] Furthermore, the step S1 further includes:

[0017] Before pre-training the image reconstruction network model, a corresponding data set is selected according to the scene of the reconstructed image, and a certain degree of random noise or additional image processing is added to the input image data.

[0018] Furthermore, the step S1 comprises:

[0019] A training set is generated according to the sampling scheme, wherein the training set includes an original image and a one-dimensional signal converted by inversely transforming the image according to the sampling scheme.

[0020] Another aspect of the present invention provides a terahertz single-pixel imaging system of an end-to-end network, comprising:

[0021] A training unit constructs an image reconstruction network model, selects a sampling scheme, and trains the image reconstruction network model based on the sampling scheme;

[0022] A generating unit, generating a corresponding mask pattern according to the sampling scheme;

[0023] An acquisition unit modulates the terahertz laser through the mask pattern, the modulated terahertz laser interacts with the target object, the formed modulated signal is received by the detector, and undersampled one-dimensional data is obtained;

[0024] A reconstruction unit, which imports the undersampled one-dimensional data into a trained image reconstruction network model to obtain a reconstructed image;

[0025] The image reconstruction network model includes a fully connected block and a triangular dense block, wherein the triangular dense block includes a convolution block, a downsampling layer, an upsampling layer and a residual dense connection.

[0026] Furthermore, the convolution block includes a convolution layer with an activation layer, a common convolution layer and a residual connection.

[0027] Furthermore, the sampling scheme is Hadamard coding, Fourier coding and wavelet transform coding.

[0028] The beneficial effects of the technical solution provided by this application include:

[0029] (1) Reduced algorithm redundancy. The current mainstream single-pixel imaging deep learning strategy is to reconstruct the image using a traditional imaging algorithm and then put it into a trained image enhancement network. The two algorithms are independent of each other. The end-to-end convolutional neural network can train single-pixel imaging reconstruction algorithms and image enhancement algorithms, which not only saves additional information storage space, but also directly completes the algorithm reconstruction from one-dimensional signals to two-dimensional images.

[0030] (2) Improving imaging speed. The present invention proposes a deep network model SIDL based on triangle dense blocks. Through residual dense connection and deep downsampling, shallow information and deep information are combined to restore undersampled patterns with low spatial resolution to high-quality images to a large extent, greatly reducing the requirements for complex pattern sampling rate, and achieving a measurement rate far lower than the Nyquist theorem sampling rate to improve imaging speed.

[0031] (3) The image reconstruction network model based on triangular dense blocks proposed in the present invention makes the overall network memory-based. Each layer of the network has all the information input before, and gradient explosion will not occur even if the network is deep. At the same time, deep compression can mine the depth information of undersampled images, so that effective information is obtained after each downsampling training, thereby improving the image quality and making it closer to the real image, and it can also adapt to the changes of different feature channels.

[0032] (4) The present invention belongs to an adaptive algorithm. The encoding method of the end-to-end convolutional neural network before input is not unique. An existing or manually set encoding method can be selected to collect a one-dimensional measurement signal. During the training process, the network parameters required for reconstruction will be adaptively trained according to the original image; compared with the traditional SIDL that can only train the existing single-pixel imaging method, this model reduces the requirements for the system.

[0033] The present invention provides a terahertz single-pixel imaging method and system. By combining an end-to-end convolutional deep neural network with terahertz single-pixel imaging and combining residual dense connection and deep compression in the network model into a triangular dense block, the algorithm redundancy of the system can be reduced and most of the currently used single-pixel imaging algorithms can be adapted. At the same time, gradient vanishing is not likely to occur in network training and the system is more suitable for single-pixel imaging. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0035] Figure 1 Schematic diagram of a terahertz single-pixel imaging system in an embodiment of the present invention;

[0036] Figure 2 This is a flow chart of terahertz single-pixel imaging in an embodiment of the present invention;

[0037] Figure 3 A schematic diagram of a network reconstruction network model structure in an embodiment of the present invention;

[0038] Figure 4 Schematic diagram of the structure of a convolution block in an embodiment of the present invention;

[0039] Figure 5 These are the original images of some handwritten digit recognition in the embodiments of the present invention, the images reconstructed by the traditional image algorithm with a sampling rate of 0.8% and 2.5%, and the images reconstructed by the SIDL algorithm. DETAILED DESCRIPTION

[0040] In order to make the purpose, technical scheme and advantages of the embodiments of the present invention clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0041] First, build a terahertz single-pixel imaging device, such as Figure 1 As shown. The main optical path of the imaging device is composed of a terahertz laser and a detector, and the modulation part is composed of a laser, a digital micromirror array (DMD), a projection lens, and an intrinsic semiconductor. The laser is masked with a DMD and then projected onto the modulator through a lens through indium tin oxide (ITO) glass. ITO is transparent to visible light and reflects terahertz light. The terahertz light is collimated by the lens, penetrates the modulator and the target X, and is focused on the detector by the lens. The terahertz light is modulated by the mask, and the modulated intensity signal I is output on the detector.

[0042] Any two-dimensional image can be regarded as a weighted image obtained by a set of complete orthogonal mask patterns. Each mask pattern corresponds to a frequency point in the transform domain. The relationship between the target pattern and the transform domain function is given by formula (1).

[0043]

[0044] Where I(x, y) is the object, M, N are the length and width of the object, u, v are the point coordinates of the frequency in the transform domain, f is a two-dimensional matrix function, the size is determined by (x, y, u, v), a uv is the weight size, which is uniquely determined by (u, v). The process of weighting all orthogonal basis patterns to obtain the original pattern is called full sampling, in which the number of measurements is equal to K = M x N; in order to reduce the number of measurements, the coefficients on the partial transform domain are collected and weighted inverted to obtain a sampling method with information missing patterns, which is called undersampling. In one embodiment, the data obtained by Hadamard coding according to undersampling is incomplete data in the transform domain; when the data is inversely transformed, the spatial resolution will decrease due to the lack of information, making the overall image quality blurred.

[0045] An embodiment of the present invention provides a terahertz single-pixel imaging method, which includes the following steps:

[0046] S1. construct an image reconstruction network model, select a sampling scheme, and train the image reconstruction network model based on the sampling scheme to obtain a trained image reconstruction network model;

[0047] The sampling schemes can be Hadamard coding, Fourier coding and wavelet transform coding.

[0048] The present invention proposes a new deep convolutional neural network SIDL, which can train various single-pixel imaging algorithms and even terahertz single-pixel imaging, realize the direct conversion from one-dimensional data to two-dimensional images, and ensure high-quality reconstruction of images. The schematic diagram of the image reconstruction network model structure is shown in the figure. Figure 3 shown.

[0049] like Figure 3 As shown in the figure, the image reconstruction network model SIDL consists of a fully connected block and a triangle dense block. The triangle dense block consists of a convolution block, a downsampling layer, an upsampling layer, and a residual dense connection. First, the input data is restored to feature map 0 through the fully connected layer. At this time, feature map 0 is consistent with the reconstructed output size. After entering the first convolution block, it enters the first feature channel 1 and obtains the feature Figure 1 , the number of feature channels is 64, and N-1 feature maps are obtained through N-1 convolution blocks in sequence; the feature map is compressed through the downsampling layer, and the number of feature channels increases by 128, obtaining feature map 11, and it enters n-1 convolution layers in sequence to obtain n-1 feature maps, where n=N / 2, that is, the number of convolution blocks entering the next feature channel is half of the number of the previous layer, and the number of feature channels is twice that of the previous layer. Similarly, feature map 11 obtains feature map 21 through the downsampling layer. The feature map of each feature channel will enter the fusion layer to obtain the total feature map, and enter the feature channel of the previous layer through the upsampling layer. Finally, all feature maps are fused through the fusion layer of the first layer, and the output map is restored through a convolution layer and residual connection.

[0050] This network model combines residual dense connection and deep compression (i.e. downsampling). At the same time, each compression will reduce the network depth of the corresponding layer, which will increase the training time while ensuring that the gradient will not disappear due to a small receptive field. Its advantages are: first, the entire network has memory, so that each layer of the network has all the information previously input, so that no matter how deep the network is, there will be no gradient explosion; second, deep compression can mine the depth information of undersampled images, so that after each downsampling training, valid information is obtained, thereby improving the quality of the image and making it closer to the real image.

[0051] The convolution block diagram is as follows Figure 4As shown in the figure, it consists of three convolutional layers with activation layers, one convolutional layer and residual connections. Using convolutional blocks instead of convolutional layers as the basic unit of the network can increase the number of network layers to achieve feature learning; adding residual connections as short connections can avoid the vanishing gradient caused by the superposition of convolutional layers and accelerate the convergence speed; at the same time, the additional convolutional layers on the residual connections can adapt to the changes in different feature channels. In addition to the convolutional block structure proposed above, the convolutional block representation structure can also be customized according to the network training requirements, such as deleting redundant convolutional layers or adding batch normalization layers before the activation layers. In essence, convolutional blocks are convolutional layers and activation layers that can achieve better training results.

[0052] Before pre-training the network, the corresponding data set is selected according to the scene of the reconstructed image, and a certain degree of random noise or additional image processing is added to the input image data. The processed image is inversely transformed into a one-dimensional signal as the input source according to the encoding method, and constitutes a training set with the original image.

[0053] S2. Generate a corresponding mask pattern according to the sampling scheme;

[0054] S3. The terahertz laser is modulated by a mask, and the modulated terahertz laser interacts with the target object, and the resulting modulated signal is received by the detector, and undersampled one-dimensional data is obtained;

[0055] S4. Importing the undersampled one-dimensional data into a trained image reconstruction network model to obtain a reconstructed image.

[0056] Another aspect of the embodiments of the present invention further provides a terahertz single-pixel imaging system, comprising:

[0057] A training unit selects a corresponding sampling scheme, constructs an image reconstruction network model, and trains the image reconstruction network model to obtain a trained image reconstruction network model;

[0058] A generating unit, generating a corresponding mask pattern according to the sampling scheme;

[0059] An acquisition unit modulates the terahertz wave through a mask, and the modulated terahertz wave interacts with the target object, and the formed modulated signal is received by the detector, and undersampled one-dimensional data is obtained;

[0060] The reconstruction unit imports the undersampled one-dimensional data into a trained image reconstruction network model to obtain a reconstructed image.

[0061] The functions of the above-mentioned units can be found in the corresponding methods and will not be described in detail here.

[0062] The imaging method flow chart of the present invention is as follows Figure 2As shown in the figure, first, a sampling scheme is selected for single-pixel imaging, such as Hadamard coding, and a training set is generated for the coding method. The handwritten digit recognition data set MNIST is used for model pre-training in the simulation; after the training is completed, the coding mask patterns equal to the number of under-sampling and full-sampling of the image are sent to the digital micromirror device respectively; the 808nm laser is irradiated on the digital micromirror device to make the mask pattern reflect onto the modulator of the target pattern area; after terahertz laser emission, the 808nm laser intensity is modulated by the terahertz modulator, and the terahertz laser is received by a terahertz single-point detector to obtain a set of one-dimensional under-sampling intensity signals and a set of one-dimensional full-sampling intensity signals, namely, transform domain data; finally, the under-sampled one-dimensional data is passed into the image reconstruction network model SIDL, restored to a 32x32 pixel image, and compared with the image after inverse transformation after full sampling to evaluate the image quality.

[0063] In the present invention, the handwritten digit recognition data set MNIST is used for experimental simulation. The training set is 60,000 28x28 pixel images, and the verification set is 10,000 28x28 pixel images. After being enlarged to 32x32 pixels, they are used as original images, and one-dimensional data with Gaussian noise added and Hadamard encoding is used as the training set. The training parameters are set as follows: the number of training rounds epoch is 25, the batch size is batch_size is 25, the learning rate learning_grate is 10-5, and the loss function Loss is the L1 function. In the simulation experiment, the model training is carried out for image sampling rates of 0.8% and 2.5%. Some effect diagrams of the verification set are shown in the figure below. Figure 5 shown.

[0064] Due to the large amount of missing information, the digital content of the image with a sampling rate of 2.5% is basically unrecognizable after reconstruction using the inverse Hadamard transform; when the sampling rate reaches 0.8%, the image content is basically unrecognizable as numbers; after reconstruction using the SIDL model, 0.8% of the data can be restored to a relatively similar number through the prediction of the input information, while 2.5% of the data can be reconstructed into the original image to a great extent. Figure 5 The image quality evaluation indicators are shown in Table 1.

[0065] Table 1

[0066]

[0067] In the table, ten different images were reconstructed by Hadamard inverse transform and SIDL algorithm respectively at two sampling rates. In the traditional algorithm, the highest SSIM=0.33 and PSNR=13 can be obtained at a sampling rate of 2.5%, while the SSIM at 0.8% is only between 0.03-0.13 and the PSNR does not exceed 13; after SIDL reconstruction, the SSIM at 0.8% is between 0.3-0.6 and the PSNR is between 12-21, while 2.5% is even better, with SSIM between 0.75-0.92 and PSNR between 17-32. Therefore, in the model simulation, we believe that the SIDL algorithm reconstruction effect is best at a sampling rate of 2.5%.

[0068] It should be noted that, in this application, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0069] The above description is only a specific implementation of the present application, so that those skilled in the art can understand or implement the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest range consistent with the principles and novel features applied for herein.

Claims

1. A terahertz single-pixel imaging method, characterized in that: The following steps are involved: S1. construct an image reconstruction network model, select a sampling scheme, and train the image reconstruction network model based on the sampling scheme; S2. Generate a corresponding mask pattern according to the sampling scheme, wherein the sampling scheme is Hadamard coding; S3. The terahertz laser is modulated by the mask pattern, and the modulated terahertz laser interacts with the target object, and the resulting modulated signal is received by the detector, and undersampled one-dimensional data is obtained; S4. Import the undersampled one-dimensional data into the trained image reconstruction network model to obtain a reconstructed image; the image reconstruction network model includes a fully connected block and a triangular dense block, wherein the triangular dense block includes a convolution block, a downsampling layer, an upsampling layer and a residual dense connection, wherein the convolution block includes a convolution layer with an activation layer, a normal convolution layer and a residual connection, firstly, the input data is restored to feature map 0 through the fully connected layer, at which time feature map 0 is consistent with the reconstructed output size, and then enters the first feature channel 1 after entering the first convolution block, and obtains feature map 1, the number of channels in feature map 1 is 64, and N-1 feature maps are obtained through N-1 convolution blocks in sequence, and feature map 1 is compressed to a scale through the downsampling layer to obtain a feature map 11, the number of channels in feature map 11 increases to 128, and enters n-1 convolution blocks in turn to obtain n-1 feature maps, where n=N / 2, that is, the number of convolution blocks entering the next layer of feature channels is half of the number of convolution blocks in the previous layer, and the number of channels in each feature map is twice that of the previous layer. Similarly, feature map 11 obtains feature map 21 through the downsampling layer, and the feature map of each feature channel enters the fusion layer to obtain the total feature map, and enters the feature channel of the previous layer through the upsampling layer. Finally, all feature maps are fused through the fusion layer of the first layer, and the output map is restored through a convolution layer and residual connection. This network model combines residual dense connection with deep compression, namely downsampling, and each compression will reduce the network depth of the corresponding layer.

2. The terahertz single-pixel imaging method according to claim 1, characterized in that: Step S3 includes: sending the mask pattern to a digital micromirror device, irradiating the digital micromirror device with a laser, and reflecting the mask pattern onto a modulator in a target pattern area, then emitting a terahertz laser and modulating the laser intensity using a terahertz modulator, and simultaneously receiving the terahertz laser through a terahertz single-point detector to obtain under-sampled and fully sampled one-dimensional data.

3. The terahertz single-pixel imaging method according to claim 2, characterized in that: Step S4 also includes: performing inverse transformation image processing on the fully sampled one-dimensional data, comparing it with the reconstructed image, and performing image quality evaluation.

4. The terahertz single-pixel imaging method according to claim 1, characterized in that: Step S1 also includes: Before pre-training the image reconstruction network model, a corresponding data set is selected according to the scene of the reconstructed image, and a certain degree of random noise or additional image processing is added to the input image data.

5. The terahertz single-pixel imaging method according to claim 1, characterized in that: Step S1 includes: A training set is generated according to the sampling scheme, wherein the training set includes an original image and a one-dimensional signal converted by inversely transforming the image according to the sampling scheme.

6. A terahertz single-pixel imaging system of an end-to-end network for implementing the terahertz single-pixel imaging method according to claim 1, characterized in that: include: A training unit constructs an image reconstruction network model, selects a sampling scheme, and trains the image reconstruction network model based on the sampling scheme; A generating unit, generating a corresponding mask pattern according to the sampling scheme, wherein the sampling scheme is Hadamard coding; An acquisition unit modulates the terahertz laser through the mask pattern, the modulated terahertz laser interacts with the target object, the formed modulated signal is received by the detector, and undersampled one-dimensional data is obtained; A reconstruction unit, which imports the undersampled one-dimensional data into a trained image reconstruction network model to obtain a reconstructed image; The image reconstruction network model includes a fully connected block and a triangular dense block, wherein the triangular dense block includes a convolution block, a downsampling layer, an upsampling layer and a residual dense connection, wherein the convolution block includes a convolution layer with an activation layer, a normal convolution layer and a residual connection.

Images