A terahertz single-pixel super-resolution imaging method and system

Abstract

The present invention discloses a terahertz single-pixel super-resolution imaging method and system, belonging to the field of terahertz imaging. The present invention first learns the stripe patterns that need to be projected for a certain type of pattern through a statistical method to achieve undersampled terahertz single-pixel imaging and achieve the purpose of improving the imaging speed. At the same time, in order to make up for the information loss caused by insufficient sampling rate and the interference noise caused by other factors, an image super-resolution network is used to perform image denoising and pixel reconstruction in the later stage to achieve high-quality imaging effects.

Description

Technical Field

[0001] The present invention belongs to the field of terahertz imaging, and more specifically, relates to a terahertz single-pixel super-resolution imaging method and system. Background Art

[0002] Terahertz waves have the advantages of low photon energy and the ability to penetrate non-polar substances, and have great potential in terahertz imaging, spectral analysis, high-speed communication, etc. Among them, terahertz imaging is of great significance in biomedicine, material detection, and security monitoring. However, due to the lack of suitable materials, the research and development progress of terahertz pixelated detector arrays is slow. Currently, most multi-pixel terahertz detector arrays are narrowband or require operation in a cryogenic refrigeration environment, which greatly restricts the practical popularization of terahertz imaging technology.

[0003] Currently, the new terahertz imaging technology is the terahertz single-pixel imaging system, which not only saves hardware costs compared to the area array terahertz imaging system, but also brings new possibilities for the miniaturization and commercialization of terahertz. Currently, the terahertz single-pixel imaging system is realized through methods such as compressive sensing, Hadamard basis, and Fourier basis. Among them, She et al. used a 220μm silicon-based graphene modulator and Fourier fringes to achieve sub-wavelength terahertz image reconstruction, and utilized the sparse characteristics of the image to reconstruct the image under a 10% modulation mask by collecting low-frequency coefficients and inverse Fourier transform; Rayko et al. used a silicon total internal reflection prism and a Hadamard mask to achieve near-real-time terahertz single-pixel video, and adopted an undersampling technique to reduce the sampling time by using the sparse characteristics in the Hadamard domain.

[0004] Due to the particularity of the terahertz wave wavelength and the limitations of single-pixel imaging, the terahertz single-pixel imaging technology has the following problems to be solved: (1) Slow imaging speed. Since single-pixel imaging sends light intensity of several mask patterns through a modulator to a single detector, and the imaging speed depends on factors such as modulation time, the number of projections, the projection speed, and the response time of the detector; when the image resolution increases, the increase in the number of samples will also slow down the imaging speed. (2) Poor imaging quality: Terahertz waves are extremely vulnerable to coherent light interference during transmission, and at the same time, the detection error caused by hardware thermal noise greatly affects the imaging quality. In addition, although the undersampling method can significantly shorten the imaging time, it will cause the loss of high-frequency information in the output image and deteriorate the image; from the observable perspective, when the sampling rate is lower than a certain ratio, the reconstructed pattern will have serious distortion or even distortion, so the sampling rate is not suitable to be lower than a certain lower limit. Summary of the Invention

[0005] Aiming at problems such as slow imaging speed and poor imaging quality in related technologies, the purpose of the present invention is to provide a terahertz single-pixel super-resolution imaging method and system. By using a statistical method to a priori learn the fringe patterns that need to be projected for a certain type of pattern, undersampled terahertz single-pixel imaging is realized to achieve the purpose of improving the imaging speed. At the same time, in order to make up for the information loss caused by insufficient sampling rate and the interference noise caused by other factors, an image super-resolution network is used to perform post-image denoising and pixel reconstruction to achieve a high-quality imaging effect.

[0006] To achieve the above object, one aspect of the present invention provides a terahertz single-pixel super-resolution imaging method, including the following steps:

[0007] S100. Perform a priori learning on the coding positions of similar patterns, extract the sampling strategy saved through a priori learning, and generate a mask pattern, where the number of the mask patterns is equal to the product of the number of sampling times and the sampling rate in the single-pixel imaging reconstruction of the pixel image;

[0008] S200. Based on the sampling strategy, select a data set for corresponding processing to generate a training set, and use the training set to initialize the residual dense network;

[0009] S300. Build a terahertz single-pixel imaging system. According to the sampling strategy, load the mask pattern using a digital micromirror array and reflect the laser onto the target pattern through a laser; the terahertz wave irradiates the terahertz modulator to modulate the laser intensity of the target pattern, and the terahertz detector periodically receives the terahertz wave, restores the transform domain information from the intensity information and decodes to obtain the first image;

[0010] S400. Based on the first image, reconstruct the second image through the residual dense network.

[0011] Further, the step S100 specifically includes the following steps:

[0012] S101. Classify similar pictures of the collected target pattern to form a data set;

[0013] S102. Perform full sampling on multiple pictures of the same class and save the transform domain data;

[0014] S103. Use a statistical method to statistically calculate the corresponding transform domain positions in the transform domain of this type of image where the modulus coefficient intensity satisfies a specific sampling rate, thereby generating the mask pattern of this type of image.

[0015] Further, in the step S102, the inverse Fourier transform is used in the transform domain.

[0016] Further, the step S400 specifically includes the following steps:

[0017] In S401, the input image extracts shallow information feature maps through a convolutional layer and then enters the residual dense block;

[0018] In S402, each residual dense block will generate corresponding feature maps. All the feature maps are concatenated to form a whole for dense feature fusion;

[0019] In S403, the first image is upsampled to a second image with high pixels through an upsampling module and reconstructed to restore a single-channel high-definition grayscale image or a three-channel high-definition color image.

[0020] Furthermore, the first image is an image with 32×32 pixels, and the second image is an image with 64×64 pixels.

[0021] Furthermore, the process of the dense feature fusion includes global feature fusion and global residual learning. First, a fusion layer is used to combine the feature maps into a whole, and deep information is further obtained through convolutional operations and restored to a feature map with 64 channels. Finally, the shallow information and the deep information are added through residual learning.

[0022] On the other hand, the present invention also provides a terahertz single-pixel super-resolution imaging system, including:

[0023] A learning unit that performs prior learning on the encoding positions of similar patterns, extracts the sampling strategy saved by the prior learning, and generates a mask pattern. The number of the mask patterns is equal to the product of the sampling times and the sampling rate of the pixel image in single-pixel imaging reconstruction;

[0024] A training unit that selects a corresponding data set for image scaling. At the same time, based on the sampling strategy, the corresponding coefficients in the transform domain of the processed image are selected for inverse transformation to form a training set, and the training set is used to initialize the residual dense network;

[0025] An acquisition unit that loads the mask pattern of the prior learning by using a digital micromirror array and reflects the laser to the target pattern through the laser; the terahertz wave irradiates the terahertz modulator to modulate the laser intensity of the target pattern, and the terahertz detector periodically receives the terahertz wave, restores the transform domain information from it and decodes to obtain the first image;

[0026] A reconstruction unit that reconstructs the second image based on the first image through the residual dense network.

[0027] Furthermore, the learning unit includes:

[0028] A classification module that classifies similar pictures of the collected target pattern to form a data set;

[0029] A full-sampling module that performs full sampling on multiple pictures of the same class and saves the transform domain data;

[0030] A generation module that uses a statistical method to count the positions in the transform domain of this type of image where the modulus coefficient intensity satisfies the corresponding transform domain at a specific sampling rate, thereby generating a mask pattern for this type of image.

[0031] Furthermore, the reconstruction unit includes:

[0032] A convolution module where the input image extracts shallow information feature maps through a convolution layer and then enters a residual dense block;

[0033] A fusion module where each residual dense block generates corresponding feature maps, and all the feature maps are stitched together to form a whole for dense feature fusion;

[0034] An interpolation module that upsamples the first image to a second image with high pixels through an upsampling module and performs reconstruction to restore it to a single-channel high-definition grayscale image or a three-channel high-definition color image.

[0035] Furthermore, the first image is an image of 32×32 pixels, and the second image is an image of 64×64 pixels.

[0036] Through the above technical solution conceived by the present invention, compared with the prior art, the prior optimal sampling scheme and the super-resolution reconstruction of deep learning are combined into a terahertz single-pixel imaging system to achieve high-definition image reconstruction within the fastest imaging time. Specifically, the present invention adopts a computer-aided optimization sampler based on prior statistics, which is applicable to the imaging and monitoring of common similar targets and is superior to most undersampling schemes; at the same time, in order to improve the generalization degree of this type, during the process of expanding and collecting subsequent samples, the sampling scheme can be further statistically optimized, so this is a learnable sampling method; in addition, the present invention applies a residual dense network to terahertz imaging, which not only eliminates the image noise brought by the system but also improves the resolution without increasing the sampling quantity. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 is a schematic diagram of a terahertz single-pixel imaging system in an embodiment of the present invention;

[0038] FIG. 2(a) is a schematic diagram of the RDN network structure in an embodiment of the present invention;

[0039] FIG. 2(b) is a schematic diagram of the RDB network structure in an embodiment of the present invention;

[0040] Figure 3 is a working flowchart of a fast terahertz single-pixel super-resolution reconstruction system in an embodiment of the present invention;

[0041] Figure 4It is the effect diagram of the samples in the embodiments of the present invention under 8% and 10% with different sampling methods. Detailed implementation manners

[0042] In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0043] First, build a terahertz single-pixel imaging device, as Figure 1 shown. The main optical path of the imaging device consists of a terahertz laser and a detector, and the modulation part consists of a laser, a digital micromirror device (DMD), a projection lens, and an intrinsic semiconductor. The laser is masked by the DMD Ф and then passes through the lens and penetrates indium tin oxide (ITO) glass and is projected onto the modulator. ITO is transparent to visible light and reflects terahertz light. The terahertz light passes through the lens for collimation, penetrates the modulator and the target object X, and is focused on the detector by the lens. The terahertz light is modulated by the mask, and the modulated intensity signal I is output on the detector.

[0044] Based on the above device, an embodiment of the present invention provides a terahertz single-pixel super-resolution imaging method, including the following steps:

[0045] S100. Perform prior learning on the coding positions of similar patterns, extract the sampling strategy saved through prior learning, and generate a mask pattern, where the number of the mask patterns is equal to the product of the sampling times and the sampling rate in the single-pixel imaging reconstruction of the pixel image.

[0046] Any two-dimensional image can be regarded as obtained by weighting a set of complete orthogonal mask patterns. Each mask pattern corresponds to a frequency point in the transform domain, and the relationship between the target pattern and the transform domain function is given by formula (1).

[0047]

[0048] In the formula, I(x, y) is the object target, M and N are the length and width of the object target, u and v are the point coordinates of the frequency in the transform domain, f is a two-dimensional matrix function, and its size is determined by (x, y, u, v), a uvis the weight size, which is uniquely determined by (u, v). The process of weighting all orthogonal basis patterns to obtain the original pattern is called full sampling, where the number of measurements is equal to K = MxN; in order to reduce the number of measurements, collecting coefficients in a partial transform domain for weighted inversion to obtain a sampling method with a pattern having missing information is called undersampling. Among them, the prior statistical sampling is given by formula (2).

[0049]

[0050] In the formula, |F(x, y)| represents the modulus of a certain point in the transform domain, p is the sampling rate, and a is the absolute value of the threshold corresponding to p, indicating that the coefficient at the selected position in the transform domain accounts for the probability p of the modulus of the entire transform domain coefficient distribution in the front.

[0051] Among them, the step S100 specifically includes the following steps:

[0052] S101. Classify similar pictures of the collected target pattern to form a data set;

[0053] S102. Perform full sampling on multiple pictures of the same class and save the transform domain data;

[0054] S103. Use statistical methods to statistically calculate the modulus coefficient intensity in the transform domain of this class of images that satisfies the corresponding transform domain position at a specific sampling rate, so as to generate a mask pattern for this class of images.

[0055] S200. Based on the sampling strategy, select a data set for corresponding processing to generate a training set, and use the training set to initialize the residual dense network.

[0056] This method is applicable to the imaging and monitoring of common similar targets and is superior to most undersampling schemes; at the same time, in order to improve the generalization degree of this class, during the process of expanding and collecting subsequent samples, the sampling scheme can be further statistically calculated and optimized. Therefore, this is a learnable sampling method. Thus, in order to improve the sampling speed, the terahertz single-pixel imaging system adopting the prior statistical undersampling scheme in the present invention uses a relatively mature inverse Fourier transform in the transform domain.

[0057] S300. Build a terahertz single-pixel imaging system. According to the sampling strategy, load the mask pattern using a digital micromirror array and reflect it onto the target pattern through a laser; the terahertz wave irradiates the terahertz modulator to modulate the laser intensity of the target pattern, and the terahertz detector periodically receives the terahertz wave, restores the transform domain information from the intensity information and decodes to obtain the first image;

[0058] S400. Based on the 32×32 pixel image, reconstruct a 64×64 pixel image through a residual dense network.

[0059] In order to reconstruct a high - definition image from an undersampled image with missing information, further processing of the imaging result is required. The Residual Dense Network (RDN) can extract feature information from the shallow and deep parts of the image for adaptive feature fusion, and use the upsampling layer for interpolation to obtain a high - definition image. The network structure of RDN is shown in Figure 2(a). The structure is divided into: two ConV layers for extracting shallow - layer features; Residual Dense Blocks (RDBs) for extracting features of each layer; Dense Feature Fusion (DFF) for splicing the features of each layer; and an upsampling module for obtaining single - channel or multi - channel results. The input image passes through two ConV layers with a convolutional kernel size of 3x3 and filters of 1 and 64 to extract the shallow - layer information feature map, and then enters the RDBs to obtain the deep - layer information. The RDBs are composed of multiple RDBs. The network structure of the RDB is given in Figure 2(b). Each RDB corresponds to a different receptive field and can extract different local features, which is given by the formula:

[0060]

[0061] Looking at the network structure, the RDB is composed of a residual block and a dense block, with a filter size of 64 and a convolutional kernel size of 3x3 for both. The RDB has the following characteristics: a continuous memory mechanism, a local fusion mechanism, and a local residual mechanism. The continuous memory mechanism: each RDB transmits several previous states to the current layer for processing. Each layer consists of a ConV layer and a ReLU layer; the more layers are towards the back, the more memory states are transmitted. The local fusion mechanism: concatenates the transmitted states and the current input, can adaptively extract deep - layer features from multiple features, and reduces the number of channels and output numbers. The local residual learning superimposes the output of the previous RDB and the feature fusion part to further improve the information flow and the expression ability of the model.

[0062] After a series of RDBs, each RDB will generate a corresponding feature map. All the feature maps are spliced together to form a whole, which is called dense feature fusion. Dense feature fusion is divided into global feature fusion and global residual learning. First, use the fusion layer to combine each feature map into a whole, and further obtain deep - layer information and restore it to a 64 - channel feature map through a 1x1 convolutional operation and a 3x3 convolutional operation. Finally, add the shallow - layer information and the deep - layer information through residual learning. The whole dense feature fusion can be expressed by the formula:

[0063] F DF = F0 + H GEF ([F0,F d ,...,F D ) (4)

[0064] Finally, the 32x32 pixel image is interpolated to 64x64 pixels through the upsampling module and reconstructed, and restored to a single-channel high-definition grayscale image or a three-channel high-definition color image. The entire RDN network can be expressed as:

[0065]

[0066] To reconstruct the target image The loss function for network training uses the L1 loss function, which can be expressed as:

[0067]

[0068] The specific steps of step S400 include the following steps:

[0069] S401. The input image extracts shallow information feature maps through the convolutional layer and then enters the residual dense block;

[0070] S402. Each residual dense block will generate corresponding feature maps, and all the feature maps are spliced together to form a whole for dense feature fusion;

[0071] S403. The first image is upsampled to a second image with high pixels through the upsampling module and reconstructed, and restored to a single-channel high-definition grayscale image or a three-channel high-definition color image.

[0072] Another aspect of the embodiment of the present invention also provides a terahertz single-pixel super-resolution imaging system, including:

[0073] A learning unit that performs prior learning on the encoding positions of similar patterns, extracts the sampling strategy saved by the prior learning, and generates a mask pattern. The number of the mask patterns is equal to the product of the sampling times and the sampling rate of the pixel image in single-pixel imaging reconstruction;

[0074] A training unit that selects a corresponding data set for image scaling, and at the same time, based on the sampling strategy, selects the corresponding coefficients in the transformed domain of the processed image for inverse transformation to form a training set, and uses the training set to initialize the residual dense network;

[0075] An acquisition unit that loads the mask pattern of the prior learning by using a digital micromirror array and reflects the laser to the target pattern through laser; the terahertz wave irradiates the terahertz modulator to modulate the laser intensity of the target pattern, and the terahertz detector periodically receives the terahertz wave, restores the information in the transformed domain therefrom and decodes to obtain the first image;

[0076] A reconstruction unit that reconstructs the second image based on the first image through the residual dense network.

[0077] For the functions of the above units, refer to the corresponding methods and will not be elaborated here.

[0078] The flowchart of the imaging method of the present invention is as Figure 3 shown. First, through computer a priori learning of the sampling strategy of the dataset similar to the target pattern, and generating the corresponding mask pattern, the number of generated stripes is equal to the number of sampling times of the pixel image in the transformation multiplied by the sampling rate (the sampling rate in this experiment is 8%); select the corresponding dataset for image scaling, and at the same time, for the sampling strategy of a priori learning, select the corresponding coefficients in the transformed domain of the processed image for inverse transformation to form a training set to initialize the residual dense network; build a terahertz single-pixel imaging system, and load the mask pattern on the DMD; secondly, using the principle of near-field imaging, place the target image and the DMD at a certain distance, close to the terahertz modulator, reflect the mask pattern on the DMD to the target by an 808nm laser, and modulate the laser intensity of the target pattern by the terahertz light irradiated on the terahertz modulator; the light intensity value of each sampling count is collected by the detector, and the computer restores the transformed domain information and decodes it into an image of 32x32 pixels. The image will have the problem of low resolution due to scattering. Finally, through the residual dense network, map the low-resolution image and the high-resolution image to restore a high-definition image of 64x64 pixels, achieving the effect of undersampled high-resolution image reconstruction.

[0079] Next, experiments are used to compare the effects of samples under different sampling methods. In order to retain as much information as possible and avoid excessive loss of details, in this experiment, the 32x32 pixel sample pattern is sampled at an 8% sampling rate, and at the same time, the effect diagram and the corresponding optimal sampling method at a 10% sampling rate are introduced for comparison. In Figure 4 and Table 1, the corresponding sampling methods are circular, square, block, a priori statistics, and optimal sampling in turn. In the present invention, two evaluation indexes, structural similarity (SSIM) and peak signal-to-noise ratio (PSNR), are adopted. It can be seen from Table 1 that the a priori statistics at an 8% sampling rate is similar to the 10% block mode, is better than the other two sampling methods, and is slightly weaker than the optimal sampling at the same sampling rate.

[0080] Table 1 Comparison of the effects of samples under different sampling methods at 8% and 10%

[0081]

[0082] Those skilled in the art can easily understand that the above are only the preferred embodiments of the present invention and are not used to limit the present invention. Any modifications, equivalent replacements, and improvements made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A terahertz single-pixel super-resolution imaging method, characterized in that It includes the following steps: S100. Conduct prior learning on the encoding positions of similar patterns, extract the sampling strategy saved through prior learning, and generate a mask pattern. The number of mask patterns is equal to the product of the sampling times and the sampling rate in the single-pixel imaging reconstruction of the pixel image. The process of generating the mask pattern includes: S101. Classify the similar pictures of the acquired target pattern to form a data set; S102. Perform full sampling on multiple pictures of the same class and save the transform domain data; S103. Use statistical methods to statistically calculate the modulus coefficient intensity in the transform domain of the same class of images at the corresponding transform domain positions under a specific sampling rate, so as to generate the mask pattern of this class of images. S200. Based on the sampling strategy, select a data set for corresponding processing to generate a training set, and use the training set to initialize the residual dense network. S300. Build a terahertz single-pixel imaging system. According to the sampling strategy, load the mask pattern using a digital micromirror array and reflect the laser onto the target pattern through the laser; the terahertz wave irradiates the terahertz modulator to modulate the laser intensity of the target pattern, and the terahertz detector periodically receives the terahertz wave, and the transform domain information is restored from the intensity information and decoded to obtain the first image. S400. Based on the first image, perform dense feature fusion through the residual dense network to reconstruct the second image. Among them, the dense feature fusion is divided into global feature fusion and global residual learning.

2. The terahertz single-pixel super-resolution imaging method according to claim 1, wherein In step S102, the inverse Fourier transform is used in the transform domain.

3. The terahertz single-pixel super-resolution imaging method according to claim 1, characterized in that Step S400 specifically includes the following steps: S401. The input image extracts the shallow information feature map through the convolutional layer and then enters the residual dense block. S402. Each residual dense block will generate the corresponding deep information feature map, splice all the feature maps together to form a whole, and perform dense feature fusion. Among them, all the feature maps include the shallow information feature map and the deep information feature map. S403. Use the upsampling module to interpolate the first image to a high-pixel second image and perform reconstruction to restore it to a single-channel high-definition grayscale image or a three-channel high-definition color image.

4. The terahertz single-pixel super-resolution imaging method according to claim 3, wherein The first image is an image of 32×32 pixels, and the second image is an image of 64×64 pixels.

5. The terahertz single-pixel super-resolution imaging method according to claim 4, characterized in that, The process of the dense feature fusion includes global feature fusion and global residual learning. First, use the fusion layer to combine each feature map into a whole, and further obtain deep information through convolutional operations and restore it to a 64-channel feature map. Finally, add the shallow information and the deep information through residual learning. Among them, each feature map includes the feature maps generated by each residual dense block.

6. A terahertz single-pixel super-resolution imaging system, characterized in that, It includes: Learning unit, which conducts prior learning on the encoding positions of similar patterns, extracts the sampling strategy saved by the prior learning, and generates a mask pattern. The number of the mask patterns is equal to the product of the sampling times and the sampling rate in the single-pixel imaging reconstruction of the pixel image. Wherein, the learning unit includes: a classification module, which classifies the similar pictures of the collected target pattern to form a data set; a full-sampling module, which conducts full sampling on multiple pictures of the same class and saves the transform-domain data; a generation module, which uses a statistical method to statistically calculate the modulus coefficient intensity in the transform domain of the same-class images that satisfies the corresponding transform-domain positions at a specific sampling rate, so as to generate the mask pattern of this class of images; Training unit, which selects the corresponding data set for image scaling, and at the same time, based on the sampling strategy, selects the corresponding coefficients in the transform domain of the processed image for inverse transformation to form a training set, and uses the training set to initialize the residual dense network. Wherein, the corresponding coefficients in the transform domain of the image are modulus coefficients; Acquisition unit, which loads the mask pattern of the prior learning by using a digital micromirror array and reflects the laser to the target pattern; the terahertz wave irradiates the terahertz modulator to modulate the laser intensity of the target pattern, and the terahertz detector periodically receives the terahertz wave, restores the transform-domain information from it and decodes to obtain the first image; Reconstruction unit, which reconstructs the second image through the residual dense network based on the first image.

7. The terahertz single-pixel super-resolution imaging system according to claim 6, wherein The reconstruction unit includes: Convolution module, the input image extracts the shallow information feature map through the convolutional layer and then enters the residual dense block; Fusion module, each residual dense block will generate the corresponding deep information feature map, splice all the feature maps together to form a whole, and conduct dense feature fusion. Wherein, all the feature maps include the shallow information feature map and the deep information feature map; Interpolation module, which up-samples the first image to the second image with high pixels through the up-sampling module and conducts reconstruction to restore it to a single-channel high-definition grayscale image or a three-channel high-definition color image.

8. The terahertz single-pixel super-resolution imaging system according to claim 7, characterized in that, The first image is an image with 32×32 pixels, and the second image is an image with 64×64 pixels.

Images