Real-time terahertz wave single-pixel imaging method, device, system and computer equipment

Abstract

The present invention relates to the technical field of terahertz wave single-pixel imaging, and discloses a real-time terahertz wave single-pixel imaging method, device, system and computer device, including obtaining terahertz wave single-pixel measurement values of an object to be imaged, where the terahertz wave single-pixel measurement values are one-dimensional signals obtained by measuring the object to be imaged through a terahertz wave single-pixel detector; and inputting the terahertz wave single-pixel measurement values into a pre-constructed image reconstruction model to obtain a terahertz wave reconstructed image of the object to be imaged. Through a physics-guided deep neural network model, the present invention can effectively improve the reconstruction quality and reconstruction time of terahertz wave images under undersampling conditions, and through a real-time terahertz wave single-pixel imaging device with a high-speed switch, effectively balances the relationship between imaging speed and imaging quality, and further improves the reconstruction efficiency of real-time terahertz wave single-pixel imaging.

Description

Technical Field

[0001] The present invention relates to the technical field of terahertz wave single-pixel imaging, and in particular, to a real-time terahertz wave single-pixel imaging method, device, system, and computer device. Background Art

[0002] Terahertz waves refer to electromagnetic waves with a frequency range of 0.1 - 10 THz, which are characterized by being non-invasive, non-destructive, and non-ionizing. Compared with X-rays, terahertz waves have lower photon energy, significantly reducing potential damage to biological tissues. In addition, terahertz wave imaging technology can detect internal structural information of opaque materials that are difficult to detect due to high absorption and strong scattering between visible light and matter. Based on these characteristics, terahertz wave imaging shows broad application prospects in fields such as non-destructive testing, security monitoring, and medical diagnosis.

[0003] Currently, commonly used terahertz wave single-pixel imaging methods include single-pixel imaging methods based on compressive sensing and single-pixel imaging methods based on deep learning. However, both of these methods have certain limitations. On the one hand, the method based on compressive sensing has the disadvantages of poor imaging quality and slow imaging speed. On the other hand, although the method based on deep learning can also be applied to terahertz wave single-pixel imaging, the supervised training dataset of a neural network with excellent performance depends on experimental data in real-time terahertz wave single-pixel imaging tasks. Due to the existence of various noises in terahertz wave systems, it is difficult to model. A data-driven deep learning model without any physical prior knowledge is not applicable to the reconstruction of terahertz wave images and cannot give the optimal result. These data-driven deep learning strategies also have common problems such as generalization and interpretability. Therefore, the current terahertz wave single-pixel imaging method based on deep learning also cannot reconstruct terahertz images with higher pixel resolution at video speed. Summary of the Invention

[0004] To solve the above technical problems, the present invention provides a real-time terahertz wave single-pixel imaging method, system, device, and computer device, so as to solve the problems of poor imaging quality and slow imaging speed existing in existing imaging methods, and achieve the effect of terahertz wave imaging with high quality, high pixel resolution, and video speed.

[0005] In a first aspect, the present invention provides a real-time terahertz wave single-pixel imaging method, the method comprising:

[0006] Obtaining a terahertz wave single-pixel measurement value of an object to be imaged, where the terahertz wave single-pixel measurement value is a one-dimensional signal obtained by measuring the object to be imaged through a terahertz wave single-pixel detector;

[0007] Input the single-pixel measurement value of the terahertz wave into a pre-constructed image reconstruction model to obtain a terahertz wave reconstruction image of the object to be imaged;

[0008] Among them, the image reconstruction model includes a physical encoding layer and a deep neural network model. The physical encoding layer is used to associate the single-pixel measurement value of the terahertz wave with a measurement matrix, generate a terahertz wave image matrix, and input it into the deep neural network model;

[0009] The deep neural network model adopts an encoder-decoder structure based on an attention gating mechanism, which is used to extract features and reconstruct the image from the terahertz wave image matrix, and output the terahertz wave reconstruction image.

[0010] Furthermore, the steps for training the image reconstruction model include:

[0011] Input a sample image into the physical encoding layer, encode the sample image through the measurement matrix to obtain a one-dimensional measurement vector, and perform correlation analysis operations on the one-dimensional measurement vector and the measurement matrix to generate a first terahertz wave image matrix;

[0012] Input the first terahertz wave image matrix into the deep neural network model, extract features and reconstruct the image from the first terahertz wave image matrix, and output a first terahertz wave reconstruction image;

[0013] Iteratively train the image reconstruction model according to the above steps until the iteration stop condition is reached to obtain the trained image reconstruction model. The iteration stop condition includes the convergence of the loss function or reaching the maximum number of iterations;

[0014] Among them, the one-dimensional measurement vector is represented by the following formula:

[0015]

[0016] In the formula, is the nth measurement value in the one-dimensional measurement vector, * is the convolution operator, is the nth measurement matrix, is the sample image, and x and y represent the image space coordinates;

[0017] The first terahertz wave image matrix is represented by the following formula:

[0018]

[0019] In the formula, is the first terahertz wave image matrix, N is the number of measurement matrices, is the mean operation;

[0020] The steps of performing image reconstruction using the trained image reconstruction model include:

[0021] Input the single-pixel terahertz wave measurement value into the physical encoding layer, perform correlation analysis operation on the single-pixel terahertz wave measurement value and the measurement matrix to obtain a second terahertz wave image matrix;

[0022] Input the second terahertz wave image matrix into the deep neural network model, perform feature extraction and image reconstruction on the second terahertz wave image matrix, and output the terahertz wave reconstructed image corresponding to the single-pixel terahertz wave measurement value.

[0023] Further, the deep neural network model includes a downsampling encoding layer, an upsampling decoding layer, and a multi-scale feature aggregation layer;

[0024] The downsampling encoding layer is used to extract feature maps of different scales, splice the feature maps of different scales through multi-scale skip connections, and transmit them to the upsampling decoding layer;

[0025] The upsampling decoding layer is used to perform feature fusion on the feature maps of different scales to obtain a terahertz wave aggregated feature map, and transmit it to the multi-scale feature aggregation layer;

[0026] The multi-scale feature aggregation layer adopts multi-scale deep supervision and is used to learn hierarchical representations from the terahertz wave aggregated feature map;

[0027] An attention gate control is set between the upsampling decoding layer and the multi-scale feature aggregation layer. The attention gate control is used to generate attention weights according to the terahertz wave aggregated feature map input by the upsampling decoding layer and the gating signal input by the multi-scale feature aggregation layer, and input them into the multi-scale feature aggregation layer;

[0028] Among them, the attention weights are represented by the following formula:

[0029]

[0030] In the formula, F represents the attention weight, x represents the terahertz wave aggregated feature map input by the upsampling decoding layer of this layer, g represents the gating signal input by the multi-scale feature aggregation layer of the next layer, represents matrix element addition, represents matrix element multiplication, represents a one-dimensional channel attention module, represents a two-dimensional spatial attention module.

[0031] Further, the loss function of the multi-scale feature aggregation layer is the minimum of the root mean square difference between the output of the multi-scale feature aggregation layer and the terahertz wave single-pixel measurement value, and the loss function of the deep neural network model is the weighted sum of the loss functions of the respective multi-scale feature aggregation layers.

[0032] Further, the loss function of the multi-scale feature aggregation layer is represented by the following formula:

[0033]

[0034] The loss function of the deep neural network model is represented by the following formula:

[0035]

[0036] In the formula, represents the loss function of the i-th multi-scale feature aggregation layer, represents the output of the i-th multi-scale feature aggregation layer, represents the sample image, x and y represent the image spatial coordinates, W represents the weight parameter, represents the loss function of the deep neural network model, represents the weight parameter of the i-th loss function, represents the noise term.

[0037] In a second aspect, the present invention provides a real-time terahertz wave single-pixel imaging device, the device comprising:

[0038] A terahertz wave source module, a terahertz wave spatial light modulation module, and a terahertz wave detection module, with the object to be imaged placed on the optical path between the terahertz wave source module and the terahertz wave spatial light modulation module;

[0039] The terahertz wave spatial light modulation module includes a pump laser, a beam expander lens, a digital micromirror device, a projection lens, a diaphragm, and a terahertz wave modulator;

[0040] The terahertz wave source module is used to emit continuous terahertz waves to the object to be imaged, and the generated terahertz wave beam carrying the information of the object to be imaged is received by the terahertz wave modulator;

[0041] The pump laser is used to generate a pump laser beam, and the pump laser beam is projected onto the digital micromirror device through the beam expander lens;

[0042] The digital micromirror device is used to modulate and reflect the pump laser beam, and the reflected pump laser beam is received by the terahertz wave modulator through the projection lens and the diaphragm;

[0043] The terahertz wave modulator is used to modulate the received terahertz wave beam according to the received pump laser beam to generate a modulated terahertz wave beam;

[0044] The terahertz wave detection module is used to detect the modulated terahertz wave beam to generate a terahertz wave single-pixel measurement value.

[0045] Further, the terahertz wave detection module includes a converging parabolic mirror group, a terahertz wave detector, and a data acquisition module;

[0046] The converging parabolic mirror group is used to converge the modulated terahertz wave beam onto the terahertz wave detector;

[0047] The terahertz wave detector is a single-pixel detector, which is used to detect the modulated terahertz wave beam to generate a photoelectric signal;

[0048] The data acquisition module is used to acquire the photoelectric signal and convert it into a terahertz wave single-pixel measurement value.

[0049] Further, the digital micromirror device is also used to modulate the pump laser beam according to the speckle image generated based on the measurement matrix pre-loaded, and a trigger signal is output to the data acquisition module each time modulation is performed, so that the data acquisition module performs synchronous data acquisition according to the trigger signal.

[0050] In a third aspect, the present invention provides a real-time terahertz wave single-pixel imaging system, and the system includes:

[0051] A data acquisition module, which is used to acquire the terahertz wave single-pixel measurement value of the object to be imaged. The terahertz wave single-pixel measurement value is a one-dimensional signal obtained by measuring the object to be imaged through a terahertz wave single-pixel detector;

[0052] An image reconstruction module, which is used to input the terahertz wave single-pixel measurement value into a pre-constructed image reconstruction model to obtain a terahertz wave reconstructed image of the object to be imaged;

[0053] Wherein, the image reconstruction model includes a physical coding layer and a deep neural network model. The physical coding layer is used to associate the terahertz wave single-pixel measurement value with the measurement matrix to generate a terahertz wave image matrix and input it into the deep neural network model;

[0054] The deep neural network model adopts an encoder-decoder structure based on an attention gating mechanism, which is used to perform feature extraction and image reconstruction on the terahertz wave image matrix and output the terahertz wave reconstructed image.

[0055] Fourthly, an embodiment of the present invention further provides a computer device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the steps of the above method are implemented.

[0056] The present invention provides a real-time terahertz wave single-pixel imaging method, device, system, and computer device. The present invention constructs a physics-guided deep neural network model, guides the terahertz wave image reconstruction process through correlation analysis operations, and provides interpretability and physical image attributes for the imaging model. Through the physics-guided physical coding layer, accurate image reconstruction priors are learned in a data-driven manner, thereby having strong global feature and local feature modeling capabilities. The attention gate mechanism is adopted in the deep neural network model to focus on the feature information related to image reconstruction and effectively suppress the feature responses in the irrelevant background regions of the multi-scale feature maps. At the same time, multi-scale deep supervision is adopted in the output of each multi-scale feature aggregation layer to learn hierarchical representations from the aggregated feature maps, enhancing the network generalization ability, ensuring the accuracy of the imaging results, and effectively improving the terahertz wave image reconstruction quality under undersampling conditions. The present invention also provides a real-time terahertz wave single-pixel imaging device with a high-speed switch, effectively balancing the relationship between imaging speed and imaging quality, and further improving the reconstruction time and reconstruction quality of real-time terahertz wave single-pixel imaging. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 is a schematic flowchart of the real-time terahertz wave single-pixel imaging method in an embodiment of the present invention;

[0058] Figure 2 is Figure 1 the structural schematic diagram of the image reconstruction model in

[0059] Figure 3 is a structural schematic diagram of the real-time terahertz wave single-pixel imaging device in an embodiment of the present invention;

[0060] Figure 4 is another structural schematic diagram of the real-time terahertz wave single-pixel imaging device in an embodiment of the present invention;

[0061] Figure 5 is a schematic diagram of the experimental imaging results in an embodiment of the present invention;

[0062] Figure 6 is a structural schematic diagram of the real-time terahertz wave single-pixel imaging system in an embodiment of the present invention;

[0063] Figure 7 is the internal structure diagram of the computer device in an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0064] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Apparently, the described embodiments are some but not all of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0065] Please refer to Figure 1 , a real-time terahertz wave single-pixel imaging method proposed in the first embodiment of the present invention, including steps S10 to S20:

[0066] Step S10, obtaining the terahertz wave single-pixel measurement value of the object to be imaged, where the terahertz wave single-pixel measurement value is a one-dimensional signal obtained by measuring the object to be imaged with a terahertz wave single-pixel detector;

[0067] Step S20, inputting the terahertz wave single-pixel measurement value into a pre-constructed image reconstruction model to obtain the terahertz wave reconstructed image of the object to be imaged.

[0068] Aiming at the problems of poor imaging quality and slow imaging speed existing in the current terahertz wave single-pixel imaging method based on deep learning, the present invention provides a physics-guided deep neural network model for terahertz wave image reconstruction. The structure of the image reconstruction model will be described in detail below.

[0069] Please refer to Figure 2 , the image reconstruction model in this embodiment includes an upstream physics encoding for physics guidance and a downstream deep neural network model. Among them, the function of the physics encoding layer is to provide a physical image guidance for the downstream deep neural network model, thereby enhancing the generalization ability and interpretability of the model. The function of the deep neural network model is to further extract features and reconstruct the image of the image matrix output by the physics encoding layer, so as to obtain a high-precision reconstructed image. When training the image reconstruction model, the physics encoding layer and the deep neural network model of the image reconstruction model are trained uniformly. The specific training steps include:

[0070] Inputting the sample image into the physics encoding layer, encoding the sample image through a measurement matrix to obtain a one-dimensional measurement vector, and performing correlation analysis operations on the one-dimensional measurement vector and the measurement matrix to generate a first terahertz wave image matrix;

[0071] Inputting the first terahertz wave image matrix into the deep neural network model, extracting features and reconstructing the image of the first terahertz wave image matrix, and outputting the first terahertz wave reconstructed image;

[0072] Iteratively train the image reconstruction model according to the above steps until the iteration stop condition is reached, and obtain the trained image reconstruction model. The iteration stop condition includes the convergence of the loss function or reaching the maximum number of iterations.

[0073] In this embodiment, the processing process of the physical encoding layer includes the encoding process of the input sample image and the measurement matrix, and the correlation analysis process between the encoded one-dimensional measurement vector and the measurement matrix, that is, it is divided into two stages: encoding process and correlation analysis. Among them, the encoding process stage is applied in the training stage of the image reconstruction model. During training, the dataset selected in this embodiment is a dataset based on sample images, such as the STL10 image dataset. When the sample image in the dataset is input into the physical encoding layer, the sample image is encoded by the measurement matrix to obtain a one-dimensional measurement vector, and then the encoded one-dimensional measurement vector is subjected to correlation analysis with the measurement matrix. That is to say, the one-dimensional measurement vector obtained in the encoding process stage of the physical encoding layer is actually an analog of the terahertz single-pixel measurement value obtained by measuring the object to be imaged by the terahertz single-pixel detector. Through the encoding process, it is not necessary to use a large number of terahertz single-pixel measurement values to construct the training dataset, but directly use the existing dataset based on sample images to train the model, thereby improving the training efficiency of the model. During one training process, the terahertz wave image matrix output by the physical encoding layer is input into the deep neural network model for feature extraction and image reconstruction, so as to complete one iteration training. When the loss function of the image reconstruction model reaches the convergence condition or reaches the maximum number of iterations, the iterative training stops, and the trained image reconstruction model is obtained.

[0074] After the image reconstruction model is trained, input the terahertz single-pixel measurement value into the physical encoding layer of the trained image reconstruction model, perform correlation analysis on the terahertz single-pixel measurement value and the policy matrix, and generate a terahertz wave image matrix; that is to say, in the actual terahertz single-pixel reconstruction process, the processing of the terahertz single-pixel measurement value in the physical encoding layer starts from the correlation analysis. The terahertz single-pixel measurement value, as Figure 2 the one-dimensional measurement vector in, that is, the measurement value, is subjected to correlation analysis with the measurement matrix to obtain a terahertz wave image matrix, and then the terahertz wave image matrix is input into the deep neural network model for feature extraction and image reconstruction, so as to obtain a terahertz wave reconstructed image.

[0075] Regarding the processing steps of the model in the physical encoding layer during one training process of the image reconstruction model, an explanation is given:

[0076] When training the image reconstruction model, input the sample image in the dataset into the physical encoding layer, denoted as , and then encode the sample image through the measurement matrix to obtain a one-dimensional measurement vector:

[0077]

[0078] In the formula, is the nth measurement value in the one-dimensional measurement vector, * is the convolution operator, is the nth measurement matrix, is the sample image, and x and y represent the image space coordinates.

[0079] In this embodiment, the measurement matrix is in the form of two mutually opposite measurement matrices to reduce the noise influence in the measurement, and its formula is expressed as:

[0080]

[0081] In the formula, represents the positive mask matrix of the nth measurement matrix, represents the anti-mask matrix of the nth measurement matrix.

[0082] In a preferred embodiment, the present invention uses a Hadamard basis matrix composed of 1 and -1 as the measurement matrix, and its mathematical matrix in the form of second-order difference is:

[0083]

[0084] Of course, other matrices such as a random Gaussian matrix, a Fourier basis matrix, and a deep learning basis matrix can also be used as the measurement matrix, which is not overly limited here.

[0085] Then perform correlation analysis operations between the measurement values in the one-dimensional measurement vector, that is, Figure 2 and the measurement matrix to obtain a sub-optimal terahertz wave image matrix. In order to distinguish it from the terahertz wave image matrix output by the physical coding layer in the actual reconstruction process, the terahertz wave image matrix output during the training process is used as the first terahertz wave image matrix, and the mathematical transfer function of this process can be expressed as:

[0086]

[0087] In the formula, N is the number of measurement matrices, is the mean operation, is the first terahertz wave image matrix, and it is used as the physically guided physical coding layer in this embodiment. The correlation analysis process of this embodiment can be any one of ghost imaging, differential ghost imaging, normalized ghost imaging, Hadamard single-pixel imaging, Fourier single-pixel imaging, or compressive sensing single-pixel imaging based on iterative operations.

[0088] In this embodiment, the physical coding layer guides the terahertz wave image reconstruction process, providing interpretability and physical image attributes for the subsequent deep neural network model, thereby improving the reconstruction quality and efficiency of the terahertz wave image.

[0089] For the terahertz wave image matrix output by the physical coding layer, it will be input into the deep neural network model for further feature extraction and image reconstruction. In this embodiment, the deep neural network model adopts an encoder-decoder structure based on the attention gating mechanism, including a downsampling encoding layer, an upsampling decoding layer, and a multi-scale feature aggregation layer. The output of the physical coding layer is input into the downsampling encoding layer. In this embodiment, there are multiple downsampling encoding layers with different parameters, that is, multiple convolutional pooling layers with different parameters. Different-scale feature maps are extracted through multiple downsampling encoding layers, and these feature maps of different sizes are spliced through multi-scale skip connections and transmitted to the upsampling decoding layer. Each upsampling encoding layer fuses the feature maps of smaller scales and the same scale, thereby obtaining a multi-scale terahertz wave aggregated feature map.

[0090] Taking Figure 2 the shown model structure as an example, a simple description of the multi-scale skip connection is given. Assume that the downsampling encoding layers from top to bottom are X En1 to X En5 , and the upsampling decoding layers from top to bottom are X De1 to X De5 . Among them, X En5 and X De5 are represented by the same layer. Taking the decoder X De3 as an example, the sources of its feature maps are divided into three parts. The first part is the small-scale feature maps in the encoder, that is, the feature maps of X De1 and X De2 . The size of the feature map can be reduced through non-overlapping max pooling operations, and the number of channels of the feature map can be changed through convolutional operations. The second part is the same-scale feature maps in the encoder, that is, the feature maps of X De3 . The third part is the large-scale feature maps in the decoder, that is, the feature maps of the lowest dimension X De5 in the figure. The size of the feature map is enlarged through upsampling operations, that is, the bilinear interpolation method, and the size and number of channels of the feature map are changed through convolutional operations. Finally, four feature maps with the same number of channels are obtained. The X De4 of the next scale is used as the gating signal for attention gate control. The size of the feature map is enlarged through upsampling operations, that is, the bilinear interpolation method, and the size and number of channels of the feature map are changed through convolutional operations. Finally, through feature aggregation, the feature map of X En3 is obtained, thus realizing multi-size feature fusion. This combination of rough-level and fine-level dense connections in this embodiment improves the extraction ability of global and local features.

[0091] In this embodiment, a corresponding multi-scale feature aggregation layer is also provided. Each multi-scale feature aggregation layer adopts deep supervision to learn hierarchical representations from the multi-scale terahertz wave aggregation feature maps. By aggregating the feature maps input to the upsampling decoding layer, a reconstructed image is obtained. In this embodiment, the deep supervision is preferably set as a 3×3 convolutional layer, which is equivalent to supervising the output of each multi-scale feature aggregation layer. After introducing deep supervision, each sub-network of the model, namely L1~L4, can output images separately. After training, only the output of the multi-scale feature aggregation layer L1 needs to be used as the finally reconstructed terahertz wave image, thereby reducing the model structure and improving the output efficiency of the model. By introducing deep supervision, the generalization ability of the deep neural network in this embodiment is enhanced, guiding the more accurate reconstruction of the terahertz wave image. It should be noted that the specific steps of the multi-scale skip connection in this embodiment can refer to the conventional multi-scale skip connection and will not be repeated here.

[0092] In a preferred embodiment, this embodiment also uses attention gate control to focus on the feature information related to the terahertz wave image reconstruction, thereby suppressing the feature responses in the irrelevant background regions in the multi-scale feature maps. Among them, the attention gate control is set between the upsampling decoding layer and the multi-scale feature aggregation layer, and the attention mechanism formula adopted can be expressed as:

[0093]

[0094] In the formula, F represents the attention weight, x represents the terahertz wave aggregation feature map input from the upsampling decoding layer of this layer, g represents the gating signal input from the multi-scale feature aggregation layer of the next layer, represents the matrix element addition, represents the matrix element multiplication, represents a one-dimensional channel attention module, represents a two-dimensional spatial attention module.

[0095] It should be noted here that since there is one less multi-scale feature aggregation layer than the upsampling decoding layer, the gating signal input in each attention gate control is from the multi-scale feature aggregation layer of the next layer.

[0096] When training the image reconstruction model, this embodiment preferably uses the Adam optimizer for optimization, and takes the minimum value of the root mean square error between the output of each multi-scale feature aggregation layer and the sample image as the loss function, and its formula is expressed as:

[0097]

[0098] In the formula, represents the loss function of the i-th multi-scale feature aggregation layer, denotes the output of the $i$-th multi-scale feature aggregation layer, denotes the sample image, $x$ and $y$ denote the image spatial coordinates, and $W$ denotes the weight parameter.

[0099] Meanwhile, the training process of the model is expressed as multi-task regression, and a multi-task learning strategy is used to balance these weights. The weighted sum of the loss functions of the four multi-scale feature aggregation layers yields the final loss function, which is represented by the following formula:

[0100]

[0101] In the formula, denotes the loss function of the deep neural network model, denotes the weight parameter of the $i$-th loss function, denotes the noise term.

[0102] In the image reconstruction model, the undersampled terahertz wave image output by each multi-scale feature aggregation layer and the sample input serve as the mapping sample data pair. Through the continuous convergence of the above loss function network, the proposed network parameters are optimized and updated. The loss function tends to converge and be minimized, obtaining the trained model, and the output of the multi-scale feature aggregation layer is used as the finally reconstructed terahertz wave image. To prevent overfitting due to excessive iteration times and ensure the stability of training, preferably, the maximum iteration times preset can also be used as the iteration stop condition for model training.

[0103] When using the trained image reconstruction model to reconstruct the terahertz wave single-pixel measurement values, the processing steps of the model include:

[0104] Input the terahertz wave single-pixel measurement values into the physical coding layer, perform correlation analysis operations on the terahertz wave single-pixel measurement values and the measurement matrix, and obtain the second terahertz wave image matrix;

[0105] Input the second terahertz wave image matrix into the deep neural network model, perform feature extraction and image reconstruction on the second terahertz wave image matrix, and output the terahertz wave reconstruction image corresponding to the terahertz wave single-pixel measurement values.

[0106] When using the trained image reconstruction model for image reconstruction, the input single-pixel terahertz wave measurement value can be regarded as the one-dimensional measurement vector obtained by encoding the sample image during the training process of the above model. At this time, the single-pixel terahertz wave measurement value is subjected to correlation analysis operation with the measurement matrix, and the generated second terahertz wave image matrix is input into the deep neural network model for feature extraction and image reconstruction, and then the terahertz wave reconstruction image corresponding to the single-pixel terahertz wave measurement value can be obtained. The specific processing steps in the actual reconstruction process can refer to the processing steps in the above training process, which will not be elaborated here one by one.

[0107] In this embodiment, a physics-guided deep neural network model is constructed, and the terahertz wave image reconstruction process is guided through correlation analysis operation, providing interpretability and physical image attributes for the imaging model. Through the physics-guided physical encoding layer, an accurate image reconstruction prior is learned in a data-driven manner, thus possessing a powerful global feature and local feature modeling ability. The attention gate mechanism is adopted in the deep neural network model to focus on the feature information related to image reconstruction and effectively suppress the feature responses in the irrelevant background regions of the multi-scale feature maps. At the same time, multi-scale deep supervision is adopted in the output of each multi-scale feature aggregation layer to learn hierarchical representations from the aggregated feature maps, enhancing the network generalization ability, ensuring the accuracy of the imaging results, and effectively improving the terahertz wave image reconstruction quality under undersampling conditions. After the model training is completed, high-quality images can be directly reconstructed from the undersampled one-dimensional terahertz wave measurements, avoiding time-consuming iterative optimization steps. Thus, the efficiency and quality of terahertz wave single-pixel image reconstruction are effectively improved.

[0108] In practical applications, according to the needs of the terahertz wave imaging task, 10,000 pictures in the STL10 image dataset are used in this embodiment to train and verify the model. All the images are converted into grayscale images, and the resolution is adjusted to 64×64 to adapt to the terahertz wave image with a reconstruction pixel resolution of 64×64. The image reconstruction inference time of the trained model on an RTX 3060 graphics processor is 20 milliseconds per frame, meeting the requirement of at least about 41 milliseconds per frame for video imaging speed. High-quality terahertz wave images with a pixel resolution of 64×64 can also be reconstructed under the condition that the number of measurements is 128. The image reconstruction model provided by the present invention not only reduces the dependence on high-performance computing resources but also reduces the energy consumption of the system, and can operate efficiently in an environment with limited computing and storage resources.

[0109] In order to further improve the rate of real-time terahertz wave single-pixel imaging, based on the real-time terahertz wave single-pixel imaging method based on the image reconstruction model provided by the present invention, a second embodiment of the present invention proposes a real-time terahertz wave single-pixel imaging device. Please refer to Figure 3, this device includes: a terahertz wave source module 20, a terahertz wave spatial light modulation module 30, and a terahertz wave detection module 40. The object to be imaged 10 is placed on the optical path between the terahertz wave source module 20 and the terahertz wave spatial light modulation module 30. Among them, the terahertz wave spatial light modulation module 30 includes a pump laser 31, a beam expander lens 32, a digital micromirror device 33, a projection lens 34, a diaphragm 35, and a terahertz wave modulator 36.

[0110] This device is used to modulate the object to be imaged to generate terahertz wave single-pixel measurement values. Specifically, after the terahertz wave interacts with the object to be imaged, the terahertz wave carrying the information of the object to be imaged is encoded, and the amplitude of the modulated light field is collected by a single-pixel terahertz wave detector.

[0111] In a preferred embodiment, the terahertz wave detection module 40 includes a converging parabolic mirror group 41, a terahertz wave detector 42, and a data acquisition module 43. The imaging optical path of this device will be described based on the above structure. In this embodiment, the terahertz wave source module 20 is used to output a continuous terahertz wave. The terahertz wave source module 20 is adjacent to the object to be imaged module 10. The continuous terahertz wave interacts with the object to be imaged in a transmission or reflection manner, generating a terahertz wave beam carrying the information of the object to be imaged; the terahertz wave spatial light modulation module 30 is adjacent to the object to be imaged module 10. The pump laser 31 therein is used to generate a pump laser beam. After passing through the beam expander lens 32, the pump laser beam is collimated and expanded to completely cover the mirror surface of the digital micromirror device 33. The digital micromirror device 33 is used to receive the expanded pump laser beam. The pump laser beam is modulated according to the preset speckle pattern of the measurement matrix and then transmitted to the terahertz wave modulator 36 through the projection lens 34 and the diaphragm 35; at the same time, the terahertz wave modulator 36 is also used to receive the terahertz wave beam carrying the information of the object to be imaged and modulate the terahertz wave beam based on the pump laser beam. The modulated terahertz wave beam is transmitted to the terahertz wave detection module 40. The converging parabolic mirror group 41 in the terahertz wave detection module 40 is used to converge the modulated terahertz wave beam onto the terahertz wave detector 42. The terahertz wave detector 42 is a single-pixel detector that generates a photoelectric signal under the excitation of the terahertz wave; the data acquisition module 43 is connected to the terahertz wave detector 42, converts the electrical signal of the detector into a digital signal available for the image reconstruction model, and is used to realize real-time terahertz wave single-pixel imaging of the object to be imaged 10 according to the data signal.

[0112] Please refer to Figure 4, in a preferred embodiment, the terahertz wave source module 20 outputs continuous terahertz waves through a micro-signal generator and a terahertz wave frequency doubling module; the pump laser 31 selects a 532 nm continuous wave pump laser beam to generate electron-hole pairs of the terahertz wave modulator 36, so as to achieve the purpose of modulating the terahertz beam. The terahertz wave modulator 36 may include passivated silicon, passivated germanium, single crystal germanium wafers, high-resistance silicon wafers, sapphire or vanadium dioxide, and can be selected according to the actual situation during specific implementation, and will not be overly limited here. The digital micromirror device 33 selects a DMD spatial light modulator composed of 768×1024 small mirrors of 13.68 microns × 13.68 microns. The digital micromirror device 33 is pre-loaded by a computer with a series of speckle images generated according to the measurement matrices selected by the image reconstruction model, and modulates the pump laser beam through the speckle images. The modulated pump laser beam is projected onto the terahertz wave modulator through a projection lens and a diaphragm (not shown). Among them, the converging paraboloid mirror 41 group includes a first off-axis paraboloid mirror and a second off-axis paraboloid mirror. The first off-axis paraboloid mirror is arranged between the digital micromirror device 33 and the terahertz wave modulator 36. The modulated pump laser beam is projected onto the terahertz wave modulator 36 through the small hole in the middle of the first off-axis paraboloid mirror. The modulated terahertz wave beam generated by the terahertz wave modulator 36 is reflected by the first off-axis paraboloid mirror and the second off-axis paraboloid mirror and is reflected to the terahertz wave single-pixel detector. A rising edge trigger signal will be output during each projection process of the DMD to synchronously trigger the data acquisition module 43 to collect data. The data acquisition module 43 selects an ADC data acquisition card with a collection bit width of 16 bits, a maximum sampling rate of 2 million times per second, and an accuracy of 1.66 mV.

[0113] Since the highest flip frequency at which the digital micromirror device 33 can work stably can reach 20 kHz, and the modulation speed of the terahertz wave modulator 36 is limited by its own material properties and the switching speed of the DMD. To make the terahertz wave modulator 36 work at 20 kHz, this requires that data acquisition and transfer must be completed within 50 microseconds. The response time of the terahertz wave detector 42 must be less than 50 microseconds. Therefore, in this embodiment, the terahertz wave detector 42 is preferably selected as an AlGaN / GaN high electron mobility transistor with a response time of 0.2 microseconds. The detector integrates a photoelectric conversion function inside and can convert the terahertz light intensity information into an analog voltage output. After the number of projections of the speckle pattern in the digital micromirror device 33 reaches a set projection cycle threshold, a frame of terahertz wave image of the object to be imaged can be realized by starting the image reconstruction model through the computer.

[0114] The real-time terahertz wave single-pixel imaging device proposed in this embodiment has the characteristics of simple structure and low cost. The terahertz continuous source adopted by this device has advantages such as high power density, high integration, and small volume. The device integrates an AlGaN / GaN high electron mobility transistor with a response time of 0.2 microseconds, and uses a terahertz wave modulator that operates under the pumping of a 532nm continuous wave pump laser to achieve a switching speed of 20 kHz. The real-time terahertz wave single-pixel imaging method proposed by the present invention, combined with a device for modulating terahertz waves for the object to be imaged, can realize the reconstruction of terahertz wave single-pixel images at the video speed level, while ensuring the high quality of the reconstructed images.

[0115] Figure 5 The terahertz single-pixel imaging results with a pixel resolution of 64×64 based on the method and device provided by the present invention are shown under different numbers of measurements and different terahertz modulator switching speeds. The experimental results show that with the device provided by the present invention at a switching speed of 20 kHz, a terahertz wave image with a pixel resolution of 64×64 is reconstructed using 128 single-pixel measurements of terahertz waves. At this time, the acquisition time is 6.4 milliseconds, that is, the recording speed of the terahertz wave image is 156 frames per second. The real-time terahertz wave single-pixel imaging method proposed by the present invention requires a reconstruction time of 20 milliseconds. In order to obtain the best balance between reconstruction quality and acquisition time, the present invention finally selects a switching speed of 15.625 kHz and a number of single-pixel measurements of 512. At this time, the acquisition time of the device is about 33 milliseconds, that is, the recording speed of the terahertz wave image is 30 frames per second, which matches the reconstruction time of the real-time terahertz wave single-pixel imaging method.

[0116] Please refer to Figure 6 , based on the same inventive concept, a real-time terahertz wave single-pixel imaging system proposed in the third embodiment of the present invention includes:

[0117] A data acquisition module 1 for acquiring the terahertz wave single-pixel measurement values of the object to be imaged, where the terahertz wave single-pixel measurement values are one-dimensional signals obtained by measuring the object to be imaged with a terahertz wave single-pixel detector;

[0118] An image reconstruction module 2 for inputting the terahertz wave single-pixel measurement values into a pre-constructed image reconstruction model to obtain a terahertz wave reconstructed image of the object to be imaged;

[0119] Wherein, the image reconstruction model includes a physical coding layer and a deep neural network model. The physical coding layer is used to associate the terahertz wave single-pixel measurement values with a measurement matrix to generate a terahertz wave image matrix and input it into the deep neural network model;

[0120] The deep neural network model adopts an encoder-decoder structure based on an attention gating mechanism, which is used to extract features and reconstruct images from the terahertz wave image matrix, and output the reconstructed terahertz wave image.

[0121] The technical features and technical effects of the real-time terahertz wave single-pixel imaging system proposed in the embodiments of the present invention are the same as those of the method proposed in the embodiments of the present invention, and will not be elaborated here. Each module in the above real-time terahertz wave single-pixel imaging system can be implemented in whole or in part by software, hardware, and their combinations. The above-mentioned modules can be embedded in the processor in the computer device in hardware form or be independent of it, or can be stored in the memory in the computer device in software form, so that the processor can call and execute the operations corresponding to the above modules.

[0122] In addition, an embodiment of the present invention also proposes a computer device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the steps of the above method are implemented.

[0123] Please refer to Figure 7 , the internal structure diagram of the computer device in one embodiment. The computer device may specifically be a terminal or a server. The computer device includes a processor, a memory, a network interface, a display, and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the real-time terahertz wave single-pixel imaging method is implemented. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device may be a touch layer covering the display screen, or a button, a trackball, or a touchpad provided on the housing of the computer device, or an external keyboard, a touchpad, or a mouse, etc.

[0124] Those of ordinary skill in the art can understand that Figure 7 the structure shown in

[0125] In summary, a real-time terahertz wave single-pixel imaging method, apparatus, system, and computer device proposed in an embodiment of the present invention. The method includes obtaining a terahertz wave single-pixel measurement value of an object to be imaged, where the terahertz wave single-pixel measurement value is a one-dimensional signal obtained by measuring the object to be imaged with a terahertz wave single-pixel detector; inputting the terahertz wave single-pixel measurement value into a pre-constructed image reconstruction model to obtain a terahertz wave reconstructed image of the object to be imaged; where the image reconstruction model includes a physical encoding layer and a deep neural network model, and the physical encoding layer is used to associate the terahertz wave single-pixel measurement value with a measurement matrix, generate a terahertz wave image matrix, and input it into the deep neural network model; the deep neural network model adopts an encoder-decoder structure based on an attention gating mechanism, and is used to extract features and reconstruct images from the terahertz wave image matrix, and output the terahertz wave reconstructed image. The present invention constructs a deep neural network model guided by physics, and guides the terahertz wave image reconstruction process through correlation analysis operations, providing interpretability and physical image attributes for the imaging model. Through the physically guided physical encoding layer, an accurate image reconstruction prior is learned in a data-driven manner, thereby having a strong global feature and local feature modeling ability. An attention gate mechanism is adopted in the deep neural network model to focus on the feature information related to image reconstruction, and effectively suppress the feature responses in the irrelevant background regions of the multi-scale feature maps. At the same time, multi-scale deep supervision is adopted in the output of each multi-scale feature aggregation layer to learn hierarchical representations from the aggregated feature maps, enhancing the network generalization ability, ensuring the accuracy of the imaging results, and effectively improving the terahertz wave image reconstruction quality under undersampling conditions. The present invention also provides a real-time terahertz wave single-pixel imaging device with a high-speed switch, effectively balancing the relationship between imaging speed and imaging quality, and further improving the reconstruction time and reconstruction quality of real-time terahertz wave single-pixel imaging.

[0126] Each embodiment in this specification is described in a progressive manner. For parts that are the same or similar in each embodiment, reference can be made to each other. The key point of each embodiment is to illustrate the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can refer to the partial description of the method embodiment. It should be noted that the technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered that the scope described in this specification.

[0127] The above-described embodiments merely represent several preferred embodiments of the present application. The description thereof is relatively specific and detailed, but it should not be construed as a limitation on the scope of the invention patent. It should be noted that for those of ordinary skill in the art, without departing from the technical principle of the present invention, several improvements and substitutions can be made, and these improvements and substitutions should also be regarded as the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the protection scope of the claims described above.

Claims

1. A real-time terahertz wave single-pixel imaging method, characterized in that, Including: Obtain the terahertz wave single-pixel measurement value of the object to be imaged, where the terahertz wave single-pixel measurement value is a one-dimensional signal obtained by measuring the object to be imaged with a terahertz wave single-pixel detector; Input the terahertz wave single-pixel measurement value into a pre-constructed image reconstruction model to obtain the terahertz wave reconstruction image of the object to be imaged; Wherein, the image reconstruction model includes a physical encoding layer and a deep neural network model. The physical encoding layer is used to associate the terahertz wave single-pixel measurement value with a measurement matrix, generate a terahertz wave image matrix, and input it into the deep neural network model; The deep neural network model adopts an encoder-decoder structure based on an attention gating mechanism, which is used to extract features and reconstruct the image from the terahertz wave image matrix, and output the terahertz wave reconstruction image; Wherein, the deep neural network model includes a downsampling encoding layer, an upsampling decoding layer, and a multi-scale feature aggregation layer; The downsampling encoding layer is used to extract feature maps of different scales, splice the feature maps of different scales through multi-scale skip connections, and transmit them to the upsampling decoding layer; The upsampling decoding layer is used to perform feature fusion on the feature maps of different scales to obtain a terahertz wave aggregated feature map, and transmit it to the multi-scale feature aggregation layer; The multi-scale feature aggregation layer adopts multi-scale deep supervision and is used to learn hierarchical representations from the terahertz wave aggregated feature map; An attention gate control is set between the upsampling decoding layer and the multi-scale feature aggregation layer. The attention gate control is used to generate an attention weight according to the terahertz wave aggregated feature map input by the upsampling decoding layer and the gating signal input by the multi-scale feature aggregation layer, and input it into the multi-scale feature aggregation layer; Wherein, the following formula is used to represent the attention weight: Wherein, F represents the attention weight, x represents the terahertz wave aggregation feature map input from the upsampling decoding layer of this layer, g represents the gating signal input from the multi-scale feature aggregation layer of the next layer, represents the addition of matrix elements, represents the multiplication of matrix elements, represents a one-dimensional channel attention module, represents a two-dimensional spatial attention module.

2. The real-time terahertz wave single-pixel imaging method according to claim 1, wherein The steps for training the image reconstruction model include: Input the sample image into the physical encoding layer, encode the sample image through the measurement matrix to obtain a one-dimensional measurement vector, and perform correlation analysis operations on the one-dimensional measurement vector and the measurement matrix to generate a first terahertz wave image matrix; Input the first terahertz wave image matrix into the deep neural network model, perform feature extraction and image reconstruction on the first terahertz wave image matrix, and output a first terahertz wave reconstruction image; Iteratively train the image reconstruction model according to the above steps until the iteration stop condition is reached to obtain the trained image reconstruction model. The iteration stop condition includes the convergence of the loss function or reaching the maximum number of iterations; Wherein, the following formula is used to represent the one-dimensional measurement vector: In the formula, is the nth measurement value in the one-dimensional measurement vector, * is the convolution operator, is the nth measurement matrix, is the sample image, and x and y represent the image space coordinates; The following formula is used to represent the first terahertz wave image matrix: In the formula, is the first terahertz wave image matrix, N is the number of measurement matrices, is the mean operation; The steps for performing image reconstruction using the trained image reconstruction model include: Input the terahertz wave single-pixel measurement value into the physical encoding layer, perform correlation analysis operations on the terahertz wave single-pixel measurement value and the measurement matrix to obtain a second terahertz wave image matrix; Input the second terahertz wave image matrix into the deep neural network model, perform feature extraction and image reconstruction on the second terahertz wave image matrix, and output the terahertz wave reconstructed image corresponding to the terahertz wave single-pixel measurement value.

3. The real-time terahertz wave single-pixel imaging method according to claim 1, wherein The loss function of the multi-scale feature aggregation layer is the minimum value of the root mean square difference between the output of the multi-scale feature aggregation layer and the terahertz wave single-pixel measurement value, and the loss function of the deep neural network model is the weighted sum value of the loss functions of each multi-scale feature aggregation layer.

4. The real-time terahertz wave single-pixel imaging method according to claim 3, characterized in that, The loss function of the multi-scale feature aggregation layer is expressed by the following formula: The loss function of the deep neural network model is expressed by the following formula: In the formula, represents the loss function of the i-th multi-scale feature aggregation layer, represents the output of the i-th multi-scale feature aggregation layer, represents the sample image, x and y represent the image spatial coordinates, W represents the weight parameter, represents the loss function of the deep neural network model, represents the weight parameter of the i-th loss function, represents the noise term.

5. A real-time terahertz wave single-pixel imaging system, characterized in that, The system is applied to the method according to any one of claims 1 to 4. The system includes: A data acquisition module, configured to acquire the terahertz wave single-pixel measurement value of the object to be imaged, where the terahertz wave single-pixel measurement value is a one-dimensional signal obtained by measuring the object to be imaged with a terahertz wave single-pixel detector; An image reconstruction module, configured to input the terahertz wave single-pixel measurement value into a pre-constructed image reconstruction model to obtain the terahertz wave reconstructed image of the object to be imaged; Wherein, the image reconstruction model includes a physical encoding layer and a deep neural network model. The physical encoding layer is configured to associate the terahertz wave single-pixel measurement value with a measurement matrix, generate a terahertz wave image matrix, and input it into the deep neural network model; The deep neural network model adopts an encoder-decoder structure based on an attention gating mechanism, and is configured to perform feature extraction and image reconstruction on the terahertz wave image matrix, and output the terahertz wave reconstructed image.

6. A computer device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 4.

Images