Computational ghost imaging method based on conditional generative adversarial network under low sampling rate

Abstract

This invention discloses a computational ghost imaging method based on conditional generative adversarial networks (GANs) for low sampling rates. This method can extract internal features from images in harsh environments or at low sampling rates. By further improving and optimizing traditional autoencoder algorithms, it effectively solves the problems of image quality and speed in reconstructed images under low sampling conditions. Compared with traditional CGI, CSCGI, CNN-CGI, DCAN-CGI, and DGI methods, this invention can reconstruct target images faster at low sampling rates, and significantly improves both PSNR and SSIM, effectively filtering out background speckle. Practical physical experiments further verify the feasibility of this method. While ensuring the quality of the reconstructed image, the high efficiency of GPU multi-core parallel computing can significantly shorten both the training time of the neural network and the target object reconstruction time, which is of great significance for engineering applications.

Description

Technical Field

[0001] This invention relates to a computational ghost imaging method based on conditional generative adversarial networks (GANs) under low sampling rates, and to a method for optimizing target images reconstructed from computational ghost imaging using GAN algorithms. Background Technology

[0002] In traditional ghost imaging, since the distribution of the light field cannot be predicted artificially, a high-resolution camera is needed to directly receive the light field distribution as a reference light path in order to obtain the light field intensity fluctuations, but this will increase the detection time.

[0003] Compared to previous methods, computational ghost imaging uses a spatial light modulator (SLM) or digital micromirror devices (DMDs) to pre-determine the changes in the light field distribution before the light illuminates the object, i.e., the known light field intensity fluctuation relationship. This simplifies the original two-beam path into a single beam path, eliminating the need for a reference beam splitter. The more times the bucket detector collects light intensity within the same time frame, the significantly improved the quality of the reconstructed image. In recent years, compressed sensing computational imaging (CSCGI) has been applied to ghost imaging reconstruction. Although compressed sensing (CS) has broad application prospects in computational ghost imaging, it still faces two major problems. First, CS algorithms require prior knowledge of the target image to reconstruct the image from some samples; furthermore, the image is not sparse and fixed, limiting its practical application. Second, most high-performance CS algorithms have high computational requirements, increasing image reconstruction time and making them unsuitable for low sampling rates.

[0004] In this context, the rise of deep learning (DL) has proven to be a powerful force in solving the complex technical problems of computational ghost imaging. DL has the potential to significantly improve the performance of real-time applications in computational ghost imaging; moreover, it is a technique for data modeling and decision-making using neural networks trained on large amounts of data. However, current DL-based computational ghost imaging methods primarily aim to improve the signal-to-noise ratio of the entire image, but at low sampling rates, both imaging quality and speed remain unsatisfactory. Summary of the Invention

[0005] The purpose of this invention is to provide a computational ghost imaging (CGI) method and system based on Conditional Generative Adversarial Network (CGAN), which effectively solves the imaging quality problem of reconstructed images under low sampling rates.

[0006] To achieve the above objectives, the present invention provides the following solution: a computational ghost imaging method based on conditional generative adversarial networks at low sampling rates, comprising the following steps:

[0007] Step 1: Obtain the dataset;

[0008] Step 2: Reconstruct the dataset to obtain the reconstructed dataset; the reconstructed dataset includes a training set, a test set, and a validation set;

[0009] Step 3: Input the training set into CGAN, compress the hidden layer through forward convolution in the generator and deconvolution in the discriminator, and decompress it in the output layer to obtain the actual output result;

[0010] Step four: Calculate the error between the actual output and the ideal output using the backpropagation algorithm;

[0011] Step 5: Using the actual output results and the original image corresponding to the reconstructed dataset, the error is reduced by propagating the error through the backpropagation algorithm to obtain the CGANCGI model;

[0012] Step 6: Using the CGANCGI model, images from a portion of the test set in the reconstructed dataset are input into CGAN for prediction, resulting in predicted CGI images, which are then validated using a validation set.

[0013] Furthermore, in step one, a dataset is acquired through an imaging system, which includes a Light Crafter4500, an attenuator, an object under test, an optical lens, and a CMOS camera. The object under test, the optical lens, and the CMOS camera are placed on the same optical axis. The speckle pattern is projected sequentially through the DLP4500, then sequentially through the attenuator and the object under test. Finally, the CMOS camera is used instead of the barrel detector to collect the total light intensity of the object.

[0014] Furthermore, in step two, the dataset is reconstructed to obtain the reconstructed dataset, which specifically includes:

[0015] Step 21: Randomly select M training set images and N test set images from a dataset with diversity, and increase their resolution to amplify the feature information;

[0016] Step 22: Treat the images in the dataset as the objects to be measured, and modulate the light field containing object information into a binary random matrix using DLP4500.

[0017] Step 23: Use a photodetector to detect the total light intensity data transmitted through the object, and finally use the correlation calculation formula to directly reconstruct the image to form a new image dataset, thus obtaining the reconstructed dataset.

[0018] Furthermore, the correlation calculation formula in step 23 is as follows:

[0019] Based on the second-order correlation calculation principle: by calculating the random fluctuations in the light field intensity and the light intensity information from the bucket detector, the object's information is obtained. Since the modulation of the DMD is controllable, the light field intensity distribution I(x,y) acting on the object can be calculated according to the Huygens-Fresnel principle. Ideally, the actual light field intensity distribution is assumed to be equal to the calculated data. Then, the light intensity value obtained by the bucket detector is:

[0020] S i =∫I i (x,y)T(x,y)dxdy (1)

[0021] In the formula: T(x,y) represents the transmittance function of the object, I i (x,y) represents the light field intensity distribution matrix of the reference light, and (x,y) represents the coordinate position;

[0022] The algorithm for calculating the second-order correlation of correlated imaging is as follows:

[0023]

[0024] Where <> represents the average of N measurements, S i The sum of light intensity values ​​measured by the bucket detector in the i-th iteration can be understood as the sum of modulated light intensity information. Ideally, the final CGI image G can be reconstructed from formulas (1) and (2). CGI (x,y).

[0025] Furthermore, the network structure of the generator in the CGANCGI model is as follows:

[0026] The generator network model employs a U-shaped network structure, reconstructing the original image through an encoder consisting of a zero-padding layer and four downsampling convolutional modules, and a decoder consisting of four upsampling convolutional modules. The downsampling convolutional module in the encoder comprises convolutional layers, a normalization layer (BN), and an activation function layer; the upsampling convolutional module in the decoder also applies a dropout layer to prevent overfitting and achieve better reconstruction results. First, an image of a certain size is taken as input and enlarged by a zero-padding layer. Then, the image is continuously compressed by the downsampling convolutional modules to increase the number of image channels and extract image features. Next, the obtained feature maps are gradually restored to their spatial resolution and the output image size is increased by the upsampling convolutional modules. Finally, a convolutional layer restores the enlarged image to the size of the input image.

[0027] The activation function layer is Leaky ReLU.

[0028] Furthermore, the network structure of the discriminator in the CGANCGI model is as follows:

[0029] The overall network structure adopts a CNN convolutional network. The first four parts of the network consist of convolutional modules that alternate between convolutional layers, normalization layers, and activation function layers. Finally, a fully connected layer maps the output. In each convolutional module, the convolutional layer first performs feature extraction at different scales; then a normalization layer is added to accelerate the network's feature mapping capability and can also act as a regularizer; finally, the Leaky ReLU activation function is used to prevent the gradient vanishing problem during training.

[0030] Furthermore, the specific implementation method of step five is as follows:

[0031] First, the parameters from the generator are updated to the discriminator. The initialized image is input into the generator, and then the generator gradient is set to 0. The generator generates samples, which are input into the discriminator to evaluate the loss. Then, the gradient is calculated in reverse to update the generator parameters. The updated generator parameters are put into the discriminator to zero the gradient. Then, the loss of the real samples and generated samples is calculated to reduce the difference between the detailed information of the tested object and the overall image. The discriminator parameters are updated in reverse to calculate the gradient.

[0032] Then, the image output by the generator is input into the discriminator, and the parameters of the generator are adjusted according to the discriminator so that the generated result matches the distribution recognized by the discriminator.

[0033] Repeat the above two steps to balance the training process.

[0034] Compared with the prior art, the present invention has the following technical advantages:

[0035] 1. This invention can extract internal features of images under harsh environments or low sampling rates. By further improving and optimizing the traditional autoencoder algorithm, it can effectively solve the problems of imaging quality and imaging speed of reconstructed images under low sampling conditions.

[0036] 2. Compared with traditional CGI, CSCGI, CNN-CGI, DCAN-CGI and DGI methods, this invention can reconstruct the target image faster at low sampling rates, and the PSNR and SSIM of the image are significantly improved. It can effectively filter out background speckle. The feasibility of this method has been further verified through actual physical experiments.

[0037] 3. While ensuring the quality of the reconstructed image, the high efficiency of GPU multi-core parallel computing can be used to significantly shorten the training time of the neural network and the reconstruction time of the target object, which is of great significance for engineering applications. Attached Figure Description

[0038] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0039] Figure 1 This is a flowchart of the computational ghost imaging method based on conditional generative adversarial networks according to an embodiment of the present invention;

[0040] Figure 2 This is a comparison chart of the actual experimental effects of different methods in the embodiments of the present invention;

[0041] Figure 3 This is a schematic diagram of the network structure of the generator and discriminator in an embodiment of the present invention;

[0042] Figure 4 The figures show the PSNR and SSIM curves of five methods at different frequencies in this embodiment of the invention.

[0043] Figure 5 This is a schematic diagram of the physical optical path of the computational ghost imaging system based on conditional generative adversarial networks, according to an embodiment of the present invention. Detailed Implementation

[0044] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0045] like Figure 1 As shown, the present invention provides a computational ghost imaging method based on conditional generative adversarial networks at low sampling rates, the method specifically including the following steps:

[0046] Step 1: Obtain the dataset;

[0047] Step 2: Reconstruct the dataset to obtain the reconstructed dataset; the reconstructed dataset includes a training set, a test set, and a validation set;

[0048] Step 3: Input the training set into CGAN, compress the hidden layer through forward convolution in the generator and deconvolution in the discriminator, and decompress it in the output layer to obtain the actual output result;

[0049] Step 4: Calculate the error between the actual output and the ideal output using the backpropagation algorithm;

[0050] Step 5: Using the actual output results and the original image corresponding to the reconstructed dataset, the error is reduced by propagating the error through the backpropagation algorithm to obtain the CGANCGI model;

[0051] Step 6: Use the CGANCGI model to input images from a portion of the test set in the reconstructed dataset into CGAN for prediction, and obtain the predicted CGI images.

[0052] Step two involves reconstructing the dataset to obtain the reconstructed dataset, which specifically includes:

[0053] Step 21: Randomly select 2000 training set images and 200 test set images from a dataset with diversity, and increase their resolution to amplify the feature information;

[0054] Step 22: Treat the images in the dataset as the objects to be measured, and modulate the light field containing object information into a binary random matrix using DLP4500.

[0055] Step 23: Use a photodetector to detect the total light intensity data transmitted through the object, and finally use the correlation calculation formula to directly reconstruct the image to form a new image dataset, thus obtaining the reconstructed dataset.

[0056] The correlation calculation formula mentioned in step 23 is as follows:

[0057] Based on the second-order correlation calculation principle: the random fluctuations in the light field intensity and the light intensity information from the bucket detector are calculated to obtain the object's information. Since the modulation of the DMD is controllable, the light field intensity distribution I(x,y) acting on the object can be calculated according to the Huygens-Fresnel principle. Ideally, the actual light field intensity distribution is assumed to be equal to the calculated data; then the light intensity value obtained by the bucket detector is:

[0058] S i =∫I i (x,y)T(x,y)dxdy (1)

[0059] In the formula: T(x,y) represents the transmittance function of the object, I i (x,y) represents the light field intensity distribution matrix of the reference light, and (x,y) represents the coordinate position.

[0060] The algorithm for calculating the second-order correlation of correlated imaging is as follows:

[0061]

[0062] Where <> represents the average of N measurements, S i Let G represent the sum of light intensity values ​​measured by the bucket detector in the i-th iteration, which can be understood as the sum of modulated light intensity information. Ideally, the final CGI image G can be reconstructed from formulas (1) and (2). CGI (x,y).

[0063] Compared to SLM, DMD obtains the desired light field by controlling the switching of each microarray. It can be directly used as a reference light intensity distribution field, and the wavelength of the incident light does not affect the modulation result of DMD. It is not limited by the type of light source and is applicable to various spectral applications, thus possessing higher practical value. When the light source illuminates the DMD, N random binary orthogonal matrices generated periodically by the randi() function in Matlab are pre-loaded onto the DMD, and these matrices are converted into N frames of binary speckle patterns. When a point in the speckle pattern is 1, the micromirrors of the DMD are "on"; when a point is 0, the micromirrors are "off," meaning the light field intensity distribution of the light source is modulated and updated N times, achieving artificial control of the light field intensity distribution. In this way, the DMD's light field intensity distribution modulation can replace the high-resolution camera in the original reference optical path. The modulated beam illuminates the object under test, and the light carrying information about the object is focused by a collecting lens onto a barrel detector. By combining the light field intensity distribution of the DMD with the received values, a second-order correlation operation is performed on a computer to obtain the image of the object.

[0064] CGANCGI is a deep learning framework in the field of unsupervised learning. It typically adds conditional information to the generator in CGAN and replaces the original data with the generated features, achieving good results. To improve the quality and speed of CGI-reconstructed images at low sampling rates, this invention will be described in further detail. The specific training process of the CGANCGI network designed in this invention is as follows:

[0065] Step 51: In real-world scenarios, there will be a large amount of uncertain measurement data. Randomly select m training set images and n test set images from different datasets X, and increase the resolution and feature information based on our experience.

[0066] Step 52: Treat the image x in the dataset as the object to be measured. The light field information of the target is modulated by a binary random matrix through DLP4500. The total light intensity of the target is detected by a photodetector. Finally, construct a new image dataset Y according to the relevant imaging formula.

[0067] Step 53: Feed the dataset Y as the training set into the generator network of CGAN for training. The input image is compressed by the convolutional layer in the generator.

[0068] Step 54: Input the reconstructed image and the original image output by the generator into the discriminator for training, or input the original image and the CGI image together into the discriminator for training to obtain the CGANCGI model. Comparing the two images with the discriminator can improve the quality of image reconstruction.

[0069] Step 55: Use the trained model to predict some test set images in dataset Y to the CGANCGI model. The output of the model is the predicted CGI image Z.

[0070] Through the above iterative training process, a generator and a discriminator can be obtained. The CGI image is input into the generator, which outputs a reconstructed image. This image is then input into the discriminator, and finally, a high-quality image with rich texture information and low background noise is output in an end-to-end manner.

[0071] The specific algorithm structure design of the CGANCGI model is as follows:

[0072] GANs are inspired by the Nash mean in game theory, which states that both sides in an adversarial game hope to maximize their expected profit in the other's game. The main network consists of a generator G and a discriminator D. The generator receives sampled random noise Z and outputs the generated image G(z), while the discriminator receives the image data and outputs the real and fake labels of the image.

[0073] The GAN architecture's countermeasures are now located in the generator and discriminator. The generator aims to generate an image G(z) that can be determined as true by the discriminator, while the discriminator aims to determine that the real image x is true and the generated image G(z) is false. The generator's goal is to learn through the network using real data to generate a sample distribution that approximates the real data, while the discriminator's main purpose is to determine whether the input data is real or generated by the generator. After a series of learning and optimization processes, the discriminator can no longer accurately identify the source of the input data, i.e., it reaches Nash equilibrium, and training ends.

[0074] The objective function of GAN is as follows:

[0075]

[0076] From the perspective of the objective function, E x～Pdata(x) [log2D(x)] represents the expected value of the actual input data in D, which needs to be maximized. E Z～PZ(Z) log2[1-D(G(Z))] represents the expected value of the input generated data in D. D wants to maximize this objective, while G wants to minimize it, which reflects the adversarial process of GAN networks. In actual training, the common approach is to first fix G to train D, and then fix D to train G.

[0077] The generator network model includes (e.g.) Figure 3 a):

[0078] Internal generator structure as follows Figure 3As shown in Figure a, the generative network model employs a U-shaped network structure, reconstructing the original image through an encoder consisting of a zero-padding layer and four downsampling convolutional modules, and a decoder consisting of four upsampling convolutional modules. The encoder's downsampling convolutional module mainly consists of convolutional layers, a normalization layer (BN), and an activation function layer. The decoder's upsampling convolutional module also applies a dropout layer to prevent overfitting and obtain better reconstruction results. First, a 28-pixel × 28-pixel image is taken as input and enlarged through a zero-padding layer. Then, the image is continuously compressed by the convolutional modules to increase the number of image channels and extract image features. Next, the obtained feature map is gradually restored to its spatial resolution and the output image size is increased through the upsampling convolutional modules. Finally, a convolutional layer restores the enlarged image to its original 28-pixel × 28-pixel size. This generator network model incorporates a skip connection mechanism, simply adding the input image to the generator's output. This concatenates the feature maps of the encoder and decoder, which are of the same size, reducing the semantic gap between them and achieving better reconstruction results. It also alleviates the vanishing gradient problem, ensuring that the result of each convolutional step is not solely dependent on the result of the previous step. Changing the activation function in the convolutional layer from Rectified Linear Unit (ReLU) to Leaky ReLU reduces the number of silenced neurons.

[0079] For generative models, the goal is to make the data generated by G(z) as similar as possible to the dataset, i.e., to have the same data distribution. Therefore, this means minimizing the error of the generative model by only passing the error generated by G(z) to the generative model.

[0080] The discriminator network model includes (e.g.) Figure 3 b):

[0081] Internal discriminator structure as follows Figure 3 As shown in b, the overall network structure adopts a CNN convolutional network. The first four parts of this network mainly consist of convolutional layers, normalization layers, and activation function layers, forming a convolutional neural network. Finally, a fully connected layer maps the output. In each convolutional module, the convolutional layer first performs feature extraction at different scales; secondly, a normalization layer is added to accelerate the network's feature mapping ability and also acts as a regularizer; finally, the Leaky ReLU activation function is used to prevent the gradient vanishing problem during training. The reconstructed image from the generator and the real image are transmitted to the discriminator, which gives a "real" or "fake" judgment result to guide the generator to output results close to the real image. In this case, the CGANCGI model is established. Finally, the test set images are input into the CGANCGI model to output high-quality images with clear texture information.

[0082] The specific physical experiments of the method of the present invention are given below.

[0083] Step 1: The imaging system was designed and built. The imaging system mainly includes a Light Crafter 4500, an attenuator, the object under test, optical lenses, and a CMOS camera, such as... Figure 5 As shown, the object under test, optical lens, and CMOS camera are placed on the same optical axis. Fifty randomly generated speckle patterns are projected sequentially through a DLP4500, then through an attenuator, and finally through the object under test. The total light intensity of the object is then collected using a CMOS camera instead of a barrel detector. The Light Crafter 4500 is used as the light source, with a projected speckle pattern resolution of 1140 pixels × 912 pixels and a wavelength range of 420-700 nm. The hardware environment of the computer used in this experiment was as follows: processor: Intel(R) Xeon(R) CPU E5-2640 v4; installed memory: 64GB; operating system: Win10, 64-bit; GPU model: NVIDIA GeForce GTX1650. The program was written in Python 3.5 and implemented using the Keras framework based on Tensorflow to create the CGANCGI model.

[0084] Step 2: Preprocess the collected data to reconstruct an initial CGI image with noise, thus forming a test set.

[0085] Using the test set images from the experiment as samples, a physical experiment was conducted. The DLP4500 projected 50-pixel × 50-pixel speckle patterns sequentially. After passing through an attenuator and the test object, a CMOS camera was used instead of a barrel detector to collect the total light intensity transmitted through the object. Due to the difference between theoretical and actual speckle patterns, the speckle field collected by the CMOS camera was used to directly reconstruct the test target by incorporating the actual speckle pattern into the correlation calculation formula (1), thus forming a new image test set.

[0086] Step 3: Train the CGANCGI model. First, the parameters from the generator are updated to the discriminator. The initialized image is input into the generator, and then the generator gradient is set to 0. The generator generates samples, which are input into the discriminator to evaluate the loss. Then, the gradient is calculated in reverse to update the generator parameters. The updated generator parameters are put into the discriminator to zero the gradient. Then, the loss of the real samples and generated samples is calculated to reduce the difference between the detailed information of the tested object and the overall image. The discriminator parameters are updated in reverse to calculate the gradient.

[0087] Then, the image output by the generator is input into the discriminator, and the parameters of the generator are adjusted according to the discriminator so that the generated result matches the distribution recognized by the discriminator. In this way, the above two steps can be repeated to balance the training process.

[0088] Step 4: Reconstruct the object image. After the CGANCGI model is trained, input the 16 initial CGI images from the test set into the CGANCGI model to complete the reconstruction of the object being tested. The reconstructed image has clear details and comprehensive overall information.

[0089] Step 5: Result Analysis. The full sampling data is defined as 784 times (28 pixels × 28 pixels). The sampling rate β refers to the ratio of the number of samples to the full sampling. A sampling rate of 0.2 represents 157 tests, and the low sampling rate range is 0 ≤ β ≤ 0.3. In the experiment, when the low sampling rate β is 0.2 and 0.08, the continuous data acquired by the camera and the binary random matrix modulated by DMD are used to reconstruct images using CGI, CSCGI, CNN-CGI, DCAN-CGI, DGI, and CGANCGI algorithms. The optical verification experimental results are as follows: Figure 2 As shown. Figure 2 The ground truth portion of the diagram represents the original image of the test target. The experimental outputs of CGI, CSCGI, DCAN-CGI, DGI, and CGANCGI in their respective rows are compared and analyzed. Practical experimental results show that CGI, CSCGI, CNN-CGI, DCAN-CGI, and DGI exhibit significant digital distortion. In particular, CGI and CSCGI struggle to directly reconstruct the shape of the numbers. Since CNNs have difficulty handling the relationship between image details and the overall picture, the CNN-CGI model heavily relies on convolution to simulate the dependencies between different image regions. Both DCAN-CGI and DGI use encoder structures, but the DGI method exhibits less background noise and easier morphological differentiation, resulting in superior image restoration quality compared to DCAN-CGI. Convolution operators have local receptive fields, meaning they can only handle long-range dependencies after multiple convolutional layers, hindering the neural network's ability to learn long-range dependencies. In contrast, the proposed CGANCGI method effectively filters background noise while reconstructing more texture information in the tested object image, resulting in stronger overall observability and greater future application prospects.

[0090] To more intuitively compare the quantitative results of CGI, CSCGI, CNN-CGI, DCAN-CGI, DGI, and CGANCGI methods, we selected the average PSNR and SSIM of all images in the test set, and plotted the average sampling rates of the six methods under different conditions into a line graph, as shown below. Figure 4As shown in the figure, the horizontal axis represents the number of training iterations, and the vertical axis represents the corresponding values ​​of these metrics. It can be seen that as the number of iterations increases, the PSNR and SSIM values ​​of the tested object image continuously increase at sampling rates β = 0.2 and 0.08. The CGI method consistently performs the worst, while our proposed method consistently performs the best. Compared to CGI, CSCGI, CNN-CGI, DCAN-CGI, and DGI, our proposed method achieves a maximum PSNR value of 27.31 and a maximum SSIM value of 0.93. Therefore, these concrete implementation experiments fully demonstrate that our method can improve the image quality of the tested object at low sampling rates.

[0091] The specific embodiments described herein are merely illustrative of the spirit of the invention. Those skilled in the art to which this invention pertains may make various modifications or additions to the described specific embodiments or use similar methods to substitute them, without departing from the spirit of the invention or exceeding the scope defined by the appended claims.

Claims

1. A computational ghost imaging method based on conditional generative adversarial networks at low sampling rates, characterized in that, Includes the following steps: Step 1: Obtain the dataset; Step 2: Reconstruct the dataset to obtain the reconstructed dataset; the reconstructed dataset includes a training set, a test set, and a validation set; Step 3: Input the training set into CGAN, compress the hidden layer through forward convolution in the generator and deconvolution in the discriminator, and decompress it in the output layer to obtain the actual output result; Step four: Calculate the error between the actual output and the ideal output using the backpropagation algorithm; Step 5: Using the actual output results and the original image corresponding to the reconstructed dataset, the error is reduced by propagating the error through the backpropagation algorithm to obtain the CGANCGI model; The network structure of the generator in the CGANCGI model is as follows: The generator network model employs a U-shaped network structure, reconstructing the original image through an encoder consisting of a zero-padding layer and four downsampling convolutional modules, and a decoder consisting of four upsampling convolutional modules. The downsampling convolutional module in the encoder comprises convolutional layers, normalization layers, and activation function layers; the upsampling convolutional module in the decoder also applies a dropout layer to prevent overfitting and achieve better reconstruction results. First, an image of a certain size is used as input and enlarged by a zero-padding layer. Then, the image is continuously compressed by the downsampling convolutional modules to increase the number of image channels and extract image features. Next, the obtained feature maps are gradually restored to their spatial resolution and the output image size is increased by the upsampling convolutional modules. Finally, a convolutional layer restores the enlarged image to the size of the input image. The activation function layer is Leaky ReLU; The network structure of the discriminator in the CGANCGI model is as follows: The overall network structure adopts a CNN convolutional network. The first four parts of the network consist of convolutional modules composed of convolutional layers, normalization layers, and activation function layers, and finally the output is mapped through a fully connected layer. In each convolutional module, the convolutional layer first performs feature extraction at different scales; then a normalization layer is added to accelerate the network's feature mapping ability and act as a regularizer; finally, the Leaky ReLU activation function is used to prevent the gradient vanishing problem during training. Step 6: Using the CGANCGI model, images from a portion of the test set in the reconstructed dataset are input into CGAN for prediction, resulting in predicted CGI images, which are then validated using a validation set.

2. The computational ghost imaging method based on conditional generative adversarial networks at low sampling rates as described in claim 1, characterized in that: In step one, a dataset is acquired through an imaging system, which includes a Light Crafter 4500, an attenuator, the object under test, an optical lens, and a CMOS camera. The object under test, the optical lens, and the CMOS camera are placed on the same optical axis. The speckle pattern is projected sequentially through the DLP4500, then sequentially through the attenuator and the object under test. Finally, the CMOS camera is used instead of the barrel detector to collect the total light intensity of the object.

3. The computational ghost imaging method based on conditional generative adversarial networks at low sampling rates as described in claim 1, characterized in that: Step two involves reconstructing the dataset to obtain the reconstructed dataset, which specifically includes: Step 21: Randomly select M training set images and N test set images from a dataset with diversity, and increase their resolution to amplify the feature information; Step 22: Treat the images in the dataset as the objects to be measured, and modulate the light field containing object information into a binary random matrix using DLP4500. Step 23: Use a photodetector to detect the total light intensity data transmitted through the object, and finally use the correlation calculation formula to directly reconstruct the image to form a new image dataset, thus obtaining the reconstructed dataset.

4. The computational ghost imaging method based on conditional generative adversarial networks at low sampling rates as described in claim 3, characterized in that: The correlation calculation formula in step 23 is as follows: Based on the second-order correlation principle, the random fluctuations in the light field intensity and the light intensity information from the bucket detector are calculated to obtain information about the object. Since the modulation of the DMD is controllable, the light field intensity distribution acting on the object can be calculated according to the Huygens-Fresnel principle. Ideally, if the intensity distribution of the actual light field is assumed to be equal to the calculated data, then the light intensity value obtained by the bucket detector is: （1） In the formula: The transmissivity function of an object. I i ( x , y ) represents the light field intensity distribution matrix of the reference light. Indicates coordinate position; The algorithm for calculating the second-order correlation of correlated imaging is as follows: （2） Where, < > represents N The average of the measurements. Indicates the first i The sum of the light intensity values ​​measured by the secondary barrel detector can be interpreted as the sum of the modulated light intensity information. Ideally, the final CGI image can be reconstructed using formulas (1) and (2). .

5. The computational ghost imaging method based on conditional generative adversarial networks at low sampling rates as described in claim 1, characterized in that: The specific implementation method for step five is as follows: First, the parameters from the generator are updated to the discriminator. The initialized CGI image is input into the generator, and then the generator gradient is set to 0. The generator generates samples, which are input into the discriminator to evaluate the loss. Then, the gradient is calculated in reverse to update the generator parameters. The updated generator parameters are put into the discriminator to zero the gradient. Then, the loss of the real samples and generated samples is calculated to reduce the difference between the detailed information of the tested object and the overall image. The discriminator parameters are updated in reverse to calculate the gradient. Then, the image output by the generator is input into the discriminator, and the parameters of the generator are adjusted according to the discriminator so that the generated result matches the distribution recognized by the discriminator. Repeat the above two steps to balance the training process.

Images