A physical driving deep learning-based non-diffraction computing ghost imaging optimization method
By using a physics-driven, cyclic consistent generative adversarial network model combined with a quasi-diffraction-free point spread function, the problems of small depth of field and low image quality in traditional ghost imaging techniques are solved, achieving high-resolution and robust image reconstruction.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- NANJING UNIV OF SCI & TECH
- Filing Date
- 2026-03-13
- Publication Date
- 2026-06-23
AI Technical Summary
Traditional computational ghost imaging techniques are limited by diffraction, have a small depth of field, and their image quality is affected by side lobes. Pure data-driven deep learning methods cannot accurately recover the target image when the propagation distance changes.
A physics-driven recurrent consistent generative adversarial network model is adopted, combined with a quasi-diffraction speckle-free point spread function. The network training is guided by physical laws to suppress multi-level sidelobes and improve imaging quality and depth of field.
It achieves imaging effects with large depth of field and high resolution. The network exhibits robustness and generalization ability far exceeding that of pure data-driven methods, and can accurately reconstruct images under real noise and long-distance propagation.
Smart Images
Figure CN122265087A_ABST
Abstract
Description
Technical Field
[0001] This invention pertains to computational ghost imaging optimization methods, specifically a diffraction-free computational ghost imaging optimization method based on physics-driven deep learning. Background Technology
[0002] In recent years, computational ghost imaging technology has shown broad application prospects in various fields due to its single-pixel detection and high anti-interference capability. Traditional computational ghost imaging techniques mostly employ Gaussian random speckle, but it is limited by diffraction, and the speckle field changes rapidly with propagation distance, resulting in extremely small system depth of field. Introducing quasi-diffraction speckle can effectively overcome the limitations of diffraction evolution and achieve long-distance, large-depth-of-field imaging. However, the point spread function of quasi-diffraction speckle exhibits strong multi-level sidelobes in the Fourier transform domain. During the correlation reconstruction process of computational ghost imaging, these sidelobes severely degrade image quality.
[0003] In recent years, deep learning has been introduced into the field of computational imaging to solve image reconstruction problems. However, traditional pure data-driven deep learning methods heavily rely on massive amounts of training data. Furthermore, pure data-driven methods often fail to accurately reconstruct the target image when the propagation distance changes. Therefore, there is an urgent need for a novel ghost imaging optimization method that can incorporate physical laws to guide network training. Summary of the Invention
[0004] This invention proposes a physics-driven deep learning-based optimization method for diffraction-free computational ghost imaging. It uses a physics-driven cyclic consistent generative adversarial network model to suppress high-order oscillations caused by multi-level sidelobes of the spread function at the corresponding point of quasi-diffraction speckle, thereby improving the imaging quality and depth of field of computational ghost imaging.
[0005] The technical solution to achieve the purpose of this invention is: a diffraction-free ghost imaging optimization method based on physics-driven deep learning, comprising the following steps:
[0006] Step 1: Obtain quasi-diffraction speckle-free speckle by using a phase modulation method that constructs a ring-shaped phase template;
[0007] Step 2: Determine the point spread function for quasi-diffraction speckle-free conditions;
[0008] Step 3: Reconstruct the MNIST dataset images using ghost imaging techniques and quasi-diffraction speckle-free imaging.
[0009] Step 4: Train the physics-driven recurrent consistent generative adversarial network model using the reconstructed training data;
[0010] Step 5: Use the total light intensity of the target image acquired by the computational ghost imaging system to perform correlation calculations to reconstruct the image, and then input it into the trained physical drive network model to obtain a high-quality reconstructed image.
[0011] Compared with existing technologies, the significant advantages of this invention are: (1) It utilizes the stable characteristics of quasi-diffraction speckle propagation over long distances to achieve a large depth of field, while simultaneously suppressing the high-order oscillation artifacts caused by its inherent sidelobes through a physically driven network, thus achieving high-resolution imaging. (2) This invention innovatively embeds the point spread function convolution physical degradation model into the network architecture. The guidance of physical rules enables the network to more accurately infer the distribution of real images, exhibiting robustness and generalization ability far exceeding that of pure data-driven methods when facing real experimental noise or long-distance propagation.
[0012] The present invention will now be described in further detail with reference to the accompanying drawings. Attached Figure Description
[0013] Figure 1 The generative adversarial network framework and algorithm flow designed for this invention.
[0014] Figure 2 These are images from a dataset used throughout this invention.
[0015] Figure 3 The generator network structure designed for this invention.
[0016] Figure 4 The discriminator network structure designed for this invention.
[0017] Figure 5 This is a schematic diagram illustrating the performance of the optimal network model of this invention.
[0018] Figure 6 This is an experimental setup for computing ghost imaging systems. Detailed Implementation
[0019] like Figure 1 As shown, the concept of this invention is as follows: a physics-driven deep learning-based optimization method for diffraction-free computational ghost imaging. This method uses a physics-driven recurrent consistent generative adversarial network (RCA) model to suppress image quality degradation caused by multi-level sidelobes of the point spread function corresponding to quasi-diffraction speckle, thereby improving the imaging quality and depth of field of computational ghost imaging. Quasi-diffraction speckle can maintain the stability of the light field morphology during long-distance propagation, and its introduction can effectively improve the depth of field of the computational ghost imaging system. Binary images are used and subjected to a series of preprocessing steps as the dataset for network training. A RCA model is designed to construct a generator and discriminator, while the convolution process of the target image and the point spread function is embedded into the network as an explicit physical prior. The model is trained, and the accuracy of the images generated by the network is trained based on the generator loss and discriminator loss. During this process, the computation process is optimized to ensure the differentiability of each processing step. Finally, network model parameters that can accurately eliminate target image oscillations are obtained, and the corresponding images are generated. The specific steps are as follows:
[0020] Step 1: Obtain quasi-diffraction speckle-free speckle by using a phase modulation method that constructs a ring-shaped phase template.
[0021] Step 1.1: Establish a ring-shaped phase template and set the inner diameter of the template. and outer diameter .
[0022] Step 1.2: Divide the annular region of the annular phase template into equal parts along the angular direction. Each sector is assigned an independent random phase of 0-2π.
[0023] Step 1.3: Using the constructed annular phase template as the initial complex amplitude distribution of the incident light field, the lens Fourier transform process in physical optics is simulated. A two-dimensional digital matrix loaded with the annular phase template information is used as the initial complex amplitude distribution. A two-dimensional discrete fast Fourier transform algorithm including zero-frequency normalization is applied to perform a spatial-frequency domain transformation to obtain the fundamental frequency domain matrix. Then, according to the set incident light wavelength... Lens focal length Pixel area , and the fundamental frequency domain matrix The accurate complex amplitude distribution matrix of the optical field at the back focal plane of the lens was calculated. Specifically, it means:
[0024]
[0025]
[0026] in, The initial complex amplitude matrix after zero-padding and frequency shifting is in the th... OK The element values of the column; , This represents the total number of rows and columns of the matrix; , Spatial frequency coordinates; The table shows the complex values at the corresponding frequency domain coordinates after the transformation; The imaginary unit; This is the calculated complex amplitude value of the light field at the back focal plane of the lens.
[0027] Step 2: Calculate the point spread function for quasi-diffraction-free speckle. The specific process is as follows:
[0028]
[0029] in, These are the normalization coefficients; , These are the outer and inner diameters of the annular phase template, respectively. It is a first-order Bessel function; For wave number, equal to ; The focal length of the lens for performing Fourier transform; For radial coordinates.
[0030] Step 3: Reconstruct the MNIST dataset images using ghost imaging techniques and quasi-diffraction speckle-free imaging to obtain the training dataset. The specific process is as follows:
[0031] Step 3.1: Extract from the standard MNIST dataset A handwritten digit image, the original image enlarged to 128. 128 pixels, normalizing the image pixel values to the [0,1] interval, denoted as .
[0032] Step 3.2: Simulate transmission, using the signal generated in Step 1. The i-th speckle field in a quasi-diffraction-free speckle pattern Illuminate the target object Above, calculate all the light energy after it has passed through the object. The specific process is as follows:
[0033]
[0034] Repeat the above process Next, obtain a containing A sequence of one-dimensional light intensity measurements of elements .
[0035] Step 3.3: Given a series of speckle field matrices... The corresponding collected light intensity value of the bucket detector The reconstructed image is obtained by performing second-order correlation operations. The specific process is as follows:
[0036]
[0037]
[0038]
[0039] in, It is the average of the measurements from all the barrel detectors. It is the average light intensity distribution of all speckle fields at each pixel.
[0040] Repeat steps 3.2 to 3.3 until... The image reconstruction is complete.
[0041] Step 4: Train the physics-driven recurrent generative adversarial network model using the reconstructed training data. The physics-driven network model employs a dual-loop generative adversarial architecture, including a first generator. Second generator and the corresponding two discriminators , Among them, the physics-driven recurrent consistent generative adversarial network model is as follows: Figure 1 As shown, the specific process of training a physics-driven recurrent consistent generative adversarial network model with the reconstructed training data is as follows:
[0042] Step 4.1: Obtain the reconstructed images from the MNIST dataset processed in Step 3 as input domain data, and the corresponding original target images as ground truth domain data. Perform batch normalization operations on the two sets of data respectively.
[0043] Step 4.2: Perform the forward physics-driven mapping of the upper loop path: normalize the initial reconstructed image. Input the first generator Output the predicted target amplitude image ;Predict the target amplitude image The point spread function obtained in step 2 is convolved with the physical degradation process of quasi-diffraction speckle-free imaging to obtain the physically degraded diffraction feature matrix; the physically degraded diffraction feature matrix is then input into the second generator. The output is the first cycle-consistent diffraction image corresponding to the original input. ;
[0044] Step 4.3: Perform the reverse physics-driven mapping of the lower loop path: First, normalize the ground truth image... Convolution is performed with the point spread function obtained in step 2 to obtain a ground-value diffraction image with true physical degradation characteristics. ; True value diffraction image Input the second generator The predicted diffraction degradation image is then generated; subsequently, the predicted diffraction degradation image is input into the first generator. Perform feature restoration and output the second cycle consistency amplitude image corresponding to the original ground truth. ;
[0045] Step 4.4: Update the discriminator weights: update the predicted target amplitude image. Second cycle consistency amplitude image Corresponding real image Send to the first discriminator ;The first cycle-consistent diffraction pattern True value diffraction image Corresponding real image Send to the second discriminator Calculate the adversarial loss of the discriminator and optimize the discriminator parameters preferentially through backpropagation;
[0046] Lock discriminator parameters and update generator weights: Calculate the generator's total loss based on a physical double-loop process, and use backpropagation of the total loss to jointly update the first generator. With the second generator The parameters are used to ensure physical bidirectional loop consistency.
[0047] In a further embodiment, a U-net network structure is used as a generator for deep image feature extraction. The U-net network has a roughly U-shaped structure, as shown below. Figure 3 As shown, the input data of the network is the initial reconstructed image obtained in step 3. The Encoder part of the generator performs backbone feature extraction. The Encoder part contains four backbone feature extraction processes. Each feature extraction first passes through two identical convolutional layers to obtain multi-channel features, and then uses the SE module to recalibrate the channel weights of the multi-channel features. After average pooling downsampling, the feature set is obtained. The deepest feature set obtained after four backbone feature extractions is input into the Decoder part of the generator for enhanced feature extraction. The Decoder part of the generator contains four enhanced feature extraction processes. The enhanced feature extraction process performs transposed convolution upsampling on the feature set and adds a skip connection process during the upsampling process. Specifically, the multi-channel features extracted by the Encoder part at the corresponding depth and recalibrated by the SE module are concatenated with the upsampled result to obtain the enhanced feature set. Then, the features are restored through convolutional layers. The final output of the first generator is a high-quality reconstructed image after denoising.
[0048] In further embodiments, the discriminator adopts a global discriminator structure, the structure of which is as follows: Figure 4 As shown, the input to the discriminator is the image data to be judged. The discriminator performs downsampling feature extraction through continuous convolutional layers with strides, and embeds an SE channel attention mechanism module after the first convolutional feature extraction layer. Finally, the extracted deep feature matrix is fed into a global average pooling layer for global feature compression in spatial dimension, and then mapped to a single scalar value through a convolutional layer. This scalar value represents the global evaluation probability of the authenticity of the entire input image, which is the final output of the discriminator.
[0049] By training a double-loop generative adversarial network model, the performance of the generator in the network is trained to minimize the total loss function, which is expressed as:
[0050]
[0051] in, Let be the generator loss function. The discriminator loss function;
[0052] The generator loss function optimizes the generator by calculating the feature deviation between the reconstructed image and the ground truth image, as well as the consistency before and after physical degradation based on the point spread function. Specifically:
[0053]
[0054]
[0055]
[0056] in, Generative adversarial loss for the generator, Ensures the cycle-consistent prediction loss for the reversibility of bidirectional physical mapping. For the perceptual loss of the generator, and The corresponding weighting coefficients;
[0057] Discriminator loss function Specifically:
[0058]
[0059]
[0060] in, This is the adversarial loss of the discriminator, used to calculate the difference in Wasserstein distance distribution between the real and generated images; Gradient penalty loss is used to improve the training stability of the discriminator network. represents the weight coefficient of the gradient penalty term.
[0061] The network parameter model is obtained through training.
[0062] Step 5: Use the total light intensity of the target image acquired by the computational ghost imaging system to perform correlation calculations to reconstruct the image, and then input it into the trained physical drive network model to obtain a high-quality reconstructed image.
[0063] Step 5.1: Based on the phase modulation method designed in Step 1, a quasi-diffraction-free speckle pattern is generated using a spatial light modulator; the image is projected onto the target image, and a detector is used to receive the light signal. The value of the light signal is used as the single-pixel barrel detection measurement value. The recorded single-pixel barrel detection measurement value is compared with the predetermined quasi-diffraction-free speckle light intensity distribution matrix to perform a second-order correlation operation, thereby reconstructing the initial target image with higher-order oscillation artifacts.
[0064] Step 5.2: Input the initial target image with high-order oscillation artifacts obtained in Step 5.1 into the physical-driven loop-based generative adversarial network model to reconstruct a high-quality target image.
[0065] Verification of the high-quality image effects generated by the physics-driven cyclic consistent generative adversarial network model of this invention, corresponding to... Figure 5 .
[0066] The optimized image is obtained by inputting the test set images into the network model, such as... Figure 5 As shown in (d), Figure 5 (a) is the test set image. Figure 5 (b) The result of correlation reconstruction using quasi-diffraction speckle-free speckle. Figure 5 (c) Results of data-driven generative adversarial networks.
[0067] This invention proposes a diffraction-free computational ghost imaging optimization method based on physics-driven deep learning, which is a high-quality diffraction-free computational ghost imaging method with generalization ability.
Claims
1. A diffraction-free computational ghost imaging optimization method based on physics-driven deep learning, characterized in that, Includes the following steps: Step 1: Obtain quasi-diffraction speckle-free speckle by using a phase modulation method that constructs a ring-shaped phase template; Step 2: Determine the point spread function for quasi-diffraction speckle-free conditions; Step 3: Reconstruct the MNIST dataset images using ghost imaging techniques and quasi-diffraction speckle-free imaging. Step 4: Train the physics-driven recurrent consistent generative adversarial network model using the reconstructed training data; Step 5: Use the total light intensity of the target image acquired by the computational ghost imaging system to perform correlation calculations to reconstruct the image, and then input it into the trained physical drive network model to obtain a high-quality reconstructed image.
2. The diffraction-free ghost imaging optimization method based on physics-driven deep learning according to claim 1, characterized in that, The specific process of obtaining quasi-diffraction speckle-free speckle by using phase modulation with a constructed annular phase template is as follows: Step 1.1: Create a ring-shaped phase template and set the inner diameter of the ring-shaped phase template. and outer diameter ; Step 1.2: Divide the annular region of the annular phase template into equal parts along the angular direction. Each sector is assigned an independent random phase of 0-2π. Step 1.3: Using the constructed annular phase template as the complex amplitude distribution of the initial incident light field, the lens Fourier transform process in physical optics is simulated to calculate the complex amplitude of the light field at the focal plane, and the intensity distribution of the complex amplitude of the light field is obtained, thereby generating quasi-diffraction-free speckle.
3. The diffraction-free ghost imaging optimization method based on physics-driven deep learning according to claim 2, characterized in that, The specific process of simulating the lens Fourier transform process in physical optics and calculating the complex amplitude of the light field at the focal plane in step 1.3 is as follows: Using the two-dimensional digital matrix loaded with the ring phase template information as the initial complex amplitude distribution, a two-dimensional discrete fast Fourier transform algorithm including zero-frequency normalization is used to perform a spatial-frequency domain transformation on the initial complex amplitude distribution to obtain the fundamental frequency domain matrix. ; According to the set incident light wavelength Lens focal length Pixel area , and the fundamental frequency domain matrix The accurate complex amplitude distribution matrix of the optical field at the back focal plane of the lens was calculated. Specifically: in, The initial complex amplitude matrix after zero-padding and frequency shifting is in the th... OK The element values of the column; , This represents the total number of rows and columns of the matrix; , Spatial frequency coordinates; The table shows the complex values at the corresponding frequency domain coordinates after the transformation; The imaginary unit; This is the calculated complex amplitude value of the light field at the back focal plane of the lens.
4. The diffraction-free computational ghost imaging optimization method based on physics-driven deep learning according to claim 1, characterized in that, The determined quasi-diffraction-free speckle diffusion function is as follows: in, These are the normalization coefficients; , These are the outer and inner diameters of the annular phase template, respectively. It is a first-order Bessel function; Wave number; The focal length of the lens for performing Fourier transform; For radial coordinates.
5. The diffraction-free computational ghost imaging optimization method based on physics-driven deep learning according to claim 1, characterized in that, The specific process of training a physics-driven cycle-consistent generative adversarial network model using the reconstructed training data is as follows: Step 4.1: Obtain the reconstructed images from the MNIST dataset processed in Step 3 as input domain data, and the corresponding original target images as ground truth domain data. Perform batch normalization operations on the two sets of data respectively. Step 4.2: Perform the forward physics-driven mapping of the upper loop path: normalize the initial reconstructed image. Input the first generator Output the predicted target amplitude image ;Predict the target amplitude image The point spread function obtained in step 2 is convolved with the physical degradation process of quasi-diffraction speckle-free imaging to obtain the physically degraded diffraction feature matrix; the physically degraded diffraction feature matrix is then input into the second generator. The output is a cycle-consistent diffraction image corresponding to the original input. ; Step 4.3: Perform the reverse physics-driven mapping of the lower loop path: First, normalize the ground truth image... Convolution is performed with the point spread function obtained in step 2 to obtain a ground-value diffraction image with true physical degradation characteristics. ; True value diffraction image Input the second generator The predicted diffraction degradation image is then generated; subsequently, the predicted diffraction degradation image is input into the first generator. Perform feature restoration and output a cycle-consistent magnitude image corresponding to the original ground truth. ; Step 4.4: Update the discriminator weights: update the predicted target amplitude image. Second cycle consistency amplitude image Corresponding real image Send to the first discriminator ;The first cycle-consistent diffraction pattern True value diffraction image Corresponding real image Send to the second discriminator The adversarial loss of the discriminator is calculated, and the discriminator parameters are optimized preferentially through backpropagation.
6. The diffraction-free ghost imaging optimization method based on physics-driven deep learning according to claim 5, characterized in that, First generator Second generator Both methods employ the U-net network structure. The input data for the generator is the initial reconstructed image obtained in step 3. The generator's Encoder part performs backbone feature extraction, which includes four backbone feature extraction processes. Each feature extraction first passes through two identical convolutional layers to obtain multi-channel features, and then uses the SE module to recalibrate the channel weights of the multi-channel features. After average pooling downsampling, a feature set is obtained. The deepest feature set obtained after four backbone feature extractions is input into the generator's Decoder part for enhanced feature extraction. The generator's Decoder part includes four enhanced feature extraction processes. The enhanced feature extraction process performs transposed convolutional upsampling on the feature set and adds a skip connection process during the upsampling process. Specifically, the multi-channel features extracted from the Encoder part at the corresponding depth and recalibrated by the SE module are concatenated with the upsampled result to obtain the enhanced feature set. Then, the features are restored through convolutional layers.
7. The diffraction-free ghost imaging optimization method based on physics-driven deep learning according to claim 5, characterized in that, The first discriminator Second discriminator Both methods employ a global discriminator structure. The input to the discriminator is the image data to be judged. The discriminator performs downsampling feature extraction through continuous convolutional layers with strides, and embeds an SE channel attention mechanism module after the first convolutional feature extraction layer. Finally, the extracted deep feature matrix is fed into a global average pooling layer for global feature compression in spatial dimension, and then mapped to a single scalar value through a convolutional layer. The scalar value represents the global evaluation probability of the authenticity of the entire input image, which serves as the final output of the discriminator.
8. The diffraction-free computational ghost imaging optimization method based on physics-driven deep learning according to claim 1, characterized in that, By training a cyclic consistent generative adversarial network model, the total loss function is minimized. The total loss function is expressed as: in, Let be the generator loss function. Let be the discriminator loss function.
9. The diffraction-free computational ghost imaging optimization method based on physics-driven deep learning according to claim 8, characterized in that, The generator loss function optimizes the generator by calculating the feature deviation between the reconstructed image and the ground truth image, as well as the consistency before and after physical degradation based on the point spread function. Specifically: in, Generative adversarial loss for the generator, Ensures the cycle-consistent prediction loss for the reversibility of bidirectional physical mapping. For the perceptual loss of the generator, and The corresponding weighting coefficients.
10. The diffraction-free ghost imaging optimization method based on physics-driven deep learning according to claim 8, characterized in that, Discriminator loss function Specifically: in, This is the adversarial loss of the discriminator, used to calculate the difference in Wasserstein distance distribution between the real and generated images; Gradient penalty loss is used to improve the training stability of the discriminator network. represents the weight coefficient of the gradient penalty term.