A ghost imaging method and system based on physical driving untrained double networks

Abstract

This application belongs to the field of optical computational imaging and machine vision technology, and more specifically, relates to a ghost imaging method and system based on a physically driven untrained dual-network architecture. The method first constructs a two-stage untrained neural network architecture of "initial reconstruction - fine reconstruction". The first-stage network quickly locks the low-frequency spatial structure of the object and introduces it as a priori constraint into the optimization process of the second-stage network. Combining a physical imaging model and regularization constraints, high-frequency detail features are recovered from bucket probe signals with extremely low sampling rates. Finally, through a physically consistent light intensity scale recovery mechanism, the normalized features output by the network are mapped back to the real physical light intensity space. This invention does not require pre-training data and effectively overcomes the technical difficulties of existing technologies that easily get trapped in local optima, suffer from significant noise, and lose physical light intensity scale at extremely low sampling rates (e.g., 5%), achieving high signal-to-noise ratio and high-fidelity target imaging.

Description

Technical Field

[0001] This application belongs to the field of optical computational imaging and machine vision technology, and more specifically, relates to a ghost imaging method and system based on a physically driven untrained dual network. Background Technology

[0002] Ghost imaging, also known as single-pixel imaging, is a novel computational imaging technology developed based on the principle of second-order correlation of light fields. This technology collects the total light intensity signal that matches the preset spatial light modulation pattern through a single-pixel detector, and then uses a computational reconstruction algorithm to recover the image of the target object. With its advantages of high detection sensitivity and wide spectral adaptability, it has great application potential in fields such as low-light imaging, non-line-of-sight imaging, and multispectral imaging.

[0003] However, in practical engineering applications, ghost imaging technology always faces the core contradiction between sampling efficiency and image quality. Existing reconstruction technologies generally have the following shortcomings: First, traditional correlation imaging algorithms suffer from a sharp decline in image signal-to-noise ratio under low sampling rate conditions, resulting in blurred images and significant noise. Although compressed sensing algorithms can reduce sampling requirements to some extent, the reconstruction quality deteriorates significantly in extreme scenarios with sampling rates below 10%, and hardware sampling is time-consuming, computationally inefficient, and cannot meet the requirements of real-time imaging.

[0004] Secondly, while current supervised learning-based deep ghost imaging solutions can improve imaging performance at low sampling rates, their performance relies entirely on pre-training with massive labeled datasets. In specialized fields such as medical detection, infrared imaging, and deep ultraviolet imaging, obtaining large-scale, high-quality datasets is extremely difficult. Furthermore, such pre-trained models have very poor generalization ability in unknown scenarios, making it difficult to adapt to complex and ever-changing actual imaging environments.

[0005] Third, the single-stage untrained network scheme based on deep image priors, although free from the pre-training dependence of external datasets and using physical model-driven self-supervised optimization, is prone to getting trapped in local optima at extremely low sampling rates, resulting in image structure distortion, loss of details, and poor robustness of reconstruction results.

[0006] Fourth, existing deep learning-based reconstruction schemes generally include normalization operations, which can cause the reconstructed image to deviate from the real physical light intensity space, resulting in drift of the light intensity mean and variance. Ultimately, the imaging results will have problems such as contrast distortion and grayscale deviation, failing to truly reflect the physical light intensity attributes of the target object and significantly reducing the practical value of the imaging results.

[0007] In summary, existing ghost imaging technology cannot simultaneously address the technical challenges of low sampling rate reconstruction quality, no data dependency, optimized stability, and physical light intensity accuracy. Summary of the Invention

[0008] This invention provides a ghost imaging method and system based on a physical-driven untrained dual network, which aims to solve the technical problem that existing ghost imaging technologies cannot simultaneously achieve low sampling rate reconstruction quality, no data dependency, optimization stability, and physical light intensity authenticity.

[0009] On one hand, the present invention provides a ghost imaging method based on a physically driven untrained dual network, comprising the following steps: Acquire one-dimensional real barrel detection signals from single-pixel imaging; A two-stage untrained neural network model is constructed. The two stages are the trained neural network model, which includes an initial reconstruction network and a fine reconstruction network. Both networks are initialized with random parameters and do not require pre-training with an external dataset. At the same time, both networks are coupled and bound to the ghost imaging physical forward propagation model to form a physics-driven optimization framework. Random features are input into the initial reconstruction network, and a simulated barrel detection signal is generated by combining the ghost imaging physical forward propagation model. The network parameters are optimized by minimizing the total loss function in the initial reconstruction stage to obtain an initial reconstructed image containing the target's low-frequency structure. Random features are input into the fine reconstruction network, the initial reconstructed image is used as the structural prior, and the ghost imaging physical forward propagation model is combined to generate a fine-stage simulated bucket detection signal. The network parameters are optimized by minimizing the total loss function of the fine reconstruction stage to obtain a normalized fine feature image. A linear mapping rule is established based on the statistical relationship between real barrel detection signals and fine-stage simulated barrel detection signals. The normalized fine-feature image is then mapped to the real physical light intensity space to obtain the ghost imaging reconstruction result.

[0010] This invention constructs a two-stage untrained neural network that does not require pre-training with external datasets and couples it with a ghost imaging physical forward propagation model to form a purely physics-driven optimization framework. First, an initial reconstruction network is used to extract the low-frequency basic structure of the target object to avoid the optimization of local optima. Then, the initial reconstructed image is used as a structural prior constraint to refine the reconstruction network and restore high-frequency details. Finally, a linear mapping is established based on the statistical relationship between real and simulated detection signals to complete the light intensity and size calibration. This fundamentally breaks through the dataset dependency bottleneck, significantly improves the reconstruction stability and imaging fidelity in extremely low sampling rate scenarios, and eliminates image structure distortion and physical light intensity distortion problems. It achieves high-fidelity ghost imaging reconstruction with low sampling, no pre-training, and high robustness.

[0011] Preferably, after acquiring the one-dimensional real barrel detection signal, the one-dimensional real barrel detection signal is denoised, normalized, and abnormal signal points are removed to obtain standardized signal data before the subsequent network reconstruction steps are performed.

[0012] Preferably, both the initial reconstruction network and the fine reconstruction network adopt an encoding-decoding convolutional architecture, and the network depth and the number of feature channels of the fine reconstruction network are greater than those of the initial reconstruction network, in order to enhance the ability to reconstruct high-frequency details.

[0013] Preferably, the total loss function in the initial reconstruction stage is formed by linearly weighting the mean square error terms of the simulated bucket detection signal and the real bucket detection signal, and the total variation regularization term of the initial reconstructed image.

[0014] Preferably, the total loss function of the fine reconstruction stage is composed of the mean square error term of the simulated bucket detection signal and the real bucket detection signal in the fine stage, the total variation regularization term of the fine feature image, and the structural similarity constraint term of the fine feature image and the initial reconstructed image through linear weighted summation.

[0015] Preferably, the solution process for the linear mapping rule is as follows: The real barrel detection signal and the simulated barrel detection signal in the fine stage are decentered respectively. The scale coefficient and offset parameter are obtained by fitting with the least squares method. Then the parameters are transformed into the two-dimensional image domain to complete the light intensity calibration.

[0016] On the other hand, the present invention provides a ghost imaging system based on a physical-driven untrained dual network, comprising a modulation projection module, a single-pixel detection module and a computation processing module connected in sequence. The modulation projection module is used to generate and project a random spatial light modulation substrate to perform spatial light modulation on the target object. The single-pixel detection module is used to collect the modulated total light intensity signal of the target object and convert it into a digital real barrel detection signal for transmission to the computing and processing module. The computational processing module is configured to execute a ghost imaging method based on a physics-driven untrained dual network as described in any one of claims 1 to 6.

[0017] Preferably, the modulation projection module uses a digital micromirror as a spatial light modulator, which, together with a white light source, enables the projection of a binary random modulation substrate.

[0018] Preferably, the computing module has a built-in GPU acceleration unit for parallel execution of dual-network iterative optimization, physical forward simulation, and light intensity scale calibration calculations.

[0019] The beneficial effects of this invention include: This invention constructs a two-stage untrained neural network that does not require pre-training with external datasets and couples it with a ghost imaging physical forward propagation model to form a purely physics-driven optimization framework. First, an initial reconstruction network is used to extract the low-frequency basic structure of the target object to avoid the optimization of local optima. Then, the initial reconstructed image is used as a structural prior constraint to refine the reconstruction network and restore high-frequency details. Finally, a linear mapping is established based on the statistical relationship between real and simulated detection signals to complete the light intensity and size calibration. This fundamentally breaks through the dataset dependency bottleneck, significantly improves the reconstruction stability and imaging fidelity in extremely low sampling rate scenarios, and eliminates image structure distortion and physical light intensity distortion problems. It achieves high-fidelity ghost imaging reconstruction with low sampling, no pre-training, and high robustness.

[0020] This invention, through a dual-network cascade and prior-guided strategy, can recover clear object structure and details at a sampling rate of only 5%, significantly outperforming traditional DGI and existing single-stage deep learning methods (GIDC). Secondly, it requires no training data, relying entirely on a physical model and self-supervised learning, requiring no pre-training dataset and exhibiting strong generalization ability, thus solving the problem of data scarcity in specific scenarios. Furthermore, through a light intensity recovery mechanism, it corrects the scale drift caused by normalization, ensuring that the reconstructed image matches the true light field distribution in terms of mean and variance, thereby improving imaging accuracy and practical value. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a schematic diagram of the hardware optical path structure of the ghost imaging system according to an embodiment of the present invention.

[0023] Figure 2 This is a schematic diagram of the overall process of the ghost imaging method described in an embodiment of the present invention.

[0024] Figure 3 This is a comparison diagram of the specific network architecture of the two-stage neural network in the embodiments of the present invention.

[0025] Figure 4 This is a comparison diagram of the principle and effect of the physically consistent light intensity scale recovery module in this embodiment of the invention.

[0026] Figure 5 This is a comparison of the convergence curves of the reconstruction error (MSE) of the method of the present invention and the single network method as a function of the number of iterations.

[0027] Figure 6 This is a comparison chart showing the simulation reconstruction effects of the method of this invention and existing technologies (DGI, GIDC) on a standard test image (Pepper) at different sampling rates.

[0028] Figure 7 This is a comparison chart of the simulation reconstruction effects of the method of the present invention and the prior art (GIDC-Correction) including affine correction at different sampling rates.

[0029] Figure 8 This is a comparison of experimental reconstruction results of a three-dimensional object (golf ball) using the method of this invention in a real imaging system.

[0030] Figure 9 This is a comparison chart of experimental reconstruction results and quantitative indicators of handwritten text (Hello) using the method of this invention in a real imaging system. Detailed Implementation

[0031] To make the technical problems, solutions, and beneficial effects of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0032] Example 1 See appendix Figure 2 This embodiment provides a ghost imaging method based on a physics-driven untrained dual-network, aiming to achieve high-fidelity imaging at extremely low sampling rates through a two-stage untrained neural network driven by a physics model. The specific steps are as follows: S1. Control the modulation projection module to project onto the object under test. A random binary speckle pattern. A single-pixel detection module synchronously acquires the corresponding bucket detection signal intensity value to obtain a one-dimensional measurement signal. In this embodiment, the image resolution is set to... Total number of image pixels Sampling rate The value is set to vary from 5% to 50%. At this point, the base pattern matrix... The dimension is .

[0033] S2. See also Figure 3 As shown, two cascaded convolutional neural networks are constructed in the computational processing module: Initial Reconstruction Network It employs a U-Net architecture with skip connections, consisting of a 4-layer downsampling encoder and a 4-layer upsampling decoder, with fixed one-dimensional single-channel random features as input. The output is the initial reconstructed image. ; Fine-scale network reconstruction : Adopting a more A more complex structure, comprising a 5-layer downsampling encoder and a 5-layer upsampling decoder; its input is an extended 32-channel random feature. The output is a fine feature image. .

[0034] Secondly, it should be noted that neither network requires pre-training with external data, but rather undergoes self-supervised learning through physical model constraints in subsequent steps.

[0035] S3. Use gradient descent algorithms (such as the Adam optimizer) to reconstruct the initial network. Perform iterative optimization to construct the first loss function. : In the formula: The value of the first loss function; (in (The projection substrate pattern) is a simulated barrel detection signal calculated based on the ghost imaging physical forward propagation model; These are the detector signals actually collected during the experiment; This represents the mean square error of MSE; It is a total variation (TV) regularization loss function used to suppress noise in the reconstructed image while preserving edge information; , serving as weighting coefficients for balancing the mean square error term and the total variation regularization term. This is achieved by minimizing... ,network Capable of quickly recovering the low-frequency spatial structure of an object from random noise (such as...) Figure 6 (The result shown in DUGIN-1); In this embodiment, the number of iterations in this stage is set to approximately 100.

[0036] S4. Second-stage prior-guided fine reconstruction after obtaining the initial image Then, this information is frozen and used as prior information in the second-stage optimization process to refine the network reconstruction. Perform iterative optimization to construct a second loss function. : ; In the formula: The value of the second loss function; This is the simulated barrel detection signal for the second stage; , (in (where is the sampling rate), and both serve as weighting coefficients for balancing the mean square error term and the total variation regularization term.

[0037] In this embodiment, a priori structural constraints are introduced. This constraint forces fine-grained networks In the initial reconstructed image A fine-grained search is performed within the neighborhood of the defined solution space, effectively avoiding the problem that a single network can easily get trapped in local optima (such as noise accumulation) at extremely low sampling rates; for example... Figure 5 As shown, after introducing this constraint, the MSE convergence curve is significantly lower than that of the comparative method (GIDC), proving the effectiveness of the strategy.

[0038] S5. The normalization layers inside the neural network cause changes in the output image. The numerical distribution of light intensity deviates from the true physical dimensions of light intensity (i.e., scale drift occurs); in order to output a physically meaningful high-fidelity image, this invention configures a physically consistent light intensity recovery module at the network output end; such as Figure 4 As shown, this step specifically includes: Parameter analytical solution: Based on the least squares principle, calculate the optimal linear scaling factor that maps the analog signal to the real signal. : ; Simultaneously calculate the bias. .

[0039] In the formula: The calculated optimal linear scaling factor is used to map the amplitude of the analog signal to the scale of the real signal. The calculated intensity bias is used to compensate for background noise or mean shift during the measurement process; This represents the actual bucket detection signal vector collected during the experiment; This is the simulated barrel detection signal vector calculated based on the physical model; For actual barrel detection signals The arithmetic mean; For simulating barrel detection signals The arithmetic mean; The actual bucket detection signal after removing the mean satisfies ; The mean-removed analog bucket detection signal satisfies ; The transpose operator for a vector; The preset regularization constant (set to a value in this embodiment) (), used to prevent the denominator from being zero and to improve the stability of numerical calculations.

[0040] Spatial back projection mapping: using the basis pattern matrix The statistical properties of the one-dimensional signal bias Offset converted to two-dimensional image space .

[0041] In the formula: The average row sum of the projection substrate pattern matrix is ​​used to transform the bias in the one-dimensional signal space to the two-dimensional image space.

[0042] Final imaging output: Perform linear transformation .

[0043] In the formula: This is the final reconstructed image output after affine transformation correction; The initial image to be corrected output by the network; This is the offset in the image space.

[0044] This step not only makes the reconstructed image structurally clear, but also highly consistent with the ground truth in terms of grayscale distribution (mean and variance of light intensity).

[0045] Furthermore, to demonstrate the effectiveness of the invention, a standard dataset (Pepper) and real-world physical scenarios (golf balls, handwritten text) were selected for testing.

[0046] Extremely low sampling rate performance: such as Figure 6 , Figure 8 and Figure 9 As shown, under extreme conditions with a sampling rate of only 5%, the reconstruction results of existing technologies (DGI, GIDC) are basically submerged in noise and cannot be identified; while the method of the present invention can clearly recover the pit texture and handwritten text features on the surface of a golf ball.

[0047] Quantitative Analysis: At a 5% sampling rate, the quantitative improvement of the method of this invention (DUGIN) compared to existing technologies is extremely significant. Table 1 below lists the quantitative comparison results on the standard test set (Pepper) at sampling rates ranging from 5% to 50%: Table 1 Comparison of Quantitative Indicators for Simulated Reconstruction of “Pepper” Test Images

[0048] As shown in Table 1 and the actual experiment Figure 9 As shown, at an extremely low sampling rate of 10%, the mean squared error (MSE) of the method of this invention is reduced by an order of magnitude compared to the GIDC method (e.g., in handwritten text). Figure 9 In this context, MSE decreased from 0.0309 to 0.0066, and structural similarity (SSIM) improved significantly. Meanwhile, as... Figure 7The ablation experiments show that even with the addition of affine correction (GIDC-Correction) to the existing GIDC method, the structural reconstruction of the present invention is still clearer, which fully demonstrates the superiority of the two-stage network combined with the physically consistent light intensity recovery strategy.

[0049] Example 2

[0050] This embodiment provides a ghost imaging system based on a physically driven untrained dual network for executing the imaging method described in this invention. See also... Figure 1 As shown, the system includes: Modulation projection module: Employs a digital micromirror device (DMD, model VIALUX V-7001) as the spatial light modulator for loading and projecting a resolution of [resolution missing]. (For example Random binary basis pattern sequence The light source is a white LED, which is collimated by a lens group and then irradiates the surface of the DMD.

[0051] Single-pixel detection module: A single-point detector (model Thorlabs PDA 100 A2, wavelength response range 320-1100 nm) is used to collect the total light intensity signal after being scattered or transmitted by the target object; the analog signal output by the detector is digitized by a high-speed data acquisition card (DAQ) and then transmitted to the computing and processing module.

[0052] Computational processing module: Configured as a computer workstation with a GPU (such as an Intel Core i5-10400 CPU and an NVIDIA GeForce GTX 1660 GPU) to perform the construction, optimization and light intensity scale recovery operations of a two-stage neural network.

[0053] The above are merely preferred embodiments of this application and are not intended to limit this application. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A ghost imaging method based on a physically driven untrained dual network, characterized in that, Includes the following steps: Acquire one-dimensional real barrel detection signals from single-pixel imaging; A two-stage untrained neural network model is constructed. The two stages are the trained neural network model, which includes an initial reconstruction network and a fine reconstruction network. Both networks are initialized with random parameters and do not require pre-training with an external dataset. At the same time, both networks are coupled and bound to the ghost imaging physical forward propagation model to form a physics-driven optimization framework. Random features are input into the initial reconstruction network, and a simulated barrel detection signal is generated by combining the ghost imaging physical forward propagation model. The network parameters are optimized by minimizing the total loss function in the initial reconstruction stage to obtain an initial reconstructed image containing the target's low-frequency structure. Random features are input into the fine reconstruction network, the initial reconstructed image is used as the structural prior, and the ghost imaging physical forward propagation model is combined to generate a fine-stage simulated bucket detection signal. The network parameters are optimized by minimizing the total loss function of the fine reconstruction stage to obtain a normalized fine feature image. A linear mapping rule is established based on the statistical relationship between real barrel detection signals and fine-stage simulated barrel detection signals. The normalized fine-feature image is then mapped to the real physical light intensity space to obtain the ghost imaging reconstruction result.

2. The ghost imaging method based on a physically driven untrained dual network according to claim 1, characterized in that, After acquiring the one-dimensional real bucket detection signal, the one-dimensional real bucket detection signal is denoised, normalized, and abnormal signal points are removed to obtain standardized signal data before the subsequent network reconstruction steps are performed.

3. The method of claim 1, wherein the method is based on a physical-driven untrained dual-network-based ghost imaging. Both the initial reconstruction network and the fine reconstruction network adopt an encoding-decoding convolutional architecture, and the network depth and the number of feature channels of the fine reconstruction network are greater than those of the initial reconstruction network, which is used to enhance the ability to reconstruct high-frequency details.

4. The method of claim 1, wherein the method is a physical-driven untrained dual-network based ghost imaging method. The total loss function in the initial reconstruction stage is composed of the mean square error terms of the simulated bucket detection signal and the real bucket detection signal, and the total variation regularization term of the initial reconstructed image, which are obtained by linear weighted summation.

5. The method of claim 1, wherein the method is a physical-driven untrained dual-network based ghost imaging method. The total loss function of the fine reconstruction stage is composed of the mean square error term of the simulated bucket detection signal and the real bucket detection signal in the fine stage, the total variation regularization term of the fine feature image, and the structural similarity constraint term between the fine feature image and the initial reconstructed image, which are linearly weighted and summed.

6. The method of claim 1, wherein the method is a physical-driven untrained dual-network based ghost imaging method. The solution process for the linear mapping rule is as follows: The real barrel detection signal and the simulated barrel detection signal in the fine stage are decentered respectively. The scale coefficient and offset parameter are obtained by fitting with the least squares method. Then the parameters are transformed into the two-dimensional image domain to complete the light intensity calibration.

7. A physical-driven untrained double-network based ghost imaging system, characterized in that, It includes a modulation projection module, a single-pixel detection module, and a calculation and processing module connected in sequence; The modulation projection module is used to generate and project a random spatial light modulation substrate to perform spatial light modulation on the target object. The single-pixel detection module is used to collect the modulated total light intensity signal of the target object and convert it into a digital real barrel detection signal for transmission to the computing and processing module. The computational processing module is configured to execute a ghost imaging method based on a physics-driven untrained dual network as described in any one of claims 1 to 6.

8. The physical-drive-untrained dual-network-based ghost imaging system according to claim 7, wherein, The modulation projection module uses a digital micromirror as a spatial light modulator, which, together with a white light source, enables the projection onto a binary random modulation substrate.

9. The physical-drive-untrained dual-network-based ghost imaging system according to claim 7, wherein, The computing processing module is internally provided with a GPU acceleration unit for performing parallel operation of double-network iterative optimization, physical forward simulation and light intensity scale calibration.

Images