Unsupervised undersampling non-vision field imaging method based on virtual scanning
Through virtual scanning and unsupervised deep learning technology, the low quality problem of non-sight imaging in irregular relay surface environment is solved, efficient and robust non-sight imaging is achieved, reducing dependence on paired data, and significantly improving the Rebuild quality and speed.
Patent Information
- Application Number
- CN202411076964.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-07
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2044-08-07
AI Technical Summary
Existing non-domain imaging technologies are difficult to achieve high-quality reconstruction in irregular relay surface environments, and are highly dependent on paired data based on deep learning methods, and are expensive to obtain and align.
An unsupervised undersampled non-sight imaging method based on virtual scanning is proposed. By constructing a simulated transient data set and designing a denoiser network with partial convolutional encoder-decoder structure, combining physical priors and attention mechanisms, Unsupervised training is used to achieve the recovery of hidden scenes from irregular undersampled transient data.
High-quality non-sight imaging in irregular relay surface environment is achieved, reducing dependence on paired data, improving reconstruction efficiency and robustness, and significantly better than the reconstruction quality and inference speed of existing methods.
Smart Images

Figure CN119006711A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computational imaging technology, and in particular to an unsupervised under-sampling non-viewing-area imaging method based on virtual scanning. Background Art
[0002] Non-line-of-sight (NLOS) imaging aims to reconstruct hidden scenes beyond the detector's line of sight. This emerging field has attracted widespread interest from researchers in various fields such as robotic vision, autonomous driving, rescue operations, remote sensing, and medical imaging. In a typical active confocal non-line-of-sight imaging system, both the laser source and the detector are focused on the same point on the relay surface. The pulse emitted by the laser is reflected by the surface to illuminate the hidden scene. The detector captures the photons that bounce back from the scene to the relay surface, which is called a photon histogram. In order to improve the reconstruction resolution and quality, the relay surface is often scanned at multiple points, and the captured photon time volume becomes a transient measurement (with dimensions of H×W×T). The information of the hidden scene can be recovered from the transient measurement through a carefully designed algorithm.
[0003] Existing work often focuses on improving reconstruction quality or scanning efficiency. Although significant breakthroughs have been made, these methods also face significant limitations, which seriously hinder the practical application of non-line-of-sight. These methods require that the imaging environment contains a large area of relay surfaces, which enables the imaging system to perform dense and regular scanning. In some real-world application scenarios, there may only be irregular relay surfaces such as lattices or fences, which makes the assumptions of these methods on the imaging environment invalid. Restoring hidden scenes using undersampled transient measurements obtained from irregular scanning is a serious ill-posed problem, which causes artifacts and even structural loops in the reconstructed volume. This leads to a challenging task, namely, to achieve high-quality non-line-of-sight imaging from irregular undersampled transient measurements. To solve this problem, CC-SOCR introduces a manually designed strong regularization term under the framework of functional optimization. However, this method requires lengthy numerical iterative calculations, which seriously limits its application efficiency.
[0004] Recent deep learning-based non-line-of-sight imaging methods have achieved high-quality single-stage fast inference imaging, but they require a large amount of paired data as training supervision, and these data need to be accurately aligned with the true albedo volume. In fact, such real data pairs are difficult to obtain and expensive, so in order to eliminate the heavy reliance of deep learning-based methods on paired data, it is very valuable to explore non-line-of-sight imaging under unsupervised learning.
[0005] In order to solve the above problems, the method of the present invention proposes an unsupervised under-sampled non-field of view imaging method based on virtual scanning, and the test experiments on simulation and real under-sampled transient data show the optimal reconstruction results. Summary of the invention
[0006] The purpose of the present invention is to propose an unsupervised under-sampling non-line-of-sight imaging method based on virtual scanning to solve the problems raised in the background technology and achieve higher quality non-line-of-sight reconstruction.
[0007] In order to achieve the above object, the present invention adopts the following technical solutions:
[0008] The unsupervised under-sampling non-viewing-area imaging method based on virtual scanning specifically includes the following steps:
[0009] S1. Constructing an under-sampled non-line-of-sight reconstruction simulation transient dataset: Using a public transient renderer and ShapeNet3D object data to construct a simulation transient dataset, adding noise and time jitter characteristics according to the non-line-of-sight imaging single-photon detector imaging model, and generating an under-sampled non-line-of-sight transient dataset according to the set scanning pattern;
[0010] S2. Build a network framework: Design a denoiser network with an encoder-decoder structure consisting of partial convolutions and a reconstruction network combined with physical priors, connect the denoiser network and the reconstruction network to reconstruct hidden scenes from irregular undersampled transients;
[0011] S3, design framework, build model: design a non-line-of-sight reconstruction scheme based on the under-sampled non-line-of-sight reconstruction simulation transient data set and network framework described in S1-S2. The scheme specifically includes the following contents:
[0012] ① Undersampled transient denoising: The captured irregular undersampled transient measurements are filled into the full sampling grid by zero-filling method, and the captured irregular undersampled transient measurements and the mask are input into the denoiser network to obtain the initial denoised transient, which is then multiplied by the mask to remove the artifacts introduced by the network and obtain the denoised undersampled transient.
[0013] ②Hidden scene reconstruction: The denoised undersampled transient is initially reconstructed using the traditional LCT algorithm to obtain the value domain information of the true value reconstruction result, which is input into the reconstruction network to obtain the zero domain information of the true value reconstruction result. The value domain information is combined with the zero domain information to obtain the albedo reconstruction volume of the hidden scene.
[0014] ③ Design virtual scanning training strategy: Based on the principle of zero-value domain decomposition, design an unsupervised virtual scanning training strategy and its loss function to enable the neural network to learn the ability to restore zero-domain information;
[0015] S4. Training the unsupervised non-line-of-sight reconstruction framework: Use the deep learning Pytorch framework to train the constructed network framework and the virtual scanning reconstruction network model in stages, and repeatedly traverse the undersampled non-line-of-sight reconstruction simulation transient data set constructed in S1 until the model converges.
[0016] Preferably, the S1 specifically includes the following contents:
[0017] S1.1, select and organize ShapeNet 3D object data;
[0018] S1.2, Modeling single photon detector imaging under non-line-of-sight imaging conditions;
[0019] S1.3, using the public non-line-of-sight imaging transient measurement renderer to simulate ideal transient measurements based on ShapeNet 3D object data;
[0020] S1.4, post-processing the simulated ideal transient measurement according to the single-photon detector imaging model to make it close to the fully sampled transient measurement of the real acquisition;
[0021] S1.5. Extract undersampled data from the simulated full-sampled transient measurement according to the selected scanning pattern for training and construct a data set.
[0022] Preferably, S2 specifically includes the following contents:
[0023] S2.1. A non-line-of-sight single-photon detector imaging model is given. The model is an imaging model that takes into account a low-photon time-resolution detector. The specific function is expressed as:
[0024]
[0025] where u represents the noise-free measurement, represents the noisy measurement, b represents the dark photons and background noise;
[0026] The system model of non-line-of-sight imaging is simplified to a linear inverse problem form, namely:
[0027] u=Hρ
[0028] Where H represents the forward degradation operator; ρ represents the three-dimensional albedo volume of the hidden scene;
[0029] S2.2. According to the imaging model given in S2.1, the unsupervised unbiased estimator based on Stein's Unbiased Risk Estimation is derived, and the unsupervised denoising loss function is derived. The specific function is expressed as:
[0030]
[0031] in, represents the unsupervised denoising loss function based on Stein's Unbiased Risk Estimation; i∈{0,…,I}, g∈{0,…,G}, I, G represent the number of transients and the number of sampling operators respectively; s represents the number of scanning points; t represents the time dimension, F φ represents the denoiser network; ε is a positive number; e is a binary vector;
[0032] S2.3. Use partial convolution to build a network model based on the encoder-decoder structure, where the partial convolution operator is expressed as:
[0033]
[0034] Among them, M represents the mask; x represents the input; W represents the convolution kernel parameter; b represents the bias, and x' represents the output;
[0035] S2.4. Construct a Unet-like network that combines physical priors and attention mechanism. Use the traditional algorithm LCT to perform domain conversion and initial reconstruction on the denoised transient to obtain value domain information. Then use the reconstruction network to restore the zero domain information. Add the restored zero domain information and value domain information to complete the hidden scene reconstruction experiment.
[0036] Preferably, the solution ③ specifically includes the following contents:
[0037] 3.1) Select a fully sampled transient state and a sampling matrix, extract data from the fully sampled transient state according to the sampling matrix to form under-sampled data;
[0038] 3.2) Fill the undersampled data to the full sampling grid by zero filling method, input it and the mask into the denoiser network to obtain the initial denoised transient, and then multiply it with the mask to remove the artifacts introduced by the network to obtain the denoised undersampled transient;
[0039] 3.3) The denoised undersampled transient is initially reconstructed using the traditional LCT algorithm to obtain the range information of the true value reconstruction result, the range information of the true value reconstruction result is input into the reconstruction network, and the output of the reconstruction network is combined with the range information to obtain the albedo reconstruction volume of the hidden scene;
[0040] 3.4) observe the albedo reconstruction volume through the fully sampled forward operator, and then select a sampling matrix different from that in step 3.1) and multiply it with the new fully sampled observation to obtain a new undersampled transient;
[0041] 3.5) Repeat step 3.3) with the newly obtained undersampled transient in step 3.4) to obtain a new albedo reconstruction volume;
[0042] 3.6) The reconstruction network in step 3.3 is supervised by a measurement consistency loss function and a virtual scanning loss function, wherein the measurement consistency loss function is expressed as:
[0043]
[0044] Among them, L MC represents the measurement consistency loss function; Represents the sampling matrix M g The obtained undersampled transient; H g Representation and sampling matrix M g The corresponding forward operator; f θ Represents the traditional algorithm LCT and the reconstruction network F θ The reconstruction modules composed of
[0045] The virtual scanning loss function is expressed as:
[0046]
[0047] Among them, L VS is the virtual scanning loss function; H k Representation and sampling matrix M k The corresponding forward operator.
[0048] Preferably, S4 includes the following contents:
[0049] S4.1. Build an optimizer and gradient update strategy based on the Pytorch framework, and use the undersampled non-viewing reconstruction simulation transient data set constructed in S1 to train the network;
[0050] S4.2, use the unsupervised denoising loss function based on Stein's Unbiased Risk Estimation in S2 to train the denoiser network;
[0051] S4.3. Freeze the denoiser network weights and use the virtual scanning strategy proposed in Scheme ③ to train the reconstruction network proposed in S2.4.
[0052] Compared with the prior art, the present invention has the following beneficial effects:
[0053] (1) We propose an unsupervised non-line-of-sight imaging framework that can learn implicit priors from arbitrary undersampled transient measurements, overcoming the reliance on paired data that are difficult to obtain and align.
[0054] (2) We introduce a virtual scanning strategy that enables the network to learn in both value space and null space, thereby achieving high-quality reconstruction and extending non-line-of-sight imaging to realistic scenarios with irregular relay surfaces.
[0055] (3) The present invention introduces a SURE-based denoiser, which is an unsupervised physics-guided module that incorporates a physical model of low-photon time-resolution detectors to enhance the robustness of the present method to noise.
[0056] (4) The present invention evaluates the method on simulated data, publicly available data, and captured real data, demonstrating superior reconstruction quality and inference speed significantly faster than current state-of-the-art methods. BRIEF DESCRIPTION OF THE DRAWINGS
[0057] Figure 1 The unsupervised under-sampling non-viewing-area imaging system model based on virtual scanning constructed in Example 1 of the present invention;
[0058] Figure 2 It is an overall framework diagram of the unsupervised under-sampling non-viewing-area imaging method based on virtual scanning proposed in Example 1 of the present invention;
[0059] Figure 3-4 This is a diagram showing the effect of testing the method of the present invention and the traditional reconstruction method on a data set as proposed in Example 2 of the present invention. DETAILED DESCRIPTION
[0060] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.
[0061] Embodiment 1:
[0062] See also Figure 2 The present invention proposes an unsupervised under-sampling non-viewing area imaging method based on virtual scanning, comprising the following steps:
[0063] S1. Constructing an under-sampled non-line-of-sight reconstruction simulation transient dataset: Using a public transient renderer and ShapeNet3D object data to construct a simulation transient dataset, adding noise and time jitter characteristics according to the non-line-of-sight imaging single-photon detector imaging model, and generating an under-sampled non-line-of-sight transient dataset according to the set scanning pattern;
[0064] S2. Build a network framework: Design a denoiser network F with an encoder-decoder structure consisting of partial convolutions φ , a reconstruction network F that combines physical priors θ ,The two networks are connected to achieve the reconstruction of hidden scenes from irregular undersampled transients;
[0065] S3, design scheme and build model: design a non-viewing reconstruction scheme based on the under-sampled non-viewing reconstruction simulation transient data set and network framework described in S1-S2;
[0066] S4, training unsupervised non-view reconstruction framework: using the deep learning Pytorch framework to train the unsupervised denoiser and virtual scan reconstruction network model constructed in S1 and S2 in stages, repeatedly traversing the simulated transient data set constructed in S4 until the model converges;
[0067] The above method further includes the following contents:
[0068] 3.1 Denoiser
[0069] SURE-based denoiser. The SURE-based denoiser consists of two parts: the SURE loss function used to supervise the denoiser training and the network modules that constitute the denoiser itself.
[0070] The denoiser can be trained by end-to-end supervised training using noise-free and noisy data pairs. The mean square error loss (MSE) is generally used for supervised training:
[0071]
[0072] In the unsupervised case, it is not possible to obtain a noise-free transient dataset {u j} is used as supervision. By approximating the loss function based on SURE, an unbiased estimate of the above supervised loss can be obtained, so that denoiser training can be achieved in the absence of supervision.
[0073] Existing SURE-based work has considered Gaussian noise models, Poisson noise models, and Gaussian-Poisson mixed noise models, but lacks algorithms suitable for non-line-of-sight imaging. In this regard, the noise model of non-line-of-sight single-photon detector imaging is considered.
[0074]
[0075] where u represents the noise-free measurement, represents the noisy measurement, and b represents the dark photons and background noise.
[0076] Substituting the above noise model into the derivation process, we can obtain the following unsupervised SURE denoising loss suitable for non-viewing area imaging:
[0077]
[0078] in, represents the unsupervised denoising loss function based on Stein's Unbiased Risk Estimation; i∈{0,…,I}, g∈{0,…,G}, I, G represent the number of transients and the number of sampling operators respectively; s represents the number of scanning points; t represents the time dimension, F φ represents the denoiser network; ε is a positive number; e is a binary vector;
[0079] In order to make full use of the spatial correlation of undersampled transients, this method restores the undersampled transient data to the full sampling grid through zero padding to reconstruct its spatial correlation. At the same time, considering that the ordinary convolution layer regards all voxel values as valid, the ordinary convolution layer will introduce zero padding information, resulting in errors.
[0080] Therefore, this method constructs a network of encoder-decoder structure composed of partial convolutions, which adjust the part involved in the calculation according to the input mask, expressed as:
[0081]
[0082] Among them, M represents the mask; x represents the input; W represents the convolution kernel parameter; b represents the bias, and x' represents the output.
[0083] After passing through the denoising network, this method multiplies the mask by the initial denoising result to remove artifacts introduced by the network at non-scanned locations.
[0084] 3.2 Virtual Scanning Training Strategy
[0085] The non-line-of-sight imaging forward model can be approximately simplified to a linear model:
[0086] u=Hρ
[0087] where u represents the noise-free measurement, H is the forward degradation operator, and ρ is the 3D albedo volume of the hidden scene.
[0088] According to the zero-domain decomposition theorem, any albedo volume ρ can be decomposed into:
[0089]
[0090] in, is the pseudo-inverse operator of H. The first term ρ1 on the right is called the range information of ρ (D r (ρ)), the second term ρ2 is the zero-domain information of ρ (D n (ρ)).
[0091] Standard unsupervised training generally uses measurement consistency loss to supervise network training, expressed as:
[0092]
[0093] Among them, H is the forward degeneration operator, f θ To reconstruct the network, u is the transient measurement. However, this loss function can only enable the network to restore the value range information of the target, and cannot accurately restore the zero domain information. Specifically, this method selects the LCT algorithm as the pseudo-inverse operator, and the output can be uniformly expressed as D r (ρ)+v n , where D r (ρ) is the range information restored by the traditional algorithm LCT, v n The zero-field information is recovered for the network. In Theorem and Hv n = 0, there are multiple output results that satisfy the measurement consistency loss constraint, namely:
[0094]
[0095] in, And Hv n = 0 and. Therefore, the measurement consistency loss function makes f θ Uniquely recover the true value result.
[0096] In this regard, this method proposes a virtual scanning strategy to enable the network to learn the ability to restore the null space. Specifically, the network is first reconstructed from Reconstruct the 3D albedo volume ρ (1) Then select and obtain Different forward degradation operators H k , here, in order to simplify the calculation, H k Decomposed into H r ⊙M k , through H k P (1) Implement virtual scanning to obtain new transient data u kr At this time, the same virtual scanning network is used to obtain a new three-dimensional resolution volume ρ (2) The virtual scanning loss requires ρ (1) =ρ (2) Under this loss constraint, according to the zero-field value decomposition theorem, the network needs to recover ρ (1) The null space information D n (ρ (1) ) to perfectly reconstruct ρ (1) ,Right now:
[0097] ρ (1) =D r (ρ (1) )+D n (ρ (1) )
[0098] During training, by traversing different forward degradation operators to enhance the network's robustness to different scanning patterns, the network can acquire the ability to recover null space information, thereby achieving high-quality 3D reconstruction.
[0099] 3.3 Training Model
[0100] The method of the present invention is implemented using PyTorch and uses the Adam optimizer with a weight decay of 10 -8 In the first stage, the SURE-based denoising model F φ The training was performed for 40 cycles with a batch size of 4. The initial learning rate was set to 1×10 -3 , and reduced it by a factor of 10 in the 30th cycle. Subsequently, in the second stage, the virtual scan reconstructs the network F θ The training was performed for 20 epochs with a batch size of 2 and an initial learning rate of 5×10 -4 , and reduce it by 10 times in the 10th cycle. In each cycle, the present invention randomly selects 40 complete simulated transient signals for each relay surface, and extracts signals from them to generate irregular undersampled transients for training. All models are trained on 2 NVIDIA 3090 GPUs, and the total time is nearly 40 hours. Regarding the loss function, the hyperparameters ε, β, and b are set to 0.1, 0.001, and 4, respectively.
[0101] Embodiment 2:
[0102] Based on Example 1, but with the following differences:
[0103] The method of the present invention is compared with three traditional direct reconstruction algorithms (LCT, FK, RSD), two learning-based algorithms (unsupervised NeTF, supervised USM) and one iterative algorithm (CC-SOCR). Since LCT, FK, RSD and USM are designed for dense regular scanning, the present invention uses a zero-filled version of irregular undersampled transients as input. NeTF and CC-SOCR are able to directly input irregular undersampled transients. For a fair comparison, the present invention also applies a SURE-based denoiser to all compared methods for pre-denoising. As an exception, the present invention does not apply a pre-denoising step to CC-SOCR because there is no obvious improvement in the results of CC-SOCR reconstruction. The reason is that the powerful CC-SOCR regularization inherently performs denoising to some extent.
[0104] We first tested our method on publicly available real datasets and then on captured data. Without proper regularization, direct reconstruction methods such as LCT and RSD exhibit severe aliasing artifacts due to undersampling. FK is not able to recover deep neural networks due to its sensitivity to depth. Figure 3The distal structure shown. NeTF tends to generate blurred shapes due to its difficulty in utilizing the limited information in the IUT. For the case of irregular undersampling, USM is almost overwhelmed by aliasing artifacts, but can recover acceptable results in the case of regular undersampling (e.g. Figure 4 ). CC-SOCR is able to recover the main object structure in all test cases, but loses detailed structure at farther depths (such as "Man Deer", "Letter", "Deer" and "Teaser"), and performs poorly on the "Dragon" object with smooth material or the letter "SU" object with anti-reflective material. The proposed method achieves the best quality across a variety of objects and sampling modes. In short, despite being trained only on a simple alphanumeric dataset and blinds-like relay surfaces, the proposed method generates stable results on real-world data with a variety of attributes and relay surfaces. It significantly outperforms the range space solver LCT, and the virtual scan reconstruction network proposed by the method successfully removes aliasing artifacts while preserving the scene structure. The excellent generalization ability highlights the effectiveness of the proposed method in learning out-of-domain spatial information.
[0105] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.
Claims
1. An unsupervised under-sampling non-viewing-area imaging method based on virtual scanning, characterized in that: The specific steps include: S1. Construct an undersampled non-line-of-sight reconstruction simulation transient dataset: Use the public transient renderer and ShapeNet 3D object data to construct a simulation transient dataset, add noise and time jitter characteristics according to the non-line-of-sight imaging single-photon detector imaging model, and generate an undersampled non-line-of-sight transient dataset according to the set scanning pattern; S2. Build a network framework: Design a denoiser network with an encoder-decoder structure consisting of partial convolutions and a reconstruction network combined with physical priors, connect the denoiser network and the reconstruction network to reconstruct hidden scenes from irregular undersampled transients; S3, design framework, build model: design a non-line-of-sight reconstruction scheme based on the under-sampled non-line-of-sight reconstruction simulation transient data set and network framework described in S1-S2. The scheme specifically includes the following contents: ① Undersampled transient denoising: The captured irregular undersampled transient measurements are filled into the full sampling grid by zero-filling method, and the captured irregular undersampled transient measurements and the mask are input into the denoiser network to obtain the initial denoised transient, which is then multiplied by the mask to remove the artifacts introduced by the network and obtain the denoised undersampled transient. ②Hidden scene reconstruction: The denoised undersampled transient is initially reconstructed using the traditional LCT algorithm to obtain the value domain information of the true value reconstruction result, which is input into the reconstruction network to obtain the zero domain information of the true value reconstruction result. The value domain information is combined with the zero domain information to obtain the albedo reconstruction volume of the hidden scene. ③ Design virtual scanning training strategy: Based on the principle of zero-value domain decomposition, design an unsupervised virtual scanning training strategy and its loss function to enable the neural network to learn the ability to restore zero-domain information; S4. Training the unsupervised non-line-of-sight reconstruction framework: Use the deep learning Pytorch framework to train the constructed network framework and the virtual scanning reconstruction network model in stages, and repeatedly traverse the undersampled non-line-of-sight reconstruction simulation transient data set constructed in S1 until the model converges.
2. The unsupervised under-sampling non-viewing-area imaging method based on virtual scanning according to claim 1, characterized in that: The S1 specifically includes the following contents: S1.1, select and organize ShapeNet 3D object data; S1.2, Modeling single photon detector imaging under non-line-of-sight imaging conditions; S1.3, using the public non-line-of-sight imaging transient measurement renderer to simulate ideal transient measurements based on ShapeNet 3D object data; S1.4, post-processing the simulated ideal transient measurement according to the single-photon detector imaging model to make it close to the fully sampled transient measurement of the real acquisition; S1.
5. Extract undersampled data from the simulated full-sampled transient measurement according to the selected scanning pattern for training and construct a data set.
3. The unsupervised under-sampling non-viewing-area imaging method based on virtual scanning according to claim 2, characterized in that: The S2 specifically includes the following contents: S2.
1. A non-line-of-sight single-photon detector imaging model is given. The model is an imaging model that takes into account a low-photon time-resolution detector. The specific function is expressed as: where u represents the noise-free measurement, represents the noisy measurement, b represents the dark photons and background noise; The system model of non-line-of-sight imaging is simplified to a linear inverse problem form, namely: u=Hρ Where H represents the forward degradation operator; ρ represents the three-dimensional albedo volume of the hidden scene; S2.
2. According to the imaging model given in S2.1, the unsupervised unbiased estimator based on Stein's Unbiased Risk Estimation is derived, and the unsupervised denoising loss function is derived. The specific function is expressed as: in, represents the unsupervised denoising loss function based on Stein's Unbiased Risk Estimation; i∈{0,…,I}, g∈{0,…,G}, I, G represent the number of transients and the number of sampling operators respectively; s represents the number of scanning points; t represents the time dimension, F φ represents the denoiser network; ε is a positive number; e is a binary vector; S2.
3. Use partial convolution to build a network model based on the encoder-decoder structure, where the partial convolution operator is expressed as: Among them, M represents the mask; x represents the input; W represents the convolution kernel parameter; b represents the bias, and x' represents the output; S2.
4. Construct a Unet-like network that combines physical priors and attention mechanism. Use the traditional algorithm LCT to perform domain conversion and initial reconstruction on the denoised transient to obtain value domain information. Then use the reconstruction network to restore the zero domain information. Add the restored zero domain information and value domain information to complete the hidden scene reconstruction experiment.
4. The unsupervised under-sampling non-viewing-area imaging method based on virtual scanning according to claim 3, characterized in that: The above-mentioned plan ③ specifically includes the following contents: 3.1) Select a fully sampled transient state and a sampling matrix, extract data from the fully sampled transient state according to the sampling matrix to form under-sampled data; 3.2) Fill the undersampled data to the full sampling grid by zero filling method, input it and the mask into the denoiser network to obtain the initial denoised transient, and then multiply it with the mask to remove the artifacts introduced by the network to obtain the denoised undersampled transient; 3.3) The denoised undersampled transient is initially reconstructed using the traditional LCT algorithm to obtain the range information of the true value reconstruction result, the range information of the true value reconstruction result is input into the reconstruction network, and the output of the reconstruction network is combined with the range information to obtain the albedo reconstruction volume of the hidden scene; 3.4) observe the albedo reconstruction volume through the fully sampled forward operator, and then select a sampling matrix different from that in step 3.1) and multiply it with the new fully sampled observation to obtain a new undersampled transient; 3.5) Repeat step 3.3) with the newly obtained undersampled transient in step 3.4) to obtain a new albedo reconstruction volume; 3.6) The reconstruction network in step 3.3 is supervised by a measurement consistency loss function and a virtual scanning loss function, wherein the measurement consistency loss function is expressed as: Among them, L MC represents the measurement consistency loss function; Represents the sampling matrix M g The obtained undersampled transient; H g Representation and sampling matrix M g The corresponding forward operator; f θ Represents the traditional algorithm LCT and the reconstruction network F θ The reconstruction modules composed of The virtual scanning loss function is expressed as: Among them, L VS is the virtual scanning loss function; H k Representation and sampling matrix M k The corresponding forward operator.
5. The unsupervised under-sampling non-viewing-area imaging method based on virtual scanning according to claim 4, characterized in that: The S4 includes the following: S4.
1. Build an optimizer and gradient update strategy based on the Pytorch framework, and use the undersampled non-viewing reconstruction simulation transient data set constructed in S1 to train the network; S4.2, use the unsupervised denoising loss function based on Stein's Unbiased Risk Estimation in S2 to train the denoiser network; S4.
3. Freeze the denoiser network weights and use the virtual scanning strategy proposed in Scheme ③ to train the reconstruction network proposed in S2.4.
Citation Information
Patent Citations
Non-field-of-view femtosecond imaging three-dimensional reconstruction method and device based on deep learning
CN111462299A
Three-dimensional information reconstruction method for single photon detection based on end-to-end neural network
CN118071916A
Cited By
Two-stage active steady-state non-vision-field imaging method based on data-model driving
CN119383481A
Self-supervised reconstruction method for human body 5.0 T multi-core 31P magnetic resonance spectrum imaging
CN119762608A
