Adaptive windowing based few-photon non-line-of-sight imaging method

Abstract

The application discloses a kind of few-photon non-visual imaging methods based on adaptive windowing, comprising: using non-coaxial two-dimensional scanning system to collect the photon signal reflected from hidden target, establish the probability model of signal photon and noise photon, analyze its distribution characteristics in time domain;Using space-time correlation, adjacent pixels are combined into pixel blocks, the adaptive window width of each pixel block is determined by matching filtering method, and windowing operation is applied on time domain to separate signal photons from noise photons;Full variable regularization constraint is used to fill the transient data after windowing, and the reconstruction problem is solved by alternating direction multiplier method to obtain the target reconstruction result.The method significantly improves the detection efficiency of sparse photons by using space-time correlated pixel blocks and matching filtering technology, and realizes target reconstruction at very low photon level.

Description

Technical Field

[0001] This invention relates to the field of non-line-of-sight imaging, and in particular to a few-photon non-line-of-sight imaging method based on adaptive windowing. Background Technology

[0002] Non-Line-of-Sight (NLOS) imaging technology can reconstruct targets outside the field of view by analyzing photon information after multiple scattering, and has important application value in fields such as autonomous driving, medical detection, military reconnaissance, disaster relief and rescue, and remote sensing. Recently, many methods for reconstructing NLOS targets have been proposed, such as speckle correlation, thermal imaging, acoustic imaging, Fermat path imaging, occlusion-based imaging techniques, and transient imaging.

[0003] Among these methods, transient NLOS imaging has been widely utilized due to its advantageous 3D reconstruction capabilities. By leveraging the time-of-flight (ToF) information of multiple-scattered photons, transient NLOS imaging can reconstruct hidden targets. However, unlike conventional in-field imaging, NLOS scenes suffer from severe photon attenuation due to multiple scattering, significantly increasing the complexity of signal extraction. In 2012, Velten et al. first achieved NLOS imaging using a streak camera (A. Velten, T. Willwacher, O. Gupta, A. Veeraraghavan, MG Bawendi, and R. Raskar, “Recovering three-dimensional shape around a corner using ultrafast time-of-flight imaging,” Nat. communications 3, 745 (2012).). Despite this groundbreaking success, the low detection efficiency of streak cameras limited image quality. Subsequently, single-photon avalanche diodes (SPADs) have been widely used due to their high internal gain and single-photon sensitivity. For example, O'Toole et al. proposed the Light Cone Transform (LCT) algorithm, which uses a confocal configuration and SPAD for detection (M. O'Toole, DB Lindell, and G. Wetzstein, “Confocal non-line-of-sight imaging based on the light-conetransform,” Nature 555, 338-341 (2018).). Lindell, on the other hand, used a frequency-wavenumber (FK) migration algorithm in the same system configuration (DB Lindell, G. Wetzstein, and M. O'Toole, “Wave-based non-line-of-sight imaging using fast fk migration,” ACMTransactions on Graph. (ToG) 38, 1-13 (2019).). However, since point-by-point scanning is required during imaging, single-point SPAD detection significantly increases the acquisition time, severely impacting imaging efficiency.Recently, a superconducting nanowire single-photon detector (SNSPD) has been used in NLOS scenarios, exhibiting superior detection efficiency compared to traditional detectors and enabling NLOS imaging in the near-infrared and mid-infrared bands. However, its high cost and bulky cooling system limit its integration into compact platforms. Other methods attempt to address acquisition time constraints by encoding single-point SPADs using SPAD arrays or digital micromirror devices (DMDs) for parallel scanning. However, in these non-confocal systems, the geometric mismatch between illumination and detection points leads to a sharp decrease in photon counts at the relay wall edges, often requiring higher laser power to solve this problem. All these methods rely on extending the exposure time for each scan point, employing highly efficient detectors, or excessively increasing laser power to compensate for photon insufficiency in NLOS scenarios, resulting in redundant photon detection and inefficient target reconstruction. Therefore, achieving high-precision NLOS reconstruction under photon-deficient conditions remains a critical and pressing issue.

[0004] Many computational imaging methods have been proposed to improve photon utilization efficiency in NLOS imaging. For example, Feng et al. proposed a light field tomography (LIFT) technique that uses pulsed lasers with an average power of 65 mW to achieve NLOS imaging (X. Feng and L. Gao, “Ultrafast lightfield tomography for snapshot transient and non-line-of-sight imaging,” Nat. communications 12, 2179 (2021).). Similarly, Xu et al. demonstrated an innovative method in their work (F. Xu, G. Shulkind, C. Thrampoulidis, JH Shapiro, A. Torralba, FN Wong, and GW Wornell, “Revealing hidden scenes by photon-efficient occlusion-based opportunistic active imaging,” Opt. express 26, 9945-9962 (2018).), which achieved imaging with only about 69 photons per pixel (PPP) using occluded scenes, comparable to imaging with a PPP of about 1100. Despite these methods, photon scarcity remains a significant challenge, directly leading to poor image quality. First-photon imaging techniques have also been explored and applied to NLOS scenes. Tsai et al. systematically analyzed the geometric constraints of the first-returning photons and proposed a path back-projection method for reconstructing hidden targets (C.-Y.Tsai, KN Kutulakos, SG Narasimhan, and AC Sankaranarayanan, “The geometry of first-returning photons for non-line-of-sight imaging,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, (2017), pp. 7216-7224.).Building on this, Li et al. performed NLOS reconstruction by labeling the first photon data within each time bin (Z. Li, X. Liu, J. Wang, Z. Shi, L. Qiu, and X. Fu, “Fast non-line-of-sight imaging based on first photon event stamping,” Opt. letters 47, 1928–1931 (2022).). Liu et al. further improved this by introducing a distance attenuation constraint, reducing the photon count to only one photon per pixel (J. Liu, Y. Zhou, X. Huang, Z.-P. Li, and F. Xu, “Photon-efficient non-line-of-sight imaging,” IEEE Transactions on Comput. Imaging 8, 639–650 (2022).). However, in NLOS scenarios involving multiple scattering, it is extremely challenging for the detector to distinguish whether the received first photon originates from the target or background noise. This difficulty limits these first-photon methods to reconstructing objects in a single depth plane. Summary of the Invention

[0005] To overcome the shortcomings of existing technologies, this invention provides a few-photon non-line-of-sight imaging method based on adaptive windowing.

[0006] The technical solution to achieve the objective of this invention is: a few-photon non-line-of-sight imaging method based on adaptive windowing, comprising:

[0007] Step 1: Use a non-coaxial two-dimensional scanning system to collect photon signals reflected from the hidden target, establish probability models of signal photons and noise photons, and analyze their distribution characteristics in the time domain.

[0008] Step 2: Utilize spatiotemporal correlation to group adjacent pixels into pixel blocks, determine the adaptive window width of each pixel block using a matched filtering method, and apply a windowing operation in the time domain to separate signal photons from noise photons.

[0009] Step 3: Use total variable regularization constraints to fill in the transient data after windowing, and use the alternating direction multiplier method to solve the reconstruction problem to obtain the target reconstruction result.

[0010] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the method described above.

[0011] A computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the steps of the above-described method.

[0012] A computer program product includes a computer program that, when executed by a processor, implements the steps of the above-described method.

[0013] Compared with the prior art, the significant advantages of the present invention are:

[0014] (1) By using spatiotemporally correlated pixel blocks and matched filtering techniques, this method significantly improves the detection efficiency of sparse photons and achieves target reconstruction at extremely low photon levels; reliable target reconstruction can still be achieved under the condition of 0.02 signal photons per pixel (PPP).

[0015] (2) This method can complete target reconstruction in a very short exposure time; in the actual system, it was verified that the complete target reconstruction was achieved in a very short exposure time of 0.82 seconds, which greatly improved the imaging efficiency. Attached Figure Description

[0016] Figure 1 The diagram shows the schematic of AW-NLOS, where (a) is a schematic of the experimental system for AW-NLOS imaging with a time jitter of 300 ps; (b) is the optical module of the actual system; and (c) is the synchronization circuit module.

[0017] Figure 2 This is a photon counting process involving two echo signals and noise.

[0018] Figure 3 Compare the transient histograms of single pixels and pixel blocks.

[0019] Figure 4 The algorithm flow of AW-NLOS is as follows: (a) acquiring 3D photon transient data, (b) forming pixel blocks using spatial correlation, (c) matching filtering to determine the adaptive window width, (d) windowing in the time domain, (e) using TV regularization constraints for transient completion, and (f) reconstructing the target.

[0020] Figure 5 The transient information and reconstruction results of different windowing methods are compared, where (a) is no windowing, (b) is fixed window width windowing (the window width is twice the system time jitter), (c) global windowing (the window width is the signal pulse width obtained by superimposing the photon information of all pixels), and (d) adaptive windowing.

[0021] Figure 6 To compare the transient information and reconstruction results of detectors with different detection efficiencies using AW-NLOS and FK methods, (a) shows the comparison of transient information and (b) shows the comparison of reconstruction results.

[0022] Figure 7 The transient histograms and reconstruction results of AW-NLOS and FK under different laser powers are compared.

[0023] Figure 8 The effect of time resolution on signal detection is shown in (a) and (b) respectively. (a) is the cumulative histogram of two pulses with a time resolution of 15 ps and (b) is the cumulative histogram of two pulses with a time resolution of 40 ps.

[0024] Figure 9 The single-point photon histograms and reconstruction results of AW-NLOS, FK, LCT, PF and FBP at different time bin widths of 50ps, 20ps, 10ps and 5ps, respectively.

[0025] Figure 10 To compare the transient information and reconstruction results of AW-NLOS with FK, LCT, PF and FBP methods under different exposure time conditions; where (a) is the actual scene and GT; (b) is the transient information under different exposure time conditions; SBR: signal-to-background ratio within the dashed box; (c) is the reconstruction results and corresponding SSIM under different exposure time conditions.

[0026] Figure 11 To compare the reconstruction results of AW-NLOS with FK, LCT, PF and FBP under different exposure time conditions, (a) is the actual scene and GT; the exposure time for each pixel is (b) 6.87ms and (c) 2.29ms.

[0027] Figure 12 To compare the single-point histograms and reconstruction results of AW-NLOS with FK, LCT, PF and FBP at different distances; where (a) is the actual scene; (b) shows the decrease in signal photon number with increasing distance; and (c) shows the reconstruction results and corresponding SSIM at 2m, 3m and 4m. Detailed Implementation

[0028] Non-line-of-sight (NLOS) imaging aims to reconstruct the shape of hidden targets. However, most existing NLOS systems require hundreds or even thousands of photons per pixel to accurately reconstruct the hidden targets, which necessitates very long exposure times and complex hardware configurations. This invention proposes an adaptive windowing-based non-line-of-sight imaging (AW-NLOS) method to achieve high photon utilization reconstruction at the sub-photon level. Specifically, by utilizing spatiotemporally correlated pixel blocks, the detection efficiency of sparse photons is significantly improved; then, the enhanced signal is used to achieve accurate window width estimation through matched filtering; for each pixel of the transient image, an adaptive short-time interval window can be applied to eliminate detection caused by noise; and the transient data is also subjected to total variation (TV) regularization to reconstruct measurements with a high signal-to-background ratio. Experiments show that this method can achieve reliable target reconstruction at the sub-photon level of 0.02 signal photons per pixel (PPP).

[0029] This invention designs a non-coaxial two-dimensional scanning system to achieve photon-efficient NLOS imaging. Figure 1 Figure (a) shows a schematic diagram of the experimental system, in which the non-coaxial 2D scanning system is 7m away from the relay wall, and the hidden target is located 3m away from the relay wall. Unlike traditional coaxial transceiver systems, this invention uses only two galvanometers (transmitter / receiver) to perform raster scanning on the relay wall. In this setup, a slight spatial separation is designed between the illumination point and the receiving field of view to reduce the influence of primary reflection light. The spatial separation interval is controlled at approximately 2cm, which is much smaller than the distance between the hidden target and the relay wall.

[0030] Optical modules of actual systems, such as Figure 1 As shown in (b) above. At the transmitting end, a 1064nm collimated laser with adjustable power and a repetition rate of 5MHz illuminates a transmitting galvanometer, which is used for grating scanning on the relay wall. For the receiving end, an ultrafast-gated SPAD detector (AUREA, SPD_OEM_NIR) is used. This detector is coupled to a multimode fiber and a collimating lens to collect the echo signal from the secondary reflection. A narrowband filter (Thorlabs FLH 1064-3) is also integrated into the transmitting galvanometer to reduce background noise interference to the SPAD.

[0031] To achieve accurate Time-of-Flight (ToF) measurement, a method was developed. Figure 1The synchronization circuit module shown in (c) is capable of precisely timestamping the laser emission and photon detection times. Simultaneously with each photon emission, the laser emits two identical synchronization signals: one directly sent to the Time-Correlated Single Photon Counting (TCSPC) system (Time Tagger Ultra, Swabian Instruments) as a start signal, and the other transmitted to the SPAD's gated port via a delay-compensated Field Programmable Gate Array (FPGA). When a photon reflected from the target is detected by the SPAD, the TCSPC system generates and records a corresponding stop signal, thereby calculating the Time-of-Flight (ToF) value. Furthermore, to address the synchronization and storage issues of the pixel-by-pixel scan signal, a National Instruments data acquisition device (NI-DAQ, 134 USB-6343) is used to control the scanning positions of the two galvanometers separately. After completing the scan of each point, it sends a restart signal to the TCSPC to ensure that the information and storage at each point are perfectly matched. The system's timing jitter includes laser pulse width, SPAD jitter, TCSPC jitter, and circuit delay, with a measured value of 300 ps. Figure 1 As shown in Figure (a).

[0032] In NLOS imaging systems, the received signal, including signal photons and noise photons, is affected by various factors. Due to the propagation characteristics of light, signal photons exhibit a non-uniform Gaussian distribution in the time domain during short exposure times. In contrast, noise photons, mainly composed of background noise and dark noise, are uniformly distributed in the time domain, belonging to a uniform Poisson process. The fundamental difference in the temporal distribution patterns of signal and noise photons forms the basis of the adaptive windowing technique of this invention, enabling effective separation of the two to enhance image quality.

[0033] In the NLOS imaging system proposed in this invention, the signal received by the detector is affected by various factors. At the operating wavelength... Under these conditions, the number of signal photons detected by the detector per second Represented as:

[0034] (1)

[0035] in, It is the average optical power of the pulsed laser. It is the Bronk constant, and c is the speed of light. , These are the reflectivities of the target and the relay wall, respectively. It is the angle between the normal of the relay wall and the laser beam. It is the area of ​​the hidden target. It refers to the field of view (FOV) range on the relay wall. It is the area of ​​the detector's receiving field of view. It is the transmittance of the receiving system. It is the transmittance of air. It is the distance between the optical system and the relay wall. It is the distance between the target and the relay wall.

[0036] In addition to signal photons, the NLOS imaging system also receives noise photons composed of background noise and dark noise, receiving a certain number of noise photons per second. It can be represented as:

[0037] (2)

[0038] in, It is the bandwidth of an ideal bandpass filter. It is solar spectral irradiance. It is the angle of incidence of the sun. It is the detector's dark count rate (DCR).

[0039] When the system detection efficiency (SDE) of the detector is At that time, the photon flux reaching the detector It can be represented as:

[0040] (3)

[0041] Considering the acquisition mechanism of TCSPC, the arrival time of photons acquired by SPAD can only be recorded discretely in different time bins, with a width of The average number of photons detected within the nth time bin can be expressed as:

[0042] (4)

[0043] in, It is a time bin index. It is the largest index of the time bin.

[0044] Due to the influence of the detection mechanism, the photon signal detected by SPAD follows a Poisson distribution, and the probability of observing k photon times within the nth time bin is... It can be represented as:

[0045] (5)

[0046] Furthermore, considering the dead time of SPAD In the The probability of detecting a signal photon within a time bin It can be represented as:

[0047] (6)

[0048] Detection efficiency of noisy photons It can be represented as:

[0049] (7)

[0050] For a single scan point, there is no obvious difference between the detection of signal and noise, making it difficult to distinguish them. To separate signal and noise, prior knowledge of the probability models that differentiate between signal and noise during the detection process can be utilized. Specifically, as shown in Equation (1), the signal process is related to the illumination pulse laser. When multiple pulses irradiate the same scan point, the detection will cluster together near the true depth. The noise signal mainly includes background noise and dark noise, which belong to homogeneous Poisson processes and do not exhibit clustering effects in the time domain. The corresponding process is as follows: Figure 2 As shown, the non-uniform Poisson process is represented by the yellow line, which is the sum of the non-uniform signal process and the uniform background noise process. Signal photons are represented by blue arrows, while noise photons are represented by red arrows.

[0051] Since noise photons are uniformly distributed in the time domain, spatiotemporal correlation can be used to largely separate noise photons from signal photons. This enables the system to achieve efficient NLOS imaging of photons.

[0052] In the method of this invention, an NLOS imaging system is used to acquire a complete three-dimensional (3D) transient image, including... Each pixel has multiple time bins, such as... Figure 4 As shown in (a) above. Since signal detection is more likely to cluster than noise detection, a straightforward approach is to identify the peaks of the clusters. One way to define a detection cluster is by the duration of the selection window. A well-placed window should be large enough to capture most or all of the signal photons while rejecting a large number of noisy photons. Unlike single-photon imaging systems, each scan point in an NLOS imaging system detects the superposition of echoes from all points on the hidden target, such as... Figure 1 As shown in (a) above, the window width for clustering signals detected at different angles. Time is different from time. Therefore, a fixed window is not suitable for all scan points, and exploring an adaptive window width is of great significance for acquiring signal photon events in NLOS imaging.

[0053] In the NLOS imaging system, because photons undergo three reflections—"relay wall-target-relay wall"—the photon flux obtained from each scanning point is... Very faint, and the location of clusters is difficult to identify. For example... Figure 3 As shown, the transient response of a single pixel exhibits sparsity, making it difficult to determine the start and end positions of the window. To improve the signal detection probability, it is necessary to form pixel blocks in cooperation with surrounding pixels. In the method of this invention, pixel blocks are formed by utilizing the spatiotemporal correlation of surrounding scan points, with the nearest 4×4 pixels grouped together. The photon count within a pixel block can be represented as:

[0054] (8)

[0055] in, It is a pixel block index. Probes from similar adjacent pixels can help amplify low signal levels by making signal probe clusters more prominent, such as... Figure 3 As shown. Therefore, by employing spatial correlation, the window for each pixel block becomes more reliable and useful.

[0056] To further determine the window size for each pixel block, this invention preprocesses the photon counting data using a matched filtering method. Matched filtering is typically used to analyze the similarity between two time-domain signals, thereby obtaining a fine statistical distribution of photon arrival times. In the acquired transient data, background noise is uniformly distributed over time, while the signal intensity follows a Gaussian distribution and exhibits different clustering characteristics in the time domain. Therefore, the purpose of matched filtering is to eliminate background noise and efficiently obtain the duration of the window for each pixel block from a small number of echo photons. The Instrument Response Function (IRF) is used as the template for matched filtering. The magnitude of the IRF is affected by several components of the system, such as the laser pulse width, detector jitter, and TCSPC jitter. It is obtained through prior measurement, such as Figure 1 As shown in (a). Each pixel block The matched filtering results are as follows Figure 4 As shown in (c), it can be represented as:

[0057] (9)

[0058] Larger, larger pixel blocks The higher the probability that the photon counting event originates from a hidden target, the more likely it is to occur. Subsequently, it can be utilized... The window width of the pixel block is calculated using the extreme points and the system's half-maximum bandwidth. :

[0059] (10)

[0060] When the number of echo photons from a target is sparse, it is difficult to determine the start and end times of the window using only the photon count histogram. However, spatiotemporal correlation methods based on IRF matching can utilize the clustering effect formed by a small number of photons to obtain a more refined statistical curve of photon arrival times, thereby further ensuring the selection of adaptive window width.

[0061] Based on the previously calculated adaptive window width, windowing can be applied to the original transient information in the time domain. As described in Section 2.2, signals tend to cluster together more easily than noise. Therefore, in cases of low photon counts, windowing can better separate signal photons from noise, leading to improved reconstruction results. Figure 4 As shown in (d), the window opening process can be represented as:

[0062] (11)

[0063] in, It is the beginning of the window and .

[0064] Selecting a window width based on signal features can separate the signal from noise while preserving the integrity of the target signal as much as possible. This not only reduces the impact of noise on subsequent target reconstruction but also reduces memory requirements and computational complexity. However, since the adaptive windowing operation forcibly sets photons outside the window to zero, it disrupts the spatiotemporal correlation of transient data. To address this issue, TV regularization constraints are introduced to fill in the transient data after windowing, such as... Figure 4 As shown in (e). Optimized transient information. It can be represented as:

[0065] (12)

[0066] in, It is a regular parameter associated with the measured signal-to-noise ratio (SNR) level.

[0067] After obtaining optimized transient information, the hidden target is reconstructed. In the confocal NLOS scene, the axial position of the relay wall is set to 0. At the scan point The transient information captured at that point can be represented as:

[0068] (13)

[0069] Assuming diffuse reflection occurs in the scene, where Points on the hidden object The distance to the laser or detector. This represents the attenuation of photons scattered by the relay wall after two reflections. Let be the time it takes for a photon to travel. (Dirac function) The surface of a four-dimensional spacetime supercone can be used... This function models the light propagation process from the relay wall to the hidden target and back to the relay wall. Equation (13) is then modified... Variable substitution:

[0070] (14)

[0071] This equation is essentially a 3D convolution process: .in, Represents 3D convolution kernel, and These represent resampling operations along the z-axis and t-axis, respectively. From transient information... Solving in the middle The problem can be formulated as solving regularized least squares. Let... It is with the nuclear The associated 3D convolution operation can be represented by a matrix as follows:

[0072] (15)

[0073] in, This is the regularization parameter, which serves as a weighted constant to balance the trade-off between the data fidelity term and the regularization term. Then, the Alternating Direction Method of Multipliers (ADMM) is employed to solve this minimization problem. The final reconstruction result is as follows... Figure 4 As shown in (f).

[0074] NLOS imaging can reconstruct hidden targets using photons that have undergone multiple scattering. However, during detection, the detector cannot distinguish whether the received photons originate from the target or from background noise during the acquisition process. In this experiment, a pulsed laser with an average power of 141mW was used to scan a relay wall in a 64×64 grid, with an exposure time of 1ms per point. For ease of visualization, Figure 5Figure (a) shows the captured transient information unfolded along a one-dimensional image. As shown in Figure (a), although the target signal exhibited by the original transient data is distinguishable, background noise composed of ambient light and dark noise accumulates at each scan point. Therefore, the reconstructed 3D voxel space is filled with noise. Windowing methods have been widely used in LOS single-photon imaging to separate signal and noise. Most methods typically employ a fixed window width twice the system time jitter. However, when the fixed windowing method is applied to an NLOS scene with a system time jitter of 300 ps, ​​it can be observed that although noise is effectively suppressed, significant signal loss is introduced, resulting in incomplete reconstruction of the target structure, such as... Figure 5 As shown in (b) of the figure. This limitation stems from the fundamental difference between LOS and NLOS imaging configurations. In LOS imaging, the laser scans the object point-by-point. In non-view-of-sight (NLOS) imaging, photons undergo three diffuse reflections, meaning that each scan point collects information from multiple voxels in the hidden scene. Therefore, the window width that effectively captures the full signal in LOS imaging cannot be directly applied to NLOS scenes. To retain more signal information, researchers proposed an alternative method that aggregates ToF information across the entire pixel. Due to the clustering effect, the accumulated signal forms a distinct peak, thus allowing the determination of an optimal window width of 4050 ps. Applying this full-pixel integration method to NLOS data, the reconstruction results are shown in the figure. Figure 5 As shown in (c) above. Although the method successfully reconstructs the complete target structure, the boundaries of the target signal in the transient image are not sufficiently refined, and noise still exists around them. This is clearly visible in the reconstructed 3D voxels.

[0075] To maintain the structural integrity of the target in NLOS reconstruction while maximizing noise suppression, an adaptive windowing strategy is proposed, which dynamically adjusts the window width at each scan point. Compared to fixed windowing, the method of this invention preserves the complete target geometry in the front view. Compared to global windowing, it effectively eliminates transient artifacts and noise in 3D voxels. This signal-to-noise separation at the data end prevents noise-induced reconstruction artifacts, thereby achieving high SNR target reconstruction under sparse signal conditions.

[0076] Not all photons incident on a SPAD surface trigger a detectable avalanche event. The probability that a photon will trigger a detectable avalanche is called the photon detection efficiency (PDE), a key parameter characterizing SPAD performance. Under the same conditions, SPADs with lower PDE capture fewer photons. The resulting fewer photon data significantly increases the complexity of subsequent reconstruction. In addition to this problem, SPADs cannot distinguish whether detected photons originate from the target or background noise. When using low-PDE detectors, these limitations severely degrade the performance of existing NLOS reconstruction methods, exacerbating the reconstruction challenges. Figure 6 As shown in (a), the left figure displays the transient information of the AUREA series SPADs with 20% PDE, while the right figure displays the transient information of the MPD series SPADs with less than 1% PDE. Under experimental conditions of a 287mW laser and an exposure time of 1ms per scan point, the AUREA detector obtains a photon information PPP of 0.7246, while the MPD only reaches 0.0217. For transient data with X×Y scan points, the PPP calculation formula is as follows:

[0077] (16)

[0078] In this experiment, the ability of conventional methods and the method proposed in this invention to process data acquired by low-PDE detectors was evaluated. Under high-power (2.59W) illumination, the reconstruction results of a high-PDE AUREA detector were used as the ground truth (GT). Figure 6 As shown in (b), when the laser power is reduced to 141 mW, the reconstruction quality of both detector types decreases due to the reduced photon count. In particular, the reconstruction result of the MPD detector is almost completely submerged in noise and becomes difficult to distinguish. The SSIM of the reconstruction result and the front view of the GT is compared here. The SSIM is calculated as follows:

[0079] (17)

[0080] in, , , This corresponds to the mean, standard deviation, and covariance between x and y. and It is a constant used to maintain stability. And , .and It is the dynamic range of pixel values.

[0081] When using MPD detectors with photon counts (PPP) as low as 0.02, conventional methods cannot resolve target structures, resulting in a SSIM value of approximately 0.5. However, the method proposed in this invention effectively separates target structures from background noise by leveraging the inherent spatiotemporal correlation within pixel blocks to amplify weak signal clusters and applying adaptive window widths tailored to the temporal characteristics of each block. At the same laser power, the reconstructed SSIM reaches 0.7, even surpassing the results obtained using higher PDE detectors. This significant improvement demonstrates that a combined strategy of spatially correlated signal amplification and precise noise suppression through temporally adaptive windowing enables robust reconstruction even in extremely photon-sparse conditions. This experiment proves that the proposed method can adapt to low-photon environments caused by extremely low instrument detection efficiency, further reducing the dependence of NLOS imaging technology on high-cost devices.

[0082] In the field of NLOS imaging, the selection of laser power has always been a key issue. On the one hand, higher laser power can significantly increase the number of signal photons, thereby improving image quality. However, this method usually comes at the cost of increased power consumption and reduced security. On the other hand, while lower laser power can reduce power consumption and increase security, it may lead to insufficient signal photon count, thus affecting the imaging results. Therefore, how to achieve high-quality NLOS imaging at low laser power has become a current research focus. This paper compares the reconstruction capabilities of the method of this invention with traditional methods for objects of varying complexity. In the experiment, different laser powers were selected to scan a 2m × 2m area on a relay wall, with an exposure time of 1ms per point. The hidden target was 3m away from the relay wall. Figure 7 As shown, four hidden target scenes with different complexities are illustrated. Here, the FK reconstruction results with a laser power of 2.59W under the same scanning conditions are used as the ground truth (GT).

[0083] At a laser power of 63mW, Figure 7The single-point histogram shows only 2-3 signal photons per detection, with a signal PPP as low as 0.8. At such low photon counts, traditional FK methods are almost unable to reconstruct the target's outline. As the number of targets increases, the photon distribution for each object becomes sparser, posing a greater challenge to the reconstruction algorithm. For three objects, traditional methods failed to reconstruct the "S" structure, while the proposed method clearly identifies it. In more complex scenes, such as four objects, traditional reconstruction produces indistinguishable results, with an SSIM of approximately 0.21, while the proposed method effectively separates the target from noise. When the laser power is reduced to 42mW, traditional methods struggle to distinguish letter shapes in multi-target scenes, with SSIM values ​​below 0.19. In contrast, the proposed method clearly distinguishes even closely spaced structures such as "U" and "T," achieving reconstructions with an SSIM higher than 0.75. The proposed method demonstrates significant advantages under low laser power conditions, clearly distinguishing target structures from noise even at extremely low photon counts, and maintaining high-precision reconstruction in complex multi-target scenes.

[0084] For each scan point, the quality of the photon histogram is affected not only by the laser power but also by the bin width. The decision was made. For example... Figure 8 As shown in (a), at a time resolution of 15 ps, the target echo signals from the two periods are scattered across different time bins. This dispersion affects signal accumulation, causing them to be masked by noise and rendering the target indistinguishable. Conversely, at a time resolution of 40 ps, ​​the target echo signals from the two periods converge within the same time bin. This convergence allows the signal accumulation count to remain above the noise threshold, thereby enhancing target discriminability. However, while improving signal discriminability, this introduces a larger ranging error and reduces resolution.

[0085] To verify the adaptability of AW-NLOS under low-bin conditions where photon aggregation is unlikely, the letter "U" was scanned at a different time bin width with a scan time of 1 ms per point at a laser power of 500 mW. Figure 9 Histograms and reconstruction results for a representative point under different time bin conditions are shown. As the bin width decreases, the number of signal photons decreases, making the accumulation of signal photons increasingly difficult. When the bin width is 50 ps, ​​FK, LCT, and PF can completely reconstruct the target structure, but with significant noise. However, when the bin width decreases to 10 ps, ​​signal photons are difficult to accumulate, and the target structure reconstructed by FK is almost indistinguishable from noise. In contrast, the method of this invention effectively separates signal and noise at 50 ps and 10 ps resolutions, achieving high-quality reconstruction. It is worth noting that, as... Figure 9As shown in (a), even with a bin width of 5 ps, the histogram does not exhibit clear signal clustering, with a maximum value of only 2 photons, making it difficult to distinguish from background noise at the single photon level. Under such conditions, other traditional methods cannot reconstruct the letter structure. However, AW-NLOS can maintain the structural integrity of the target and avoid noise interference, resulting in significantly better reconstruction quality than traditional methods. Experimental results demonstrate that this method can adapt to hardware devices with extremely small time bin widths. This capability offers enormous potential for high-precision imaging.

[0086] In NLOS imaging scenarios, the relay wall needs to be scanned point by point. To achieve rapid imaging, reducing the exposure time per point is a common strategy. However, shorter exposure times result in fewer signal photons collected per point, making the signal easily overwhelmed by background and dark noise. This significantly increases the burden on subsequent reconstruction algorithms. This paper demonstrates that the AW-NLOS method proposed in this invention can successfully reconstruct hidden targets even at extremely fast acquisition speeds. In the experiment, a 2.59W pulsed laser was used to scan a 64×64 grid on the relay wall. Exposure times per point were 0.5ms, 0.4ms, and 0.2ms, corresponding to total acquisition times of 2.05s, 1.64s, and 0.82s, respectively.

[0087] To quantitatively validate the improvement of the proposed adaptive windowing method over traditional methods under low exposure time conditions, two key metrics were compared: the signal-to-background ratio (SBR) of transient information and the SSIM of the reconstructed front view. SBR is defined as the ratio between the maximum peak photon count of the signal and the background photon count.

[0088] (18)

[0089] in, It is the peak photon count of the signal. This refers to the noise photon count. To determine the ground truth (GT), the Power Process (PF) algorithm, which provides the best reconstruction quality, is used to reconstruct the transient information acquired at 5ms exposure times per point. For example... Figure 10As shown in (b), the acquired transient image is unfolded along one dimension for visualization. At an exposure time of 0.5 ms per point, the measured SBR is 5.1663; however, after applying the windowing method, the signal and noise are effectively separated, increasing the SBR to 10.1895. Regarding reconstruction performance, at single-point exposure times of 0.5 ms and 0.4 ms, the conventional method can completely reconstruct the target structure, but the resulting image quality is poor and noisy, with SSIM consistently below 0.5. In contrast, the reconstruction results of the method of this invention show very little noise, with SSIMs reaching 0.6638 and 0.6174, respectively. When the scan time is reduced to only 0.2 ms per point, the imaging accuracy decreases significantly under high background noise conditions. The horizontal portion of the "L" structure in the reconstruction results of the conventional method is significantly blurred, accompanied by increased noise levels, resulting in SSIMs all below 0.3. Conversely, AW-NLOS maintains a complete and clear "L" structure, achieving an SSIM exceeding 0.4302.

[0090] Furthermore, this invention was validated on the public Stanford dataset. To reduce computational demands, an undersampled version of the dataset was used, selecting a 64×64 pixel grid within a 2m×2m scan area. Two sets of data with shorter exposure times per point (6.87ms and 2.29ms) were selected for analysis. GT, derived from PF reconstruction using data with a single-point exposure of 41.4ms, provided the best reconstruction fidelity. Under the condition of a 6.87ms single-point exposure time, the target structure reconstructed by the conventional method was overwhelmed by noise, blurring key morphological details. In contrast, AW-NLOS achieved superior feature resolution, clearly depicting the skull and chest regions of the dragon with high structural integrity, achieving an SSIM of 0.7892. Under the more extreme condition of a 2.29ms single-point exposure time, the conventional method essentially failed to recover the target geometry. The method of this invention consistently maintained high-quality reconstruction, a result that ultimately demonstrates the superior performance of AW-NLOS under photon-deprived conditions.

[0091] The method of this invention fundamentally advances low-photon NLOS imaging, achieving precise signal-noise differentiation under photon-scarce conditions. This capability facilitates high-fidelity reconstruction of hidden targets and provides excellent noise suppression, achieving significantly higher SSIM at sub-millisecond exposure times.

[0092] According to equation (18), the SBR of the signal received by the NLOS imaging system and the distance to the front end are... It is irrelevant, but with the distance of the "relay wall - target" back end... It decays rapidly with increasing intensity. The actual target and size are as follows: Figure 12As shown in (a) above. A 64×64 scan was performed within a 2m×2m field of view at a laser power of 42m, with an exposure time of 1ms per point. For quantitative comparison, the FK reconstruction result of the high-power data at 2m was selected as the GT calculated by SSIM. The reconstruction results are shown below. Figure 12 As shown in (c) in the figure.

[0093] At 2m, both the conventional method and the method proposed in this invention successfully reconstructed the target's structural information. The method of this invention exhibits significantly reduced noise and achieves a 0.1 improvement on SSIM. At 3m, it becomes clear that the conventional FK reconstruction becomes blurred: the two hypotenuses and vertical structure of the "K" are unclear. In contrast, the results of this invention still clearly reconstruct the "K" structure and eliminate most of the noise. At 4m, the conventional method can only locate the target's position but cannot reconstruct it. In contrast, the method of this invention successfully separates the signal from the noise and reconstructs the target's structure. This experiment demonstrates that even when the back-end distance is extended to 4m, the method of this invention can reconstruct the shape from extremely sparse photons, thus providing technical support for long-range NLOS imaging.

[0094] Adaptive windowing enables robust NLOS reconstruction even in photon-scarce conditions. Based on a distribution model of different probabilities for signal and noise photons, the proposed windowing method effectively separates the detection components. The method essentially has two key aspects: first, grouping signal photons returned from spatiotemporally correlated pixel blocks to enhance SNR; and second, removing as much noise as possible through adaptive windowing based on matched filtering. Experimental results show that, even with a signal density as low as 0.02 photons per pixel (one of the lowest values ​​reported in NLOS imaging to date), the method successfully reconstructs hidden targets that traditional methods cannot identify, achieving a structural similarity index 0.5 higher than other methods. Completely hidden scenes can be reconstructed with a total exposure time of less than 0.82 s, and this can be achieved even at a distance of up to 4 m, successfully overcoming a key bottleneck in practical applications.

Claims

1. A few-photon non-line-of-sight imaging method based on adaptive windowing, characterized in that, include: Step 1: Use a non-coaxial two-dimensional scanning system to collect photon signals reflected from the hidden target, establish probability models of signal photons and noise photons, and analyze their distribution characteristics in the time domain. Step 2: Utilizing spatiotemporal correlation, adjacent pixels are grouped into pixel blocks. The adaptive window width of each pixel block is determined using a matched filtering method, and a windowing operation is applied in the temporal domain to separate signal photons from noise photons; specifically: By utilizing the spatiotemporal correlation of surrounding scan points, the nearest 4×4 pixels are grouped into pixel blocks. The photon count within a pixel block is represented as: ； （8） in, It is a pixel block index. Photon flux acquired at each scan point; The photon counting data is preprocessed using matched filtering; the system response function (IRF) is used as the template for matched filtering. Each pixel block is obtained through pre-measurement. The matched filtering result is expressed as: ； （9） Larger, larger pixel blocks The higher the probability that a photon counting event originates from a hidden target, the more likely it is to occur; utilizing The window width of the pixel block is calculated using the extreme points and the system's half-maximum bandwidth. : ； （10） The number of noise photons received per second. To determine the system detection efficiency of the detector; Applying windowing to the original transient information in the time domain, the windowing process is represented as follows: ； （11） in, It was the beginning of the window. ; Step 3: Use total variable regularization constraints to fill in the transient data after windowing, and use the alternating direction multiplier method to solve the reconstruction problem to obtain the target reconstruction result.

2. The few-photon non-line-of-sight imaging method based on adaptive windowing according to claim 1, characterized in that, Step 1 is as follows: At the operating wavelength Under these conditions, the number of signal photons detected by the detector per second Represented as: ； （1） in, It is the average optical power of the pulsed laser. It is the Bronk constant, and c is the speed of light. , These are the reflectivities of the target and the relay wall, respectively. It is the angle between the normal of the relay wall and the laser beam. It is the area of ​​the hidden target. It is the field of view on the relay wall. It is the area of ​​the detector's receiving field of view. It is the transmittance of the receiving system. It is the transmittance of air. It is the distance between the optical system and the relay wall. It is the distance between the target and the relay wall; In addition to signal photons, the NLOS imaging system also receives noise photons composed of background noise and dark noise, receiving a certain number of noise photons per second. Represented as: ； （2） in, It is the bandwidth of an ideal bandpass filter. It is solar spectral irradiance. It is the angle of incidence of the sun. It is the dark count rate of the detector; When the detector's system detection efficiency is At that time, the photon flux reaching the detector Represented as: ； （3） The arrival time of photons acquired by SPAD can only be discretely recorded in different time bins, with a width of The average number of photons detected within the nth time bin Expressed as: ； （4） in, It is a time bin index. It is the maximum index of the time bin; The photon signal detected by SPAD follows a Poisson distribution. The probability of observing k photon times within the nth time bin is... Represented as: ； （5） In the The probability of detecting a signal photon within a time bin Represented as: ； （6） For SPAD's dead time; Detection efficiency of noisy photons Represented as: （7）。 3. The few-photon non-line-of-sight imaging method based on adaptive windowing according to claim 1, characterized in that, Step 3 specifically involves: TV regularization constraints are introduced to fill in the transient information after windowing, thus optimizing the transient information. Represented as: ； （12） in, It is a regular parameter associated with the measured signal-to-noise ratio level; In a confocal NLOS scenario, the axial position of the repeater wall is set to 0; at the scan point The transient information captured at that point is represented as follows: ； （13） in, Points on the hidden object Distance to the laser or detector; This represents the attenuation of photons scattered by the relay wall after two reflections. For the time of photon flight; Dirac function Representing the surface of a spacetime four-dimensional hypercone, this function models the light propagation process from the relay wall to the hidden target and back to the relay wall; Equation (13) is then... Variable substitution: ； （14） The above equation represents a 3D convolution process: ;in, Represents 3D convolution kernel, and These represent resampling operations along the z-axis and t-axis, respectively; from transient information Solving in the middle The problem is formulated as solving regularized least squares; let... It is with the nuclear The associated 3D convolution operation, this problem can be represented by a matrix as follows: ； （15） in, It is the regularization parameter, which is used as a weighted constant to balance the tradeoff between the data fidelity term and the regularization term; the alternating direction multiplier method is used to solve this minimization problem.

4. An electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method as described in any one of claims 1-3.

5. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps of the method as described in any one of claims 1-3.

6. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the steps of the method described in any one of claims 1-3.

Images