A photonic-efficient single-photon non-line-of-sight imaging method and system
The single-photon non-line-of-sight imaging method, which employs self-calibration and time-binding processing, solves the problems of photon scarcity and geometric distortion in long-distance non-line-of-sight imaging, achieving efficient and stable reconstruction of occluded targets. It is suitable for practical applications such as autonomous driving and security monitoring.
Patent Information
- Application Number
- CN202611311747.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-08-27
- Publication Date
- 2026-11-17
AI Technical Summary
Existing non-line-of-sight imaging technologies face challenges when deployed over long distances, including attenuation of effective echo signals, photon scarcity, transient histogram noise dominance, geometric distortion during wall scanning, and insufficient accuracy of camera-assisted calibration. These issues result in slow imaging speeds and distorted or blurred reconstruction results.
A photonic-efficient single-photon non-line-of-sight imaging method is employed. The scanning point coordinates are calculated based on the direct wall echo signal through a self-calibration scheme. Combined with time-binning processing and regular grid correction, data augmentation and reconstruction are achieved, including initial scanning, plane fitting, mapping relationship establishment, and target regular sampling grid generation.
It improves imaging stability and efficiency under long-distance, low-photon conditions, reduces system complexity, eliminates reconstruction errors, and is suitable for practical applications such as autonomous driving, robot perception, and security monitoring.
Smart Images

Figure FT_1 
Figure FT_2 
Figure FT_3
Abstract
Description
Technical Field
[0001] This invention relates to the field of non-line-of-sight imaging technology, and in particular to a photon-efficient single-photon non-line-of-sight imaging method and system. Background Technology
[0002] Non-line-of-sight (NLOS) imaging aims to detect and reconstruct hidden targets that are outside the direct field of view and obscured by occluded objects, using multiple scattered light from visible relay surfaces. A typical time-of-flight NLOS imaging system illuminates diffuse reflective relay surfaces such as walls with an active light source, records the time-resolved light signal from the relay surface, through the hidden target, and back to the detection end, and inverts the spatial location and reflectivity distribution of the hidden target based on the corresponding light transmission model.
[0003] Existing non-line-of-sight imaging techniques mainly include transient imaging methods based on ultrafast pulse illumination and time-correlated single-photon counting, confocal non-line-of-sight imaging methods based on optical conic transformation, and reconstruction methods based on frequency domain migration or virtual wave propagation. These methods have achieved good results at the laboratory scale, but still face significant challenges when deployed over long distances.
[0004] First, in long-distance non-line-of-sight imaging, the effective three-reflection echo signal attenuates sharply with increasing distance, and the transient histogram of a single scan point is often in a photon-depleted state. If a very narrow time grid width is used directly, although a high temporal resolution is preserved, the photon count in each time grid is too low, and the transient waveform is prone to being discrete, fragmented, and noise-dominated. If the acquisition time is simply increased, the imaging speed will be significantly reduced, which is not conducive to practical applications.
[0005] Secondly, most fast non-line-of-sight reconstruction algorithms assume that wall sampling points are located on a known and regular two-dimensional square grid. However, in long-distance systems, when applying linear voltage scanning using a two-dimensional galvanometer, the actual location of the light spot illuminating the wall is usually not a uniform square grid because the relay wall may be tilted relative to the system's optical axis. This geometric distortion can cause a mismatch between the measured transient data and the reconstruction model, resulting in deformation, blurring, or positioning errors in the reconstruction results.
[0006] Furthermore, traditional camera-assisted calibration methods require the use of visible light or near-infrared cameras to observe the position of the light spot on the wall. In long-range near-infrared non-line-of-sight systems, the wall is far away, the light spot size is large, the ambient light is complex, and the position of the light spot observed by the camera may not accurately reflect the actual lighting geometry. Therefore, camera-assisted calibration has shortcomings in terms of system complexity, deployment convenience, and calibration accuracy.
[0007] Therefore, there is an urgent need for a single-photon non-field imaging method and system that can improve the availability of transient data under long-distance, low-photon-count conditions, while correcting the wall scanning geometry without the need for an external camera. Summary of the Invention
[0008] This invention provides a photonic-efficient single-photon non-line-of-sight imaging method and system to overcome the shortcomings of existing technologies.
[0009] This invention provides a photon-efficient single-photon non-line-of-sight imaging method, the method comprising the following steps: S1. Drive the transmitting module to emit pulse detection light towards the relay wall according to the preset initial galvanometer control quantity set and form an initial scanning dot matrix on the relay wall. At each scanning position, the receiving module receives photons returned by the relay wall and the hidden target hidden behind the relay wall, and the arrival time of each photon is recorded by the time acquisition module. The original transient histogram corresponding to each scanning position is accumulated and denoted as the first transient histogram set. S2. Based on the first transient histogram set, for the original transient histogram corresponding to each scanning position, select a time window containing the direct wall echo signal, and extract the time position that meets the preset conditions within the time window as the arrival time of the direct wall echo of each scanning position, which is denoted as the first arrival time set. S3. Based on the first set of arrival times and the pre-calibrated system parameters, for each scanning position, the total optical path is calculated according to the arrival time of the direct wall echo corresponding to the scanning position, and the beam emission direction vector is calculated according to the galvanometer control quantity corresponding to the scanning position combined with the angle calibration coefficient. Then, based on the geometric relationship between the emitted ray and the equi-optical-path ellipsoid, the three-dimensional coordinates of the wall sampling point corresponding to the scanning position are obtained to form the initial wall point cloud. S4. Perform plane fitting on the initial wall point cloud to obtain the normal vector of the fitted wall plane and a point on the plane. Project each point in the initial wall point cloud along the normal vector direction onto the fitted wall plane to obtain the two-dimensional coordinates of each scanning position in the fitted wall plane, forming a set of projected point cloud plane coordinates. Based on the initial galvanometer control quantity set and the projected point cloud plane coordinate set, construct the mapping relationship between the galvanometer control quantity and the wall plane coordinates. S5. Generate a target regular sampling grid on the fitted wall plane. For each grid point in the target regular sampling grid, use the established mapping relationship between the galvanometer control quantity and the wall plane coordinates to obtain the correction galvanometer control quantity corresponding to the grid point. S6. According to the correction galvanometer control amount corresponding to each grid point in the target rule sampling grid, drive the transmitting module to scan the pulse probe light to each position on the relay wall corresponding to the target rule sampling grid according to the correction galvanometer control amount. At each correction scanning position, the receiving module receives photons returned by the relay wall and the hidden target hidden behind the relay wall, and the arrival time of each photon is recorded by the time acquisition module. The original transient histogram corresponding to each correction scanning position is accumulated and recorded as the second transient histogram set. S7. Based on the second transient histogram set, perform time binning operation on the original transient histogram corresponding to each correction scan position according to the preset time binning factor to obtain the transient histogram set after time binning, which is denoted as the third transient histogram set. S8. Based on the third transient histogram set, according to the wall distance range and background noise level preset by the single-photon non-field imaging system, determine the time interference interval of the direct wall echo and the time interference interval of the background noise in the transient histogram of each correction scanning position. Remove the data in the time interference interval from the transient histogram after time binning to obtain the effective transient data corresponding to each correction scanning position. Based on the effective transient data corresponding to each correction scanning position, organize it into a three-dimensional spatiotemporal data cube according to the row and column order of the target rule sampling grid. S9. Based on the three-dimensional spatiotemporal data cube, the fitted wall plane, and the target regular sampling grid, solve for the spatial reflectivity distribution of the hidden target to obtain the reconstructed image or three-dimensional voxel data of the hidden target.
[0010] According to the photon-efficient single-photon non-line-of-sight imaging method provided by the present invention, step S2 includes: For each scanning position, a time window containing the direct wall echo signal is selected. Within the time window, the time grid position corresponding to the maximum value of the original transient histogram is extracted by the peak method, or the weighted average time position of the original transient histogram is calculated by the centroid method. This is used as the arrival time of the direct wall echo at each scanning position and is denoted as the first arrival time set.
[0011] In one embodiment, the angle calibration coefficient between the galvanometer control quantity and the beam emission direction in step S3 adopts a linear calibration model, that is, the optical deflection angle of the beam in the horizontal and vertical directions is expressed as a linear function of the horizontal control voltage and the vertical control voltage of the galvanometer, respectively; the angle calibration coefficient is pre-determined and fixed in the single-photon non-field imaging system through a dedicated calibration process after the system is assembled and adjusted.
[0012] According to the photon-efficient single-photon non-line-of-sight imaging method provided by the present invention, step S4 includes: Perform least-squares plane fitting on the initial wall point cloud to obtain the normal vector of the fitted wall plane and a point on the plane. Project each point in the initial wall point cloud along the normal vector direction onto the fitted wall plane to obtain the two-dimensional coordinates of each scanning position in the fitted wall plane, forming a set of projected point cloud plane coordinates. Using the initial set of galvanometer control variables as input independent variables and the set of projected point cloud plane coordinates as output dependent variables, a positive mapping relationship from galvanometer control variables to wall plane coordinates is constructed, and / or, using the set of projected point cloud plane coordinates as input independent variables and the initial set of galvanometer control variables as output dependent variables, a negative mapping relationship from wall plane coordinates to galvanometer control variables is constructed. The forward and / or reverse mapping relationships are established using one or more of the following methods: polynomial mapping, bilinear interpolation, spline interpolation, radial basis function interpolation, lookup table, or neural network fitting.
[0013] In one implementation, the least squares plane fitting in step S4 uses principal component analysis, with the eigenvector corresponding to the smallest eigenvalue of the covariance matrix of the initial wall point cloud as the normal vector of the fitting plane, and the spatial mean of the initial wall point cloud as a point on the fitting plane.
[0014] According to the photon-efficient single-photon non-line-of-sight imaging method provided by the present invention, the definition parameters of the target regular sampling grid include: the grid origin position, the number of grid rows and columns, and the grid spacing or grid range in the row and column directions; the definition parameters are input by the user or automatically determined by the system according to the working distance and target size.
[0015] In one implementation, the target rule sampling grid is a square grid or a rectangular grid, and the number of grid points is one of 16×16, 32×32, 64×64 or 128×128; the grid range and the number of grid points are adaptively determined by the system according to the working distance, target size and real-time requirements, or are set by the user through an external interface.
[0016] According to the photon-efficient single-photon non-line-of-sight imaging method provided by the present invention, step S5 includes: A target regular sampling grid is generated on the fitted wall plane. For each grid point in the target regular sampling grid, the correction galvanometer control quantity corresponding to the grid point is directly calculated using the inverse mapping relationship between the established wall plane coordinates and the galvanometer control quantity. Alternatively, the galvanometer control quantity that makes the forward mapping value closest to the coordinates of the grid point is solved by numerical optimization using the established forward mapping relationship between the galvanometer control quantity and the wall plane coordinates. This value is then used as the correction galvanometer control quantity corresponding to the grid point.
[0017] In one implementation, the reverse calibration mirror control quantity can be obtained through numerical optimization, interpolation inversion, or lookup table lookup.
[0018] According to the present invention, a photon-efficient single-photon non-line-of-sight imaging method includes a time binning operation comprising: merging multiple adjacent original time cells in the original transient histogram into one effective time cell, wherein the photon count in the merged effective time cell is equal to the sum of the photon counts of the merged original time cells, and the merged time resolution is the original time resolution multiplied by the time binning factor.
[0019] In one implementation, the time binning factor can be automatically selected based on the system working distance: a larger time binning factor is selected when the working distance increases, and a smaller time binning factor is selected when the working distance decreases; the value range of the time binning factor is a positive integer between 2 and 64.
[0020] According to the photon-efficient single-photon non-line-of-sight imaging method provided by the present invention, step S9 includes: Based on the three-dimensional spatiotemporal data cube, the fitted wall plane, and the target regular sampling grid, the spatial reflectivity distribution of the hidden target is solved using the light cone transformation reconstruction algorithm, frequency domain migration reconstruction algorithm, phasor field backpropagation reconstruction algorithm, or other non-view domain reconstruction algorithms, to obtain the reconstructed image or three-dimensional voxel data of the hidden target.
[0021] In one implementation, the light cone transformation reconstruction algorithm specifically involves: converting the transient model under the confocal approximation into a convolutional form through variable resampling, and then using Wiener filtering or Richardson-Lucy deconvolution algorithm to solve for the reflectivity distribution of the hidden target.
[0022] This invention also provides a photon-efficient single-photon non-line-of-sight imaging system, comprising a transmitting module for transmitting pulsed probe light toward a relay wall according to a galvanometer control quantity; a receiving module for receiving photons returned by the relay wall and a hidden target behind the relay wall; a time acquisition module for recording the arrival time of each photon; and a control and processing module configured to execute steps S1 to S8 of the method described in any of the preceding claims; wherein the pre-calibrated system parameters of the single-photon non-line-of-sight imaging system include: the equivalent optical center position of the transmitting end, the equivalent optical center position of the receiving end, the system time zero point and the fixed optical path delay, and the angle calibration coefficient between the galvanometer control quantity and the beam emission direction.
[0023] In one embodiment, the photonically efficient single-photon non-line-of-sight imaging method of the present invention can be implemented using the photonically efficient single-photon non-line-of-sight imaging system of the present invention.
[0024] The present invention provides a photonic-efficient single-photon non-line-of-sight imaging method and system, which can bring at least the following beneficial effects: In step S7, this invention introduces time binning, which aggregates multiple adjacent original time grids into a single effective time grid. This causes the photon count within each effective time grid to increase approximately linearly with the binning factor, while the relative shot noise decreases according to the reciprocal of the square root of the binning factor. This effectively suppresses shot noise-dominated fluctuations, restoring the originally discrete and fragmented transient waveform to a continuous and discernible form. This process significantly reduces the single-point dwell time while maintaining reconstruction quality: at a distance of 50 m, a single-point dwell time of 10 ms with a 4 ps original time grid width is reduced to 1 ms after 32 times time binning; at a distance of 100 m, a single-point dwell time of 50 ms with a 4 ps original time grid width is reduced to 6 ms after 16 times time binning. Since non-line-of-sight imaging typically requires acquiring data point by point from dozens of wall scanning points, the reduction in single-point acquisition time directly leads to a significant increase in overall imaging speed, enabling long-distance non-line-of-sight imaging to have an acceptable acquisition time window in practical deployments.
[0025] In steps S2 to S4 of this invention, a self-calibration scheme based entirely on echo signals is constructed. Utilizing the direct wall echo arrival time at each scanning position and the pre-calibrated scanning angle parameters, the three-dimensional coordinates of the wall sampling points are directly calculated, replacing the traditional approach of relying on visible light or near-infrared cameras to observe the light spot position. In long-distance deployment scenarios (50 m to 100 m or even further), the wall distance is large, the light spot size is large, and the ambient light is complex, making it difficult for cameras to capture clear images. Furthermore, the observed light spot position may not accurately reflect the actual lighting geometry due to parallax and other factors. This invention's self-calibration scheme, directly based on time-of-flight measurement, is unaffected by ambient lighting conditions, fundamentally eliminating the limitations of camera-assisted calibration in terms of accuracy and reliability. Simultaneously, it eliminates the need for external cameras and their associated optical components, reducing system hardware costs, size, and assembly complexity, making the system easier to deploy on practical platforms such as autonomous vehicles, robotic platforms, or security monitoring stations.
[0026] In steps S4 and S5, this invention corrects the scanning trajectory through plane fitting and inverse mapping. Specifically, the self-calibrated wall point cloud is projected onto the fitting plane to establish a precise mapping relationship between the galvanometer control quantity and the wall plane coordinates. Then, a target regular sampling grid is generated on the fitting plane, and the corresponding corrected galvanometer control quantity is calculated inversely, ensuring that the wall sampling points corresponding to the final acquired data strictly match the regular grid assumed by the reconstruction algorithm. This correction process eliminates the geometric distortion introduced by the tilt of the relay wall relative to the system optical axis, avoiding deformation, blurring, or positioning errors in the reconstruction results. The three-dimensional spatiotemporal data cube organized in step S8 corresponds one-to-one with the regular grid coordinates and can be directly used as input for subsequent non-view reconstruction algorithms, eliminating the need to handle sampling point position deviations during reconstruction and improving the stability and geometric fidelity of the reconstruction.
[0027] This invention addresses the real-world deployment needs of long-distance non-line-of-sight imaging by constructing a reproducible and complete technical solution in steps S1 to S9, covering the entire chain from raw data acquisition, geometric calibration and correction, data augmentation to target reconstruction. The data flow between each step is clear, and the interfaces are well-defined. The self-calibration method employed is adaptable to different system architectures, such as transmitter-side single-mirror scanning, receiver-side single-mirror scanning, transmitter-receiver dual-mirror synchronous scanning, confocal scanning, or non-confocal scanning. Time-binning processing can be performed by the data processing module in software after acquisition, or by the FPGA or embedded processor within the acquisition and control module in real-time during acquisition, allowing for flexible selection based on actual system resources. The non-line-of-sight reconstruction in step S9 is compatible with various existing reconstruction algorithm frameworks, including optical conic transformation, frequency domain shifting, and phasor field backpropagation. These features give this invention excellent system scalability and engineering adaptability, enabling flexible configuration and deployment for specific constraints (working distance, target size, real-time requirements, hardware resources, etc.) in different application scenarios such as autonomous driving, robot perception, security monitoring, and disaster relief.
[0028] In summary, the present invention achieved "L"-shaped structure reconstruction results consistent with the real target in experimental verifications at distances of 50 m and 100 m, confirming the effectiveness and practicality of the method under conditions of photon scarcity at long distances. This invention provides a single-photon non-line-of-sight imaging solution for long-distance target perception that requires no external auxiliary equipment, has high acquisition efficiency, and stable reconstruction quality, which can promote the transformation of non-line-of-sight imaging technology from laboratory environments to practical engineering applications. Attached Figure Description
[0029] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.
[0030] Figure 1 This is a schematic diagram of the structure of a photonic high-efficiency single-photon non-line-of-sight imaging system provided by the present invention.
[0031] Figure 2 This is a schematic flowchart of a photonic-efficient single-photon non-line-of-sight imaging method provided by the present invention.
[0032] Figure 3 The figure shows the experimental results of long-distance single-photon non-line-of-sight imaging of an "L"-shaped hidden target under different imaging distances, time bin widths, and single-point dwell times. Detailed Implementation
[0033] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, embodiments of this invention, and should not be construed as limiting the invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention. In the description of this invention, it should be understood that the terminology used is for descriptive purposes only and should not be construed as indicating or implying relative importance.
[0034] refer to Figure 1 The single-photon non-line-of-sight imaging system provided in this embodiment of the invention includes a transmitting module 100, a receiving module 200, and an acquisition and control module 300.
[0035] The transmitting module 100 generates pulsed probe light and scans it onto the repeater wall 6. In one specific embodiment, the transmitting module 100 includes a pulsed laser 1, a beam splitter 2, a transmitting collimator 4, and a transmitting scanning mirror 5. The pulsed laser 1 outputs femtosecond pulses with a center wavelength of 1558 nm and a repetition frequency of 50 MHz. This output light is split into reference light and probe light by the beam splitter 2, with approximately 1% of the reference light incident on the photodetector 3 to generate a time reference signal, and the remaining probe light collimated by the collimator 4 and then scanned onto the repeater wall 6 by the transmitting scanning mirror 5.
[0036] The receiving module 200 is used to receive photons returned by the hidden target 8 and the relay wall 6. In one specific embodiment, the receiving module 200 includes a receiving scanning mirror 9, a receiving collimator 10, and an InGaAs single-photon detector 11. The receiving collimator 10 collects the returned photons, the receiving scanning mirror 9 scans synchronously with the transmitting scanning mirror 5 or according to a preset relationship, and the focusing element focuses the returned light onto the photosensitive surface of the single-photon detector 11.
[0037] The acquisition and control module 300 records the arrival time of the detection events output by the single-photon detector 11 relative to the reference signal and forms a transient histogram at each wall scanning position. The acquisition and control module 300 may employ a time-correlated single-photon counting module, a time-to-digital converter, or a time marker.
[0038] refer to Figure 2 The single-photon non-line-of-sight imaging process provided in this invention includes four stages: pre-scan self-calibration, correction scan acquisition, time binning processing, and hidden target reconstruction. The following is a specific embodiment of data processing for a 100 m long-distance single-photon non-line-of-sight imaging system, illustrating the intermediate processing from the original transient histogram to the hidden target reconstruction result. In this embodiment, the distance from the single-photon non-line-of-sight imaging system to the relay wall is 50 m or 100 m, the relay wall scanning area is 0.6 m × 0.6 m, the target regular sampling grid is 64 × 64, and the hidden target is located approximately 0.5 m behind the relay wall.
[0039] During the pre-scan self-calibration phase, the acquisition and control module 300 applies a set of initial linear control variables to the transmitting scanning mirror 5 and the receiving scanning mirror 10, causing the probe light to form an initial scanning dot matrix on the relay wall 6. For each scanning position, the acquisition and control module 300 records a transient histogram containing the direct wall echo, and extracts the peak value or centroid position of the direct wall echo from the transient histogram to obtain the corresponding direct echo arrival time. Let the galvanometer control variable for the i-th pre-scan position be... The corresponding original transient histogram is The original time grid width is Select a time window that includes direct wall echoes. Within this time window, the arrival time of the direct wall echo can be obtained using either the peak method or the centroid method. The peak method can be written as:
[0040] The center-of-mass method is:
[0041] Then estimate the three-dimensional coordinates of the wall surface points. Let the equivalent optical center of the transmitter be... The equivalent optical center of the receiving end is The system time zero point and fixed optical path delay are combined into Then the first The total optical path of the direct wall echo at each scanning position is:
[0042] The unit direction of the emitted beam is obtained based on the calibration relationship between the galvanometer control value and the optical scanning angle. Taking linear angle calibration as an example:
[0043]
[0044] At this point, the following is acceptable:
[0045] in , The equivalent optical deflection angle representing the direction of beam exit. Wall point. Located on the emitted ray, satisfying:
[0046]
[0047] make Then we can obtain:
[0048] When the transmitter and receiver are approximately confocal The above equation degenerates into:
[0049] Therefore, we can conclude that:
[0050] Obtain the initial wall point cloud Then, perform a plane fitting. Let the fitted wall plane be:
[0051] Find the normal vector and a point on the plane using the following least squares problem:
[0052] In actual calculations, we can take... The mean of the point cloud. This represents the eigenvector corresponding to the smallest eigenvalue of the point cloud covariance matrix. Then, each wall point is projected onto the fitting plane:
[0053] Establish a two-dimensional coordinate system within the fitting plane. Select a unit vector within the plane. As the first coordinate axis, let:
[0054] Then the first The planar coordinates of the wall sampling points are: ,in:
[0055]
[0056] Therefore, the sample pairs obtained from the pre-scan are Next, a mapping relationship is established between the galvanometer control variables and the wall plane coordinates. An optional implementation is to use a second-order polynomial forward mapping. :
[0057]
[0058] After substituting all the pre-scanned samples, construct the design matrix:
[0059] The coefficient vector can then be obtained by the following formula:
[0060]
[0061] in, This is an optional regularization parameter; it can be set to `optional` when no regularization is needed. .
[0062] To inversely calculate the control quantity corresponding to the regular sampling grid, the inverse mapping can also be directly fitted. :
[0063]
[0064] For any point in the target rule grid The control value for correcting the galvanometer can be directly calculated as follows:
[0065] If only a positive mapping is established The corrective control quantity is obtained by solving the following formula:
[0066] in, The control values from either the nearest pre-scan point or the previous scan point can be used. The target rule sampling grid can be generated as follows:
[0067]
[0068] in, , , and To fit the width and height of the target scanning area within the plane, in this embodiment, , The acquisition and control module sequentially outputs the correction control values corresponding to each regular grid point to the galvanometer, thereby ensuring that the actual wall scanning points fall as close as possible to the regular wall grid.
[0069] After completing the calibration scan, the transient histograms obtained for each regular wall point are binned over time. Let the original histogram of the (m,n)th wall point after the calibration scan be... ,but The histogram after binning over time is as follows:
[0070]
[0071] Since single-photon counting follows Poisson statistics, the expected count within a single effective time cell after binning varies. The relative shot noise increases approximately linearly, roughly according to Lower; but If the value is too large, it will reduce the effective time resolution. Therefore, in this embodiment, the time grid width is used as an adjustable parameter.
[0072] Before the hidden target reconstruction, direct wall echoes and background noise-dominated regions are removed from the binned transient data, and the effective transient data of all wall points are organized into a three-dimensional spatiotemporal data cube according to a regular grid order. The data cube corresponds one-to-one with the coordinates of the regular wall surface and can be directly used as input for subsequent non-view reconstruction algorithms.
[0073] Non-line-of-sight reconstruction can employ phasor field backpropagation, optical conic transformation, frequency domain shifting, or other existing reconstruction algorithms. Taking optical conic transformation as an example, the transient model under the confocal approximation is written as:
[0074] The above model can be transformed into an inverse convolution problem through variable resampling and then solved by deconvolution. The reconstruction algorithm itself can adopt existing non-view-domain inversion methods. This embodiment provides input data that is more suitable for long-distance low-photon scenes through temporal binning and wall self-calibration.
[0075] In one experimental example, a clear "L"-shaped target reconstruction was obtained at a distance of 50 m using a time grid width of 4 ps and a single-point dwell time of 10 ms. (Reference) Figure 3 After employing 32 ps time binning, the single-point dwell time was reduced to 1 ms, while still obtaining a reconstruction result with consistent shape. At a distance of 100 m, using a 4 ps time grid width and a 50 ms single-point dwell time as a reference, after employing 16 ps time binning, the single-point dwell time was reduced to 6 ms, while still recovering an "L"-shaped structure consistent with the real target. These results demonstrate that combining time binning with self-calibration correction can significantly reduce single-point acquisition time and maintain reconstruction stability under long-distance, low-photon-count conditions.
[0076] This invention is not limited to the specific light source, detector, and reconstruction algorithm described above. The pulsed laser can employ other near-infrared, visible, or short-wave infrared wavelengths; the single-photon detector can be an InGaAs SPAD, silicon SPAD, superconducting nanowire single-photon detector, or other detectors with time-resolved capabilities; the time acquisition module can employ any hardware capable of generating a time-resolved histogram.
[0077] The time binning process of this invention can be performed by software after acquisition, or directly by FPGA, embedded processor, or time acquisition hardware during acquisition. Time binning can use a fixed binning factor or an adaptive binning factor that varies with the time window, scan position, or photon count level.
[0078] The self-calibration method of this invention can be used in structures such as transmitter-side single-mirror scanning, receiver-side single-mirror scanning, transmitter-receiver dual-mirror synchronous scanning, confocal scanning, or non-confocal scanning. As long as direct wall echoes can be obtained and a geometric relationship between the scanning control quantity and the wall position can be established, the self-calibration concept of this invention can be adopted.
[0079] The reconstruction target of this invention can be a two-dimensional projected image, a three-dimensional voxel image, a target contour, a distance slice, or other forms of hidden scene representation. For different application scenarios, different wall sampling densities, time bin widths, and reconstruction algorithms can be selected according to the target size, distance, and real-time requirements.
[0080] In this embodiment, the laser wavelength is 1550 nm, the transmitting collimator and the receiving collimator are both Thorlabs C80APC-C, the transmitting scanning mirror and the receiving scanning mirror are both Xunlai Optoelectronics FSM50, and the single photon detector is Aurea SPD-OEM-NIR.
[0081] This invention addresses non-line-of-sight (NLOS) imaging applications under long-distance, low-photon-count conditions. It tackles problems in existing single-photon NLOS imaging systems during long-distance operation, such as weak echo signals, sparse photon counts in transient histograms, inaccurate geometric positions of wall scan points, and mismatch between the reconstruction model and the actual sampling grid. The invention proposes a photon-efficient single-photon NLOS imaging method and system that integrates direct wall echo self-calibration, regular wall grid correction, temporal binning, and NLOS image reconstruction. Compared to existing technologies, this invention eliminates the need for an additional camera to calibrate the near-infrared spot at long distances, reducing system complexity and deployment difficulty; it improves the accuracy of wall scan point coordinate estimation through direct wall echo self-calibration; it reduces the impact of actual scanning distortion on the NLOS reconstruction results through regular wall grid correction; and it improves the effective statistics of the low-photon-count transient histogram through temporal binning. Therefore, the present invention can effectively improve the imaging stability, acquisition efficiency, geometric consistency and reconstruction quality of long-distance single-photon non-line-of-sight imaging systems, and is suitable for application scenarios that require long-distance perception of occluded targets, such as autonomous driving, robot perception, security monitoring, and disaster relief.
[0082] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A photon-efficient single-photon non-line-of-sight imaging method, characterized in that, The method includes the following steps: S1. Drive the transmitting module to emit pulse detection light towards the relay wall according to the preset initial galvanometer control quantity set and form an initial scanning dot matrix on the relay wall. At each scanning position, the receiving module receives photons returned by the relay wall and the hidden target hidden behind the relay wall, and the arrival time of each photon is recorded by the time acquisition module. The original transient histogram corresponding to each scanning position is accumulated and denoted as the first transient histogram set. S2. Based on the first transient histogram set, for the original transient histogram corresponding to each scanning position, select a time window containing the direct wall echo signal, and extract the time position that meets the preset conditions within the time window as the arrival time of the direct wall echo of each scanning position, which is denoted as the first arrival time set. S3. Based on the first set of arrival times and the pre-calibrated system parameters, for each scanning position, the total optical path is calculated according to the arrival time of the direct wall echo corresponding to the scanning position, and the beam emission direction vector is calculated according to the galvanometer control quantity corresponding to the scanning position combined with the angle calibration coefficient. Then, based on the geometric relationship between the emitted ray and the equi-optical-path ellipsoid, the three-dimensional coordinates of the wall sampling point corresponding to the scanning position are obtained to form the initial wall point cloud. S4. Perform plane fitting on the initial wall point cloud to obtain the normal vector of the fitted wall plane and a point on the plane. Project each point in the initial wall point cloud along the normal vector direction onto the fitted wall plane to obtain the two-dimensional coordinates of each scanning position in the fitted wall plane, forming a set of projected point cloud plane coordinates. Based on the initial galvanometer control quantity set and the projected point cloud plane coordinate set, construct the mapping relationship between the galvanometer control quantity and the wall plane coordinates. S5. Generate a target regular sampling grid on the fitted wall plane. For each grid point in the target regular sampling grid, use the established mapping relationship between the galvanometer control quantity and the wall plane coordinates to obtain the correction galvanometer control quantity corresponding to the grid point. S6. According to the correction galvanometer control amount corresponding to each grid point in the target rule sampling grid, drive the transmitting module to scan the pulse probe light to each position on the relay wall corresponding to the target rule sampling grid according to the correction galvanometer control amount. At each correction scanning position, the receiving module receives photons returned by the relay wall and the hidden target hidden behind the relay wall, and the arrival time of each photon is recorded by the time acquisition module. The original transient histogram corresponding to each correction scanning position is accumulated and recorded as the second transient histogram set. S7. Based on the second transient histogram set, perform time binning operation on the original transient histogram corresponding to each correction scan position according to the preset time binning factor to obtain the transient histogram set after time binning, which is denoted as the third transient histogram set. S8. Based on the third transient histogram set, according to the wall distance range and background noise level preset by the single-photon non-field imaging system, determine the time interference interval of the direct wall echo and the time interference interval of the background noise in the transient histogram of each correction scanning position. Remove the data in the time interference interval from the transient histogram after time binning to obtain the effective transient data corresponding to each correction scanning position. Based on the effective transient data corresponding to each correction scanning position, organize it into a three-dimensional spatiotemporal data cube according to the row and column order of the target rule sampling grid. S9. Based on the three-dimensional spatiotemporal data cube, the fitted wall plane, and the target regular sampling grid, solve for the spatial reflectivity distribution of the hidden target to obtain the reconstructed image or three-dimensional voxel data of the hidden target.
2. The photonic-efficient single-photon non-line-of-sight imaging method according to claim 1, characterized in that, Step S2 includes: For each scanning position, a time window containing the direct wall echo signal is selected. Within the time window, the time grid position corresponding to the maximum value of the original transient histogram is extracted by the peak method, or the weighted average time position of the original transient histogram is calculated by the centroid method. This is used as the arrival time of the direct wall echo at each scanning position and is denoted as the first arrival time set.
3. The photonic-efficient single-photon non-line-of-sight imaging method according to claim 1, characterized in that, Step S4 includes: Perform least-squares plane fitting on the initial wall point cloud to obtain the normal vector of the fitted wall plane and a point on the plane. Project each point in the initial wall point cloud along the normal vector direction onto the fitted wall plane to obtain the two-dimensional coordinates of each scanning position in the fitted wall plane, forming a set of projected point cloud plane coordinates. Using the initial set of galvanometer control variables as input independent variables and the set of projected point cloud plane coordinates as output dependent variables, a positive mapping relationship is constructed from the galvanometer control variables to the wall plane coordinates, and / or, using the set of projected point cloud plane coordinates as input independent variables and the initial set of galvanometer control variables as output dependent variables, a negative mapping relationship is constructed from the wall plane coordinates to the galvanometer control variables.
4. The photonic-efficient single-photon non-line-of-sight imaging method according to claim 3, characterized in that, In step S4, the least squares plane fitting adopts the principal component analysis method. The eigenvector corresponding to the smallest eigenvalue of the covariance matrix of the initial wall point cloud is used as the normal vector of the fitting plane, and the spatial mean of the initial wall point cloud is used as a point on the fitting plane.
5. The photonic-efficient single-photon non-line-of-sight imaging method according to claim 3, characterized in that, in, Forward and / or inverse mapping relationships are established using one or more of the following methods: polynomial mapping, bilinear interpolation, spline interpolation, radial basis function interpolation, lookup table, or neural network fitting.
6. The photonic-efficient single-photon non-line-of-sight imaging method according to claim 3, characterized in that, Step S5 includes: A target regular sampling grid is generated on the fitted wall plane. For each grid point in the target regular sampling grid, the correction galvanometer control quantity corresponding to the grid point is directly calculated using the inverse mapping relationship between the established wall plane coordinates and the galvanometer control quantity. Alternatively, the galvanometer control quantity that makes the forward mapping value closest to the coordinates of the grid point is solved by numerical optimization using the established forward mapping relationship between the galvanometer control quantity and the wall plane coordinates. This value is then used as the correction galvanometer control quantity corresponding to the grid point.
7. The photonic-efficient single-photon non-line-of-sight imaging method according to claim 1, characterized in that, The definition parameters of the target rule sampling grid include: the grid origin position, the number of grid rows and columns, and the grid spacing or grid range in the row and column directions; the target rule sampling grid is a square grid or a rectangular grid, and the number of grid points is one of 16×16, 32×32, 64×64 or 128×128.
8. The photonic-efficient single-photon non-line-of-sight imaging method according to claim 1, characterized in that, The time binning operation is as follows: multiple adjacent original time cells in the original transient histogram are merged into one effective time cell. The photon count in the merged effective time cell is equal to the sum of the photon counts of the merged original time cells. The time resolution after merging is the original time resolution multiplied by the time binning factor. The time binning factor is determined according to the working distance of the single-photon non-line-of-sight imaging system.
9. The photonic-efficient single-photon non-line-of-sight imaging method according to claim 1, characterized in that, Step S9 includes: Based on the three-dimensional spatiotemporal data cube, the fitted wall plane, and the target regular sampling grid, the spatial reflectivity distribution of the hidden target is solved using the light cone transformation reconstruction algorithm, frequency domain migration reconstruction algorithm, phasor field backpropagation reconstruction algorithm, or other non-view domain reconstruction algorithms, to obtain the reconstructed image or three-dimensional voxel data of the hidden target.
10. A photonic-efficient single-photon non-line-of-sight imaging system, characterized in that, The system includes: The transmitting module is used to drive the transmission of pulsed probe light toward the relay wall surface according to the control quantity of the galvanometer; The receiving module is used to receive photons returned by the relay wall and hidden targets behind the relay wall; The time acquisition module is used to record the arrival time of each photon; The control and processing module is configured to perform steps S1 to S8 of the method according to any one of claims 1-9; The pre-calibrated system parameters of the single-photon non-field imaging system include: the equivalent optical center position of the transmitter, the equivalent optical center position of the receiver, the system time zero point and the fixed optical path delay, and the angle calibration coefficient between the galvanometer control quantity and the beam emission direction.
