Photon counting laser radar non-vision target reconstruction system and method

Abstract

The invention discloses a photon counting laser radar non-vision target reconstruction system and method, and the method comprises the steps: enabling a photon counting laser radar to be aligned with an intermediate surface, and controlling the photon counting laser radar to detect a relay surface and a target; counting the received photon signals to obtain a photon histogram of a first echo signal and a target signal of a relay surface, and improving the time resolution by adopting a hybrid regularized Richardson-Lucy deconvolution algorithm; carrying out peak value extraction by utilizing the echo signal of the relay surface, and calculating relative coordinates of sampling points on the non-flat relay surface in combination with a laser emission angle; analyzing the spectral characteristics of the photon histogram of the target echo signal, adaptively obtaining an effective frequency range for reconstruction, and giving weight distribution; a target signal is processed in combination with a frequency-selecting window, and then a non-vision-field target reconstruction algorithm of diffraction interpolation is utilized to realize reconstruction of a hidden target. The method has universality for non-vision field target reconstruction of various relay surface shapes, sparse modes and sparse rates, and can realize complete and clear reconstruction of hidden targets.

Description

Technical Field

[0001] The present invention belongs to the technical field of photon counting lidar, and specifically relates to a photon counting lidar non-line-of-sight target reconstruction system and a non-line-of-sight target reconstruction method under sparse sampling of a non-planar relay surface. Background Technique

[0002] Driven by the development strategies of intelligent manufacturing and digital economy, non-contact three-dimensional sensing technology, as a core enabling technology in fields such as high-end equipment manufacturing, intelligent transportation systems, and public safety monitoring, is facing significant development opportunities. Traditional geometric optical imaging methods are restricted by physical mechanisms and have inherent limitations in application scenarios such as deep hole structure detection, multi-layer medium penetration imaging, and occluded target perception. Non-line-of-sight imaging technology (NLOS) reconstructs hidden scenes by constructing a high-order light transport equation and analyzing the indirect photon transmission path behind the relay surface. This technology shows unique application potential in important scenarios such as aerospace precision component detection, dynamic perception of blind spots in autonomous driving, and multi-modal navigation in minimally invasive surgery.

[0003] In recent years, non-line-of-sight reconstruction technology has always been a popular topic among domestic and foreign scientific research workers. Currently, institutions such as the Massachusetts Institute of Technology and Stanford University in the United States, as well as Harbin Institute of Technology and Beijing Institute of Technology in China, are leading in the research on non-line-of-sight detection technology, and have effectively improved in aspects such as non-line-of-sight reconstruction sampling time and data processing efficiency. However, the NLOS reconstruction technology fundamentally depends on the relay surface it utilizes and the scanning mode of the relay surface. Most existing algorithms still study the non-line-of-sight target reconstruction technology based on an ideal planar relay surface, and it is difficult to adapt to curved surfaces or complex surfaces; moreover, most scanning schemes require a complete raster scan of the relay surface, and the traditional raster scan is affected by hardware and takes several minutes or even several hours to complete, resulting in low data acquisition efficiency. Summary of the Invention

[0004] The purpose of the present invention is to propose a photon counting lidar non-line-of-sight target reconstruction system and a non-line-of-sight target reconstruction method under sparse sampling of a non-planar relay surface.

[0005] The technical solution to achieve the purpose of the present invention is: A photon counting lidar non-line-of-sight target reconstruction system, comprising:

[0006] At least one precision electric control turntable;

[0007] At least one pulsed laser emission system, configured to emit a narrow pulsed laser and irradiate it on the intermediate surface;

[0008] At least one signal receiving system, configured to receive photon signals returned from the detection position of the relay surface;

[0009] At least one photon counting module that counts the detected photon signals and records the time when the photons reach the detector;

[0010] At least one timing control module that controls the emission of the laser pulse signal, sends a timing start signal to the photon counting module, and generates a gating signal to control the working state of the receiving system;

[0011] At least one data management terminal that stores and preprocesses the photon information returned by the relay surface;

[0012] At least one target reconstruction system that reconstructs the target based on the echo information of the non-line-of-sight target;

[0013] The timing control module is connected to the motor control system, the laser emission system, and the signal receiving system, and is used to control the emission of pulsed laser by the laser and the working state of the detector, and at the same time transmit the time information of the pulse emission to the signal receiving system;

[0014] The signal receiving system is connected to the photon counting module, and accumulates the photon signals collected by the signal receiving system to obtain the waveform information of the target;

[0015] The photon counting module is connected to the data management terminal, and stores and preprocesses the first reflection signal and the target echo signal at the detection position, and is used for the calibration of the detection position of the relay surface and the target reconstruction;

[0016] The data management terminal is connected to the non-planar relay surface non-line-of-sight target reconstruction system, and completes the three-dimensional reconstruction of the hidden target by combining the relay surface coordinate information and the target echo signal.

[0017] Furthermore, the pulsed laser emission system consists of a laser drive circuit, a pulsed laser, and an emission lens assembly. The pulsed laser selects a 532nm pulsed laser with a pulse width of 5ns. Using a pulsed laser with a narrow pulse width ensures the positioning accuracy of the target. The emission lens assembly is used to shape the laser pulse emitted by the laser;

[0018] The signal receiving system consists of a receiving lens assembly and a single photon detection assembly. The receiving lens assembly is used to receive the photon signal of the target and transmit it to the single photon detection assembly. The single photon detector selects the Geiger mode SPAD as the detection assembly, and the SPAD works in the gating mode.

[0019] Furthermore, the laser emission system and the signal receiving system are fixed side by side, and the laser illumination point and the detector detection point are controlled to be in the same position to achieve approximate confocal non-line-of-sight detection.

[0020] A method for reconstructing non-line-of-sight targets under sparse sampling of a non-planar relay surface, based on the non-line-of-sight target reconstruction system of a photon counting lidar according to claim 1, to achieve non-line-of-sight target reconstruction under sparse sampling of a non-planar relay surface, including the following steps:

[0021] Step 1: Align the photon counting lidar with the intermediate surface, and control the photon counting lidar to detect the relay surface and the target;

[0022] Step 2: Use time-correlated single-photon counting technology to statistically analyze the received photon signals to obtain the photon histograms of the first echo signal of the relay surface and the target signal;

[0023] Step 3: For the photon histogram statistical signal sequences of the relay surface echo and the target echo obtained in Step 2, use a hybrid-regularized Richardson-Lucy deconvolution algorithm to improve the time resolution;

[0024] Step 4: Use the relay surface echo signal for peak extraction, and combine the laser emission angle to calculate the relative coordinates of the sampling points on the non-planar relay surface;

[0025] Step 5: Analyze the spectral characteristics of the photon histogram of the target echo signal, adaptively obtain an effective frequency range for reconstruction, and assign a weight distribution;

[0026] Step 6: Combine the frequency selection window to process the target signal, and then use the non-line-of-sight target reconstruction algorithm of diffraction interpolation to achieve the reconstruction of the hidden target.

[0027] Further, in Step 3, for the photon histogram statistical signal sequences of the relay surface echo and the target echo obtained in Step 2, use a hybrid-regularized Richardson-Lucy deconvolution algorithm to improve the time resolution. The specific method is as follows:

[0028] Step 3-1: Since the observed photon histogram is actually the convolution of the real scene response and the laser pulse, which is mathematically expressed as:

[0029] y(t) = h(t) * g(t) + n(t)

[0030] where y(t) is the observed photon histogram, h(t) is the laser pulse shape, g(t) is the scene response under the ideal impulse response, n(t) is the noise, and * represents the convolution operation;

[0031] Step 3-2: At the same time, combine the RL algorithm with TV regularization and L2 regularization to iteratively solve the scene response under the ideal impulse response to improve the time resolution; the iterative formula is:

[0032]

[0033] Among them, g k and g k+1 are the results of the previous round and the current round of iteration respectively.

[0034] Furthermore, in step 4, the peak value is extracted using the relay surface echo signal, and the relative coordinates of the sampling points on the non-planar relay surface are calculated in combination with the laser emission angle. The specific method is as follows:

[0035] Step 4-1, adopt dynamic adaptive differential threshold to extract the target peak value, detect the difference to determine the position of the main wave of the signal after the first peak echo of the relay surface is processed in step 3, and identify the peak value in the case where the tidal wave is higher than the main wave;

[0036] Step 4-2, take the laser emission position as the origin, combine the rotation angle of the turntable and the arrival time of the first peak echo signal of the relay surface sampling point obtained in step 4-1, calculate the coordinate information of the relay surface sampling point by using the time-of-flight method, readjust the relative coordinates of the relay surface formed by the relay surface sampling points, and after the origin is specified at the center of the relay surface, complete the position calibration of the sampling points on the relay surface for the subsequent interpolation calculation from non-planar to planar.

[0037] Furthermore, in step 5, analyze the spectral characteristics of the photon histogram of the target echo signal, adaptively obtain the effective frequency range for reconstruction, and assign weight distribution. The specific method is as follows:

[0038] Step 5-1, determine the maximum resolvable size and the resolvable size at the target depth according to the virtual aperture V_aperture and the maximum target depth Maxdepth set in the non-line-of-sight scenario:

[0039]

[0040] From this, the frequency upper limit fre_bound = c / 2Δx is determined max and the target frequency center fre_mid = c / 2Δx target ;

[0041] Step 5-2, calculate the scene response under the impulse response of the hidden target echo signal obtained by sampling in step 3, calculate the spectral amplitude of the target scene response, calculate the threshold by combining the average, median and quartiles of the amplitude, determine this threshold as the frequency selection lower limit of the scene response of this sampling point, and adaptively circle the frequency selection range for reconstruction in combination with the frequency upper limit and the target frequency center in step 5-1;

[0042] Step 5-3: According to the custom-selected number of frequencies, with the target frequency as the center, while satisfying the upper and lower limits of the frequency, determine the final frequency range for reconstruction. To ensure the reconstruction quality under sparse sampling, perform a trailing Gaussian weight processing on the frequency selection window to increase the weight of low-frequency information while suppressing high-frequency information, ensuring the integrity of the target contour information and effectively controlling high and low-frequency artifact noise.

[0043] Further, in Step 6, process the target signal in combination with the frequency selection window, and then use the non-line-of-sight target reconstruction algorithm of diffraction interpolation to achieve the reconstruction of the hidden target. The specific method is as follows:

[0044] Step 6-1: Assume that there is also a transparent plane M in the simulation scene, and the distance between this plane and the simulation target R is equal to the distance between the transparent plane T and the hidden target V. The interpolation process is to reproduce the impulse response of the diffracted wave on the T plane on the transparent plane M using the lens reconstruction principle; Step 4 realizes the coordinate calibration of N discrete x c points on the relay plane C, and the interpolation process is expressed as:

[0045]

[0046] Assume that the distance between the plane M and the simulation target R is equal to the distance between the transparent plane T and the hidden target V. Represent the impulse response on the T plane with the impulse response on the M plane. At this time, the planes T and M can be regarded as simulating lenses that reconstruct the points x v on the hidden object to the virtual hidden object x r . Among them, the plane T realizes the correction of the spherical wave emitted by the point x v , and the plane M completes the focusing of the point x r . Given the position information and the corresponding impulse response of each grid point x m on the simulated transparent plane M, the impulse response at x r is:

[0047]

[0048] Step 6-2: The ideal reconstruction model of the non-line-of-sight scene can be simplified to the RSD propagation between the simulated transparent plane T and the hidden target, and the RSD propagation between two parallel planes is represented by Fourier transform:

[0049]

[0050] Let Introduce the scalar coordinates x t (x t ,y t ,0) and x v (x v ,y v ),zv ), whose geometric setting omits the relay plane C, makes the planes M and T coincide, and only considers two parallel planes T and the hidden target plane V with a distance of z in the Cartesian coordinate system v , so the RSD symbol changes from the point x v of to to indicate that the diffraction propagation is applicable to all points on the plane at a distance of Z from the plane T. Then the RSD diffraction formula is rewritten as: v For all points at a distance of Z from plane T on the plane, the RSD diffraction formula is rewritten as:

[0051]

[0052] Step 6-3, integrating the calculations of the above two steps, the calculation method of the overall reconstruction is written as:

[0053]

[0054] where ω for integration is the frequency range and its weight obtained in step 5. Assuming that the hidden target only receives direct illumination from the light source and ignoring the mutual reflection in the hidden scene, by calculating the time t for light to propagate from the point light source x p (x p , y p , z p ) on the relay plane to the hidden target voxel x v (x v , y v , z v ), and replacing t on each voxel to achieve:

[0055]

[0056] A computer device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, it implements the non-line-of-sight target reconstruction method under non-planar relay plane sparse sampling to achieve non-line-of-sight target reconstruction under non-planar relay plane sparse sampling.

[0057] A computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the non-line-of-sight target reconstruction method under non-planar relay plane sparse sampling to achieve non-line-of-sight target reconstruction under non-planar relay plane sparse sampling.

[0058] Compared with the prior art, the remarkable advantages of the present invention are:

[0059] 1. For the photon histogram of the echo signal, combined with the improved Richardson-Lucy deconvolution algorithm, introducing the total variation and L2 hybrid regularization constraints, significantly improves the time resolution, suppresses high-frequency noise, and the calculation efficiency is 8 times higher than that of the traditional algorithm;

[0060] 2. The phasor field reconstruction method based on diffraction interpolation solves the problem of the traditional algorithm's dependence on an ideal planar relay surface. This algorithm has comparable or even better reconstruction quality than existing algorithms in non-planar scenarios, and significantly improves the computational efficiency.

[0061] 3. When dealing with extremely sparse sampling points, the strategy of spectrum weight correction using a trailing Gaussian frequency selection window can effectively suppress high- and low-frequency artifact noises, improve the clarity of the reconstructed edge while maintaining the integrity of the target contour, and the structural similarity with the reconstructed result of complete data reaches more than 96%. Description of the Drawings

[0062] Figure 1 It is the working framework diagram of the non-line-of-sight target reconstruction method under sparse sampling of a non-planar relay surface according to the present invention.

[0063] Figure 2 It is the schematic diagram of the scenario of the non-line-of-sight target reconstruction method under sparse sampling of a non-planar relay surface according to the present invention.

[0064] Figure 3 It is the flow chart of the non-line-of-sight target reconstruction method under sparse sampling of a non-planar relay surface according to the present invention.

[0065] Figure 4 It is the comparison diagram of the reconstruction results of different non-planar relay surfaces under extreme sparsification according to the present invention.

[0066] Figure 5 It is the comparison diagram of the reconstruction results of the missing pattern and sparsity degree according to the present invention. Detailed Embodiments

[0067] In order to make the objectives, technical solutions and advantages of the present application clearer, the following further elaborates on the present application in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0068] As Figure 1 shown, it is the schematic diagram of the scenario of a non-line-of-sight target reconstruction method under sparse sampling of a non-planar relay surface.

[0069] The non-line-of-sight target reconstruction system of the photon counting lidar according to the present invention includes:

[0070] At least one precision electric control turntable;

[0071] At least one pulsed laser emission system for emitting narrow pulsed laser and irradiating it on the intermediate surface;

[0072] At least one signal receiving system for receiving photon signals returned from the detection position of the relay surface;

[0073] At least one photon counting module that counts the detected photon signals and simultaneously records the time when the photons reach the detector;

[0074] At least one timing control module that controls the emission of the laser pulse signal, sends a timing start signal to the photon counting module, and generates a gating signal to control the working state of the receiving system.

[0075] At least one data management terminal for storing and preprocessing the photon information returned by the relay surface;

[0076] At least one target reconstruction system for reconstructing the target based on the echo information of the non-line-of-sight target;

[0077] Furthermore, the pulsed laser emission system consists of a laser driving circuit, a pulsed laser, and an emission lens assembly. The pulsed laser selects a 532nm pulsed laser with a pulse width of 5ns. Using a pulsed laser with a narrow pulse width ensures the positioning accuracy of the target. The emission lens assembly is used to shape the laser pulses emitted by the laser.

[0078] Furthermore, the signal receiving system consists of a receiving lens assembly and a single-photon detection assembly. The receiving lens assembly is used to receive the photon signals of the target and transmit them to the single-photon detection assembly. The single-photon detector selects a Geiger-mode APD (SPAD) as the detection assembly, and the SPAD operates in the gated mode.

[0079] Furthermore, the timing control module is connected to the motor control system, the laser emission system, and the signal receiving system, and is used to control the emission of pulsed laser by the laser and the working state of the detector, and simultaneously transmit the time information of the pulse emission to the signal receiving system.

[0080] Furthermore, the laser emission system and the signal receiving system are fixed side by side, and the laser illumination point and the detector detection point are controlled to be in the same position to achieve approximate confocal non-line-of-sight detection.

[0081] Furthermore, the signal receiving system is connected to the photon counting module, and the photon signals collected by the signal receiving system are accumulated to obtain the waveform information of the target.

[0082] Furthermore, the photon counting module is connected to the data management terminal to store and preprocess the first reflection signal and the target echo signal at the detection position, and is used for the calibration of the detection position of the relay surface and the target reconstruction.

[0083] Furthermore, the data management terminal is connected to the non-planar relay surface non-line-of-sight target reconstruction system, and combines the relay surface coordinate information and the target echo signal to complete the three-dimensional reconstruction of the hidden target.

[0084] As Figure 2As shown, a method for non-line-of-sight target reconstruction under sparse sampling of a non-planar relay surface, the specific process includes:

[0085] Step 1, align the photon-counting lidar with the relay surface and control the photon-counting lidar to detect the target;

[0086] Step 1-1, place the photon-counting lidar on a turntable, fix the laser emission system and the signal reception system side by side, and control the laser illumination point and the detector detection point to be in the same position to achieve approximate confocal non-line-of-sight detection.

[0087] Step 1-2, set the emission frequency of the laser, as well as the time delay Δt and the duration of the gating signal in the timing control module. The specific settings need to consider initial parameters such as the detection position, laser pulse width, output delay of the signal reception system, and detection period in the non-line-of-sight scenario;

[0088] Step 1-3, the timing control module transmits a laser emission signal to the laser driver circuit to control the laser to generate pulsed laser, which reaches the relay surface after passing through the emission lens assembly;

[0089] Step 2, use time-correlated single-photon counting technology to statistically analyze the received photon signals to obtain the photon histogram of the target signal;

[0090] Step 2-1, while the timing signal outputs a laser emission signal, the timing control module transmits the gating signal to the signal reception system. After receiving the gating signal, the signal reception system starts to work and receives the photon signals scattered by the target on the relay surface;

[0091] Step 2-2, due to the effect of the gating signal, the detector only collects photon signals within the range gate, and the photon counting module records the time stamps of the photon responses;

[0092] Step 2-3, after n detections, use time-correlated single-photon counting technology to accumulate and statistically analyze the detection data of each time stamp, and then obtain the photon histogram statistical signal of the non-line-of-sight target.

[0093] Step 3, use the hybrid-regularized Richardson-Lucy deconvolution algorithm to improve the time resolution for the photon histogram statistical signal sequence of the relay surface echo and the target echo obtained from Step 2;

[0094] Step 3-1, since the observed photon histogram is actually the convolution of the real-scene response and the laser pulse, it can be mathematically expressed as:

[0095] y(t) = h(t) * g(t) + n(t)

[0096] Among them, y(t) is the observed photon histogram, h(t) is the laser pulse shape, g(t) is the scene response under the ideal impulse response, n(t) is the noise, and * represents the convolution operation. Based on this formula, the scene response under the ideal impulse response can be calculated by subsequent steps.

[0097] Step 3-2, the RL algorithm that combines TV regularization and L2 regularization can theoretically accurately extract data features and has stronger noise robustness. At this time, the iterative formula is:

[0098]

[0099] Consistent with the definition in Step 3-1, y is the observed photon histogram, h is the laser pulse shape, and g k and g k+1 are the results of the previous round and this round of iteration respectively. Thus, the scene response under the ideal impulse response can be finally calculated efficiently for subsequent relay plane calibration and hidden target reconstruction.

[0100] Step 4, use the first peak echo signal of the relay plane to extract the peak, and combine the laser emission angle to calculate the relative coordinates of the sampling points on the non-planar relay plane;

[0101] Step 4-1, adopt dynamic adaptive differential threshold to extract the target peak, detect the difference to determine the position of the main wave of the signal after the first peak echo of the relay plane is processed in Step 3, which can effectively identify the peak value in the case where the tidal wave is higher than the main wave. According to the characteristics of the actual echo signal, adopt dynamic differential threshold to extract the signal peak, and during the extraction process, use the method of symbolic signal approximate matching according to the threshold in a timely manner to detect the signal interference section, which can effectively improve the detection rate of the echo signal peak value.

[0102] Step 4-2, take the laser emission position as the origin, combine the rotation angle of the turntable and the arrival time of the first peak echo signal of the relay plane sampling point obtained in Step 4-1, and the coordinate information of the relay plane sampling point can be calculated by the time-of-flight method. After readjusting the relative coordinates of the relay plane formed by the relay plane sampling points and stipulating the origin at the center of the relay plane, the position calibration of the sampling points on the relay plane is completed for the interpolation calculation from non-planar to planar in subsequent steps.

[0103] Step 5, analyze the spectral characteristics of the photon histogram of the target echo signal, adaptively obtain the effective frequency range for reconstruction, and assign a reasonable weight distribution;

[0104] Step 5-1, determine the maximum resolvable size and the resolvable size at the target depth according to the virtual aperture V_aperture and the maximum target depth Maxdepth set in the non-line-of-sight scene:

[0105]

[0106] Thus, the upper frequency limit fre_bound = c / 2Δx is determined. max And the target frequency center fre_mid = c / 2Δx target .

[0107] Step 5-2: For the hidden target echo signal obtained by sampling, calculate its scene response under the impulse response through Step 3, calculate the spectral amplitude of the target scene response, and calculate the threshold by integrating the average, median, and quartiles of the amplitudes. This threshold can be determined as the lower frequency selection limit for the scene response at this sampling point. Combining with the upper frequency limit and the target frequency center in Step 5-1, the frequency selection range available for reconstruction can be adaptively demarcated.

[0108] Step 5-3: According to the custom-selected number of frequencies, with the target frequency as the center, determine the final frequency range for reconstruction while satisfying the upper and lower frequency limits. To ensure the reconstruction quality under sparse sampling, perform a trailing Gaussian-shaped weighting process on the frequency selection window, appropriately increasing the weight of low-frequency information while suppressing high-frequency information, ensuring the integrity of the target contour information and effectively controlling high- and low-frequency artifact noise.

[0109] Step 6: Process the target signal in combination with the frequency selection window, and then use the non-line-of-sight target reconstruction algorithm of diffraction interpolation to achieve the reconstruction of the hidden target.

[0110] Step 6-1: Assume that there is also a transparent plane M in the simulated scene, and the distance between this plane and the simulated target R is equal to the distance between the transparent plane T and the hidden target V. The interpolation process is to reproduce the impulse response of the diffracted wave on the T plane on the transparent plane M using the lens reconstruction principle. Since the sampling point x c is unevenly sampled on a non-planar surface and its surface shape cannot be described by a specific expression; but Step 4 realizes the coordinate calibration of N discrete x c points on the relay plane C. The interpolation process can be expressed as:

[0111]

[0112] Assume that the distance between the plane M and the simulated target R is equal to the distance between the transparent plane T and the hidden target V. Use the impulse response on the M plane to represent the impulse response on the T plane. At this time, the planes T and M can be regarded as an analog lens that reconstructs the point x v on the hidden object to the virtual hidden object x r . Among them, the plane T realizes the correction of the spherical wave emitted by the point x v , and the plane M completes the focusing of the point x r . Given the position information and the corresponding impulse response of each grid point x m on the simulated transparent plane M, xr The impulse response at

[0113]

[0114] Step 6-2: The ideal reconstruction model for the non-line-of-sight scenario can be simplified to simulate the RSD propagation from the virtual plane T to the hidden target, and the RSD propagation between two parallel planes can be represented by Fourier transform:

[0115]

[0116] Let Introduce the scalar coordinate x t (x t , y t , 0) and x v (x v , y v , z v ). Its geometric setting omits the relay plane C, coincides the planes M and T, and only considers two parallel planes T and the hidden target plane V with a spacing of z v in the Cartesian coordinate system. Therefore, the RSD symbol changes from that at point x v of to to indicate that the diffraction propagation applies to all points on the plane at a distance of Z v from the plane T. Then the RSD diffraction formula can be rewritten as:

[0117]

[0118] Step 6-3: Integrating the calculations of the above two steps, the calculation method for the overall reconstruction can be written as:

[0119]

[0120] where ω for integration is the frequency range and its weight obtained in Step 5. Assume that the hidden target only receives direct illumination from the light source and mutual reflections within the hidden scene are ignored. In this way, the time t for light to propagate from the point light source x p (x p , y p , z p ) on the relay plane to the hidden target voxel x v (x v , y v , z v ) can be calculated and t is replaced at each voxel to achieve:

[0121]

[0122] Replacing t with the accurate spatial coordinates described in the equation, although restricting the optical path to only from the point light source P to the hidden object V and the hidden object directly reflecting to reach the relay plane C, can reconstruct the corresponding voxels at the accurate time when the pulse arrives, which makes the reconstruction more time-saving than obtaining the complete four-dimensional wavefront.

[0123] The present invention also provides a computer device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the non-line-of-sight target reconstruction method under non-planar relay plane sparse sampling is implemented to achieve non-line-of-sight target reconstruction under non-planar relay plane sparse sampling.

[0124] A computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the non-line-of-sight target reconstruction method under non-planar relay plane sparse sampling is implemented to achieve non-line-of-sight target reconstruction under non-planar relay plane sparse sampling.

[0125] The technical features of the above embodiments can be combined arbitrarily. For the sake of concise description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope recorded in this specification.

[0126] The above-described embodiments merely represent several implementation manners of the present application. The description is relatively specific and detailed, but it should not be construed as a limitation to the scope of the present application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A non-line-of-sight target reconstruction system for a photon-counting lidar, characterized in that, Comprising: At least one precision electric control turntable; At least one pulsed laser emission system for emitting narrow pulsed laser and irradiating on an intermediate surface; At least one signal receiving system for receiving photon signals returned from the detection position of a relay surface; At least one photon counting module for counting the detected photon signals and simultaneously recording the time when photons reach the detector; At least one timing control module for controlling the emission of laser pulse signals and simultaneously sending a timing start signal to the photon counting module, and for generating a gating signal to control the working state of the receiving system; At least one data management terminal for storing and preprocessing the photon information returned from the relay surface; At least one target reconstruction system for reconstructing a target based on the echo information of a non-line-of-sight target; The timing control module is connected to the motor control system, the laser emission system, and the signal receiving system, for controlling the emission of pulsed laser by the laser and the working state of the detector, and simultaneously transmitting the time information of the pulse emission to the signal receiving system; The signal receiving system is connected to the photon counting module, and accumulates the photon signals collected by the signal receiving system to obtain the waveform information of the target; The photon counting module is connected to the data management terminal, stores and preprocesses the first reflection signal and the target echo signal at the detection position, and is used for the calibration of the detection position of the relay surface and target reconstruction; The data management terminal is connected to the non-planar relay surface non-line-of-sight target reconstruction system, and completes the three-dimensional reconstruction of the hidden target by combining the relay surface coordinate information and the target echo signal.

2. The photon counting lidar non-line-of-sight target reconstruction system according to claim 1, wherein The pulsed laser emission system is composed of a laser drive circuit, a pulsed laser, and an emission lens assembly. The pulsed laser selects a 532nm pulsed laser with a pulse width of 5ns. The use of a pulsed laser with a narrow pulse width ensures the positioning accuracy of the target. The emission lens assembly is used to shape the laser pulses emitted by the laser; The signal receiving system is composed of a receiving lens assembly and a single photon detection assembly. The receiving lens assembly is used to receive the photon signals of the target and transmit them to the single photon detection assembly. The single photon detector selects a Geiger mode SPAD as the detection assembly, and the SPAD operates in the gating mode.

3. The photon-counting lidar non-line-of-sight target reconstruction system according to claim 1, characterized in that, The laser emission system and the signal receiving system are fixed side by side, and the laser illumination point and the detector detection point are controlled to be in the same position to achieve approximate confocal non-line-of-sight detection.

4. A method for reconstructing non-line-of-sight targets under sparse sampling of a non-planar relay surface, characterized in that, Based on the non-line-of-sight target reconstruction system of the photon counting lidar according to any one of claims 1-3, realizing non-line-of-sight target reconstruction under sparse sampling of a non-planar relay surface, including the following steps: Step 1, aiming the photon counting lidar at the intermediate surface, and controlling the photon counting lidar to detect the relay surface and the target; Step 2, using time-correlated single photon counting technology to statistically analyze the received photon signals to obtain the photon histograms of the first echo signal of the relay surface and the target signal; Step 3, statistically analyzing the signal sequences of the photon histograms of the relay surface echo and the target echo obtained in step 2, and using the Richardson-Lucy deconvolution algorithm with hybrid regularization to improve the time resolution; Step 4, performing peak extraction on the relay surface echo signal, and combining the laser emission angle to calculate the relative coordinates of the sampling points on the non-planar relay surface; Step 5: Analyze the spectral characteristics of the photon histogram of the target echo signal, adaptively obtain the effective frequency range for reconstruction, and assign a weight distribution. Step 6: Process the target signal in combination with a frequency selection window, and then use the diffraction interpolation non-line-of-sight target reconstruction algorithm to realize the reconstruction of the hidden target.

5. The non-line-of-sight target reconstruction method under non-planar relay surface sparse sampling according to claim 4, wherein Step 3: For the photon histogram statistical signal sequence of the relay surface echo and the target echo obtained from Step 2, use the Richardson-Lucy deconvolution algorithm with hybrid regularization to improve the time resolution. The specific method is as follows: Step 3-1: Since the observed photon histogram is actually the convolution of the real scene response and the laser pulse, which is mathematically expressed as: y(t) = h(t) * g(t) + n(t) where y(t) is the observed photon histogram, h(t) is the laser pulse shape, g(t) is the scene response under the ideal impulse response, n(t) is the noise, and * represents the convolution operation. Step 3-2: At the same time, combine the RL algorithm with TV regularization and L2 regularization, and iteratively solve the scene response under the ideal impulse response to improve the time resolution. The iterative formula is: Among them, g k and g k+1 are the results of the previous round and the current round of iteration respectively.

6. The non-line-of-sight target reconstruction method under non-planar relay surface sparse sampling according to claim 4, characterized in that, Step 4: Use the relay surface echo signal for peak extraction, and combine the laser emission angle to calculate the relative coordinates of the sampling points on the non-planar relay surface. The specific method is as follows: Step 4-1: Adopt dynamic adaptive differential threshold to extract the target peak, detect the difference to determine the position of the main wave of the signal after the first peak echo of the relay surface is processed in Step 3, and identify the peak value in the case where the tidal wave is higher than the main wave. Step 4-2: Take the laser emission position as the origin, combine the rotation angle of the turntable and the arrival time of the first peak echo signal of the relay surface sampling point obtained in Step 4-1, and use the time-of-flight method to calculate the coordinate information of the relay surface sampling point. After readjusting the relative coordinates of the relay surface formed by the relay surface sampling points and specifying the origin at the center of the relay surface, complete the position calibration of the sampling points on the relay surface for the subsequent interpolation calculation from non-planar to planar.

7. The non-line-of-sight target reconstruction method under non-planar relay surface sparse sampling according to claim 4, wherein Step 5: Analyze the spectral characteristics of the photon histogram of the target echo signal, adaptively obtain the effective frequency range for reconstruction, and assign a weight distribution. The specific method is as follows: Step 5-1: Determine the maximum resolvable size and the resolvable size at the target depth according to the virtual aperture V_aperture and the maximum target depth Maxdepth set for the non-line-of-sight scene. Thus, the upper frequency limit fre_bound = c / 2Δx is determined max and the target frequency center fre_mid = c / 2Δx trget ; Step 5-2: Calculate the spectral amplitude of the target scene response by calculating the scene response under the impulse response of the hidden target echo signal obtained by sampling in Step 3. Calculate the threshold by combining the average, median, and quartiles of the amplitude. Determine this threshold as the lower limit of the frequency selection for the scene response of this sampling point. Combine the upper limit of the frequency in Step 5-1 and the target frequency center to adaptively delineate the frequency selection range for reconstruction. Step 5-3: According to the custom-selected number of frequencies, centered around the target frequency, while satisfying the upper and lower limits of the frequency, determine the final frequency range for reconstruction. To ensure the reconstruction quality under sparse sampling, perform a trailing Gaussian weight processing on the frequency selection window to increase the weight of low-frequency information while suppressing high-frequency information, ensuring the integrity of the target contour information and effectively controlling high- and low-frequency artifact noise.

8. The non-line-of-sight target reconstruction method under non-planar relay surface sparse sampling according to claim 4, wherein Step 6: Process the target signal in combination with the frequency selection window, and then use the diffraction interpolation non-line-of-sight target reconstruction algorithm to achieve the reconstruction of the hidden target. The specific method is as follows: Step 6-1, assume that there is also a transparent plane M in the simulation scenario, and the distance between this plane and the simulation target R is equal to the distance between the transparent plane T and the hidden target V. The interpolation process is to reproduce the impulse response of the diffracted wave on the T plane on the transparent plane M using the lens reconstruction principle; Step 4 realizes the coordinate calibration of N discretized x c points on the relay plane C, and the interpolation process is expressed as: Assume that the distance between the plane M and the simulated target R is equal to the distance between the transparent plane T and the hidden target V. Represent the impulse response on T with the impulse response on M. At this time, the planes T and M can be regarded as analog lenses that reconstruct the point x on the hidden object v to the virtual hidden object x r where the plane T realizes the correction of the spherical wave emitted by the point x v and the plane M completes the focusing of the point x r Given the position information and the corresponding impulse response of each grid point x on the analog transparent plane M m the impulse response at x r is as follows: Step 6-2: The ideal reconstruction model of the non-line-of-sight scenario can be simplified to simulate the RSD propagation from the virtual transparent plane T to the hidden target, and the RSD propagation between two parallel planes is represented by Fourier transform: Let introduce the scalar coordinate x t (x t , y t , 0) and x v (x v , y v , z v ), whose geometric setting omits the relay surface C, coincides the planes M and T, and only considers two parallel planes T and the hidden target plane V with a spacing of z v in the Cartesian coordinate system. Therefore, the RSD symbol changes from the point x v of to to indicate that the diffraction propagation applies to all points on the plane at a distance Z v from the plane T. Then the RSD diffraction formula is rewritten as: Step 6-3: Integrate the calculations of the above two steps, and write the calculation method for the overall reconstruction as: Among them, ω for integration is the frequency range and its weight obtained in step 5. Assuming that the hidden target only receives direct illumination from the light source and ignoring the mutual reflection within the hidden scene, by calculating the time t for light to propagate from the point light source x p (x p ,y p ,z p ) to the hidden target voxel x v (x v ,y v ,z v ), and replacing t on each voxel to achieve:

9. A computer device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, it implements the non-line-of-sight target reconstruction method under non-planar relay plane sparse sampling according to any one of claims 4-8, and realizes the non-line-of-sight target reconstruction under non-planar relay plane sparse sampling.

10. A computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the non-line-of-sight target reconstruction method under non-planar relay plane sparse sampling according to any one of claims 4-8, and realizes the non-line-of-sight target reconstruction under non-planar relay plane sparse sampling.

Images