An improved method for time-correlated single-photon three-dimensional imaging
Through the time-dependent single-photon counting lidar system and instrument response function fitting technology, the coarse and noise problems of depth estimation in the existing technology are solved, stable and accurate depth estimation is achieved, and the application capabilities of three-dimensional imaging technology are improved.
Patent Information
- Application Number
- CN202310511886.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-08
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2043-05-08
AI Technical Summary
Existing time-related single-photon three-dimensional imaging technology requires rough estimation of depth, which affects its application capabilities, and the noise and false alarm problems have not been effectively solved.
The X-Y plane scan is performed using the time-dependent single-photon counting lidar system to obtain the photon counting histogram, fit the instrument response function curve, calculate the optimal depth estimation position, filter out noise through the gating time-dependent single-photon counting method, and use electromagnetic formula A two-dimensional orthogonal scanning galvanometer and signal acquisition card realize raster scanning, using two generalized Gaussian functions to fit the instrument response function, and calculate the depth map.
It realizes stable and accurate estimation of depth, improves the application capabilities of three-dimensional imaging technology, reduces the impact of noise and false alarms, and improves the robustness of imaging effects.
Smart Images

Figure CN116540261B_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the technical field of laser radar, and in particular relates to an improved time-correlated single-photon three-dimensional imaging method. Background Art
[0002] Traditional 3D imaging LiDAR is a common active optical 3D imaging system, which has been widely used in industry, scientific research and even defense. However, with the increasing maturity of high-sensitivity photon detection technology and high-precision electronic timing technology, LiDAR technology represented by time-correlated single photon counting 3D imaging technology has gradually become a development trend in the field of long-distance imaging and non-line-of-sight imaging.
[0003] Traditional single-photon 3D imaging algorithms, such as the maximum likelihood method under the Poisson noise assumption, usually perform logarithmic matched filtering on the radar waveform under the premise of a single peak assumption to find the maximum value of the filtered data for each pixel. In addition, Altmann et al. proposed a Bayesian method, Shin et al., Halimi et al., and Rapp et al. proposed three different optimization schemes, and Li et al. proposed a method using a 3D deconvolution algorithm to achieve 8.2km super-resolution imaging. This method can achieve single-photon 3D imaging up to 200km.
[0004] All of the above methods have a common feature, that is, they require a rough estimate of the depth to obtain a depth map as the input of the algorithm.
[0005] Therefore, it is currently necessary to explore an algorithm that can stably and accurately estimate the depth in order to enhance the practical application capability of time-correlated single photon counting three-dimensional imaging technology. Summary of the invention
[0006] The present invention provides an improved time-correlated single-photon three-dimensional imaging method, comprising the following steps:
[0007] Step 1, use a time-correlated single photon counting laser radar system to scan the object to be measured in an XY two-dimensional plane, and obtain a histogram of the photon count of each pixel in the scanning plane;
[0008] Step 2: Fitting the instrument response function curve according to the acquired photon count histogram. Since the instrument response function model is a unimodal function, it has only one rising edge.
[0009] Step 3: Calculate the best depth estimation position of each pixel based on the curve of the fitted instrument response function, and calculate the depth of each pixel in units of distance based on this, and restore the depth map.
[0010] Furthermore, the time-correlated single photon counting laser radar system described in step 1 uses a gated time-correlated single photon counting method to obtain a photon count histogram. Here, gating refers to selecting a period of time as the gate position within the entire synchronization signal cycle and ensuring that the echo photon signal is within the gate. This method can effectively filter out noise photons and false alarms by setting a reasonable gate width and selecting appropriate start and end positions of the gate.
[0011] The laser radar system uses an electromagnetic two-dimensional orthogonal scanning galvanometer for raster scanning. The scanning galvanometer is divided into a horizontal lens and a vertical lens, which control the horizontal and vertical scanning directions respectively. When the signal acquisition card outputs a fixed voltage, the horizontal and vertical lenses are maintained in a fixed position. The voltage sequence V is written using the signal acquisition card. X and V Y , set the scan time T for each pixel x,y , realizing raster scanning of lidar.
[0012] The time-correlated single-photon counting method is based on the accumulation of photon events within multiple laser pulse synchronization cycles, and can recover the echo photon signal. Based on the synchronization signal and the detection of the echo photons, the photon count histogram of the pixel point can be obtained. That is, the laser pulse emitted by the lidar is first marked as a synchronization signal, and the time difference between the first detected echo photon signal and the synchronization signal in each synchronization signal cycle is recorded. According to this time difference, each photon event will be classified into a different histogram time bin (the horizontal axis of the histogram is the discrete time, usually called a timestamp (bin). Generally, the time interval of each bin is in the order of ps; and the vertical axis of the histogram represents the photon count in this bin). After multiple cycles of photon event accumulation, a complete photon count histogram data is finally obtained.
[0013] Furthermore, the function model used for fitting in step 2 is a two-segment generalized Gaussian function, and the specific expression of the instrument response function curve is:
[0014]
[0015] Where t represents time, q = {α, t p ,σ l ,σ r ,b},α is the amplitude coefficient, t p is the time corresponding to the peak value, σ l It means when t <t p The width coefficient of the time function, σ r It means that when t≥t pis the width coefficient of the time function, and b represents the background noise. The parameter set q can be fitted by the least squares method, and the function f(t;q) can be uniquely determined by the parameter set q.
[0016] Using f i (t;q i ) represents the instrument response function curve after fitting the i-th pixel. The specific expression is:
[0017]
[0018] Among them, q i is the single photon count histogram N of the ith pixel i The corresponding fitting parameters are
[0019] q i ={α i ,t pi ,σ li ,σ ri ,b i}.
[0020] Furthermore, the calculation method of the optimal depth estimation position in step 3 can be specifically expressed as:
[0021] d i(bin) =[f i -1 (f i >λ×(α i +b i ))] Left
[0022] Where, the subscript i refers to the i-th pixel, and d i(bin) is the depth estimate of the i-th pixel in units of timestamp (bin), f i represents the instrument response function of the ith pixel obtained by fitting the photon counting histogram by the least squares method, then f i -1 It represents f i is the inverse function of the rising edge. λ is the position factor of the rising edge. The value range of λ is (0,1], which means from the bottom to the highest point of the rising edge. i and b i is the fitting parameter of the photon counting histogram. Since the expression f i -1 (f i >λ×(α i +b i )) gets a time stamp (bin) interval, using [] Left Take the first integer on the left side of the interval as the best depth estimate for the pixel.
[0023] Furthermore, the value of λ is taken as the maximum value of the slope at the rising edge of the fitting curve. First, the reason why the depth is estimated based on the rising edge rather than the falling edge or the highest point is that the rising edge of the instrument response function fitting model described in step 2 is steeper than the falling edge. In terms of numerical expression, σ l <<σ r , the two usually differ by more than 50 times. If the edge of a pulse is smoother and the pulse is wider, the corresponding time uncertainty will be greater and the accuracy will be lower; conversely, the steeper the edge of a pulse and the narrower the pulse, the corresponding time uncertainty will be smaller and the accuracy will be higher. Therefore, in order to determine the most appropriate rising edge position factor, take the λ that makes the slope of the fitting curve at the rising edge reach the maximum value. At this time, the position corresponding to the λ has the smallest uncertainty in time. The value of this λ is approximately equal to 0.6. Subsequently, through experimental verification, it was found that the performance difference between λ at the maximum slope and λ taking 0.5 was not large, and the stability of the depth estimation values obtained was greater than that of λ taking other values. The slope of the instrument response function when λ takes 0.5 was calculated, and it was found that the slope reached 97% of the maximum value. Therefore, for convenience, λ=0.5 was taken in subsequent experiments;
[0024] Since the depth is estimated based on the position of the rising edge of the signal, there is a fixed deviation between it and the depth estimation obtained by other methods. This deviation can be eliminated by adding a fixed system error later. Furthermore, in step 3, the depth is calculated using the time-of-flight method, that is, the signal light is emitted by the laser, and is reflected after encountering an object. By calculating the time difference between the emission and the reflection of the light, the distance of the photographed object is converted to generate depth information. It can be specifically expressed as:
[0025]
[0026] Among them, d i(m) is the depth estimate of the i-th pixel in meters, d i(bin) is the depth estimate of the i-th pixel in units of timestamps (bin); r is the time resolution of time-correlated single-photon counting, that is, the length of continuous time corresponding to each discretized timestamp (bin), usually in the order of ps; c is the speed of light; the 2 in the denominator represents the flight distance, which includes both round trips. BRIEF DESCRIPTION OF THE DRAWINGS
[0027] Figure 1 is a flow chart of the method of the present invention.
[0028] Figure 2 It is a diagram of the experimental device of the present invention.
[0029] Figure 3 is the fitted curve graph of the instrument response function.
[0030] Figure 4 It is the Allen curve diagram of different rising edge position factors λ.
[0031] Figure 5 It is a horizontal depth step scan and a schematic diagram of the actual object.
[0032] Figure 6 This is a comparison chart of the mean and standard deviation of different algorithms.
[0033] Figure 7 This is the depth estimation result diagram of horizontal steps using different algorithms. DETAILED DESCRIPTION
[0034] In order to deepen the understanding of the present invention, the present invention will be further described in detail below in conjunction with examples. The examples are only used to explain the present invention and do not constitute a limitation on the protection scope of the present invention.
[0035] An improved time-correlated single-photon 3D imaging method, such as Figure 1 As shown, the following steps are included:
[0036] Step 1, use a time-correlated single photon counting laser radar system to scan the object to be measured in an XY two-dimensional plane, and obtain a histogram of the photon count of each pixel in the scanning plane;
[0037] The present invention adopts a single-photon laser radar system device structure diagram as shown in Figure 2 As shown. A pulsed laser with a wavelength of 830nm and a frequency of 500kHz is used. After the laser is emitted, it is first calibrated by the movable lens A (Lens A) and the fixed lens B (Lens B), and then scanned by an electromagnetic two-dimensional scanning galvo (Scanning galvo), and the number of scanning pixels is set to n. X ×n Y , set the horizontal and vertical scanning voltage range: V X and V Y . For V X and V Y Separately carry out n X and n Y Divide equally and sequence into voltage sequence V X and V Y The receiving end uses a single photon avalanche diode (SPAD) to detect photon events, and a fixed gate width time-correlated single photon counting (TCSPC) to record the photon count and photon arrival time of each pixel;
[0038] In gated mode, a reasonable gate width can effectively filter out noise photons and false alarms;
[0039] The specific modeling of the single-photon lidar simulation system is as follows:
[0040] (1) The number of photons recorded at the position (x, y) within the time interval n can be expressed as:
[0041]
[0042] Where c is the speed of light, s is the periodic signal light pulse, d is the depth value at a certain pixel, Δt is the time interval of TDC, g represents the actual instrument response function, η∈[0,1) represents the quantum efficiency, and b represents the influence of ambient light;
[0043] (2) The number of photons recorded during N illumination cycles of the SPAD can be approximately modeled as a Poisson process as follows:
[0044] h x,y [n]~P{N[ηr x,y [n]+b d ]}
[0045] where b d Non-zero count for detector;
[0046] Step 2, fitting a curve of the instrument response function according to the acquired photon counting histogram;
[0047] like Figure 3 As shown, the scatter plot is a photon count histogram, and the smooth curve is an instrument response function curve based on a generalized Gaussian function fit of two segments. The X-axis of the figure represents the timestamp (bin), and the Y-axis represents the photon count;
[0048] Step 3: Find the best depth estimation position according to the instrument response function, calculate the depth accordingly, and restore the depth map.
[0049] like Figure 4 As shown in the figure, different λ (rising edge position factor) values were tested and the corresponding Allen curves were drawn. The Allen curve is a commonly used curve to measure the stability of a system or method. In the figure, the X-axis represents the number of photon events, and the Y-axis represents the standard deviation of the estimated depth at the current number of photon events. The two Allen curves at the bottom are from the λ (approximately equal to 0.6) corresponding to the maximum slope of the rising edge of the instrument response function and when λ is 1 / 2. At these two λ values, when the number of photons is the same, the system has the smallest standard deviation and the best performance.
[0050] The standard deviation is calculated as follows: all photon events are equally divided into arrays, and an estimated depth is determined after fitting each group of data, and the standard deviation of this series of estimated depths is calculated. Under the premise that the total number of photon events remains unchanged, the number of photons in each group is changed, and their depths are fitted and estimated again, and the standard deviation of this series of estimated depths is calculated. The above process is repeated until the number of photon events in each group reaches a threshold (close to 250,000 in the figure).
[0051] like Figure 5 As shown, the method of the present invention is used to scan and test an object with a stepped depth distribution on a horizontal line, and the depth data of 150 pixels are collected. There are 4 layers of depth steps, marked as ①, ②, ③ and ④, and there is a 3cm depth difference between each layer. The acquisition time of a single pixel is 0.5s, and the rising edge position factor λ is 0.5.
[0052] The final test results are as follows Figure 6 , Figure 7 As shown. The method of the present invention is compared with the method proposed in Shin's paper Photon-efficient super-resolution laser radar, published in 2017 at the Massachusetts Institute of Technology, and the traditional instrument response function (IRF) fitting method. The traditional instrument response function fitting method refers to a method in which the bin where the peak value of the fitting function is located is used as the depth estimation method after fitting the photon counting histogram. Compared with the other two methods, the imaging effect of the method of the present invention is more stable, and a more robust depth estimation can be achieved.
Claims
1. An improved time-correlated single-photon 3D imaging method, It is characterized in that The steps include: Step 1, using a time-correlated single photon counting laser radar system to scan the object to be measured in an XY two-dimensional plane, and obtaining a histogram of the photon count of each pixel in the scanning plane; Step 2: Fit the instrument response function curve based on the acquired photon counting histogram. The specific expression is: Among them, f i (t;q i ) is the function fitted to the i-th pixel, t represents time, q i is the single photon count histogram N of the ith pixel i The corresponding fitting parameter, q i ={α i ,t pi ,σ li ,σ ri ,b i }, α i is the amplitude coefficient, t pi is the time corresponding to the peak value, σ li It means when t <t pi The width coefficient of the time function, σ ri It means that when t≥t pi The width coefficient of the time function, b i represents background noise; Step 3, according to the curve of the fitted instrument response function, the best depth estimation position of each pixel is calculated, and the depth of each pixel in units of distance is calculated accordingly to restore the depth map; The calculation method of the optimal depth estimation position is specifically expressed as follows: Where, the subscript i refers to the i-th pixel, and d i(bin) is the depth estimate of the i-th pixel in timestamp units, f i represents the instrument response function of the ith pixel obtained by fitting the photon counting histogram by the least squares method, then It represents f i is the inverse function of the rising edge, λ is the position factor of the rising edge, and the value range of λ is (0,1], α i and b i is the fitting parameter of the photon counting histogram; since the expression What we get is a time stamp interval, using [] Left Take the first integer on the left side of the interval as the best depth estimate for the pixel.
2. According to claim 1, an improved time-correlated single-photon three-dimensional imaging method, It is characterized in that The time-correlated single photon counting laser radar system described in step 1 uses a gated time-correlated single photon counting method to obtain a photon counting histogram.
3. According to claim 1, an improved time-correlated single-photon three-dimensional imaging method, It is characterized in that The laser radar system uses an electromagnetic two-dimensional orthogonal scanning galvanometer for raster scanning. The scanning galvanometer includes a horizontal lens and a vertical lens, which respectively control the horizontal and vertical scanning directions. When the signal acquisition card outputs a fixed voltage, the horizontal and vertical lenses are maintained at a fixed position. The signal acquisition card is used to write a voltage sequence V X and V Y , set the scan time T for each pixel x,y , realizing raster scanning of lidar.
4. The improved time-correlated single-photon three-dimensional imaging method according to claim 1, It is characterized in that The value of λ is chosen so that the slope of the rising edge of the fitting curve reaches the maximum value.
5. The improved time-correlated single-photon three-dimensional imaging method according to claim 1, It is characterized in that The value of λ is 0.
5.
6. The improved time-correlated single-photon three-dimensional imaging method according to claim 1, It is characterized in that The depth is calculated using the time-of-flight method, that is, the laser emits a signal light, which is reflected after encountering an object. The time difference between the emission and the reflection of the light is calculated, and the distance of the photographed object is converted to generate depth information; the specific expression is: Among them, d i(m) is the depth estimate of the i-th pixel in units of distance, d i(bin) is the depth estimate of the i-th pixel in timestamp units; r is the temporal resolution of time-correlated single photon counting, that is, the length of continuous time corresponding to each discretized timestamp, and c is the speed of light.
