Single-photon three-dimensional imaging method and system based on Fourier correlated imaging

Abstract

The invention discloses a single-photon three-dimensional imaging method and system based on Fourier correlated imaging. The method comprises the following steps: step 1, generating a series of Fourier stripes after laser pulses irradiate a digital micromirror array; 2, projecting the fringes onto an object, and homogenizing and filtering reflected light through a diffusion sheet and a filter; 3, acquiring a flight time histogram corresponding to each stripe by using a single-photon detector and a counter; and step 4, slicing a plurality of peak values of the histogram to obtain a photon number as a light intensity value for correlation reconstruction, and obtaining images of different depths. As the single-photon detector is easily disturbed by background noise, the background noise can be effectively eliminated by using the Fourier correlation imaging algorithm; the three-dimensional information and the intensity information of the object can be obtained at the same time through histogram partition reconstruction, the problems that an existing single-photon imaging technology is low in efficiency and limited in imaging quality are solved, and complex imaging system design is avoided.

Description

Technical Field

[0001] The present invention relates to the field of optical imaging technology, and in particular to a single-photon three-dimensional imaging method and system based on Fourier correlation imaging. Background Art

[0002] In modern optical detection, low-illuminance, long-distance depth imaging is very important, and has broad application prospects in the fields of national defense and underwater resource exploration. Traditional imaging equipment such as digital cameras are mainly based on area array charge-coupled devices (CCDs), which can obtain high-resolution scene patterns through a single exposure and have been widely used in real life. However, the current silicon-based CCD has a low detection efficiency for non-visible bands and cannot meet the needs of detection under extremely weak light conditions. At the same time, CCD cameras based on intensity detection lose the depth information of objects. Therefore, by introducing single-photon correlation imaging, the present invention realizes the simultaneous acquisition of the reflectivity and depth information of the scene under extremely weak light conditions.

[0003] Correlation imaging is a method of using a single-pixel detector to collect light intensity fluctuations for imaging. It mainly uses a digital micromirror array to load a pattern to encode the light field, then illuminates the object, and finally correlates the collected light intensity value with the pattern to obtain the reflectivity information of the object.

[0004] Single-photon detectors (SPADs) can convert single photons into electronic pulses. Compared with traditional CCDs, they have the advantages of wide response band and high detection sensitivity. Time-correlated single-photon counting (TCSPC) can accurately measure the relationship between the arrival time of photons and pulses. This process can be imagined as a stopwatch, where each emitted sample photon records its arrival time. Thousands of arrival times are iteratively recorded and sorted into a histogram by arrival time to show their distribution. The distance between the object and the detector can be calculated by obtaining the flight time at the peak of its distribution.

[0005] Traditional methods based on correlation imaging can obtain reflectivity information of objects, but they are still unable to detect the depth of objects. At the same time, due to the influence of environmental noise, the imaging quality needs to be improved. Therefore, by combining the advantages of correlation imaging and single-photon imaging, the present invention proposes a single-photon three-dimensional imaging method and system based on Fourier correlation imaging. Since only the fluctuation of light intensity needs to be detected for differential correlation, this method and system have significant advantages when facing complex environmental disturbances. Summary of the invention

[0006] In view of the problems existing in the prior art, the present invention provides a single-photon three-dimensional imaging method and system based on Fourier correlation imaging.

[0007] The technical solution adopted by the present invention is:

[0008] A single-photon three-dimensional imaging method and system based on Fourier correlation imaging, comprising the following steps:

[0009] Step S1: The laser pulse irradiates the digital micromirror array to generate a series of Fourier fringe patterns;

[0010] Step S2: Project the stripes obtained in step 1 onto the object, and then use a diffuser and a filter to homogenize and filter the reflected light;

[0011] Step S3: using a single photon detector and a counter to perform detection and time stamp recording, and obtaining a flight time histogram corresponding to each fringe according to a synchronization signal of a digital micromirror array;

[0012] Step S4: Slice the multiple peaks of the histogram to obtain the number of photons as the light intensity value for correlation reconstruction to obtain images at different depths.

[0013] The step S1 adopts: a 1550nm pulse light source in the atmospheric window with a pulse width of 100ps, which is expanded by a fiber coupler and a collimating beam expander; a digital micromirror array encodes the light beam by pre-loaded Fourier fringes to obtain a fringed light field. Furthermore, the number of Fourier fringes is determined according to the resolution of the image to be imaged.

[0014] The step S2 adopts: a diffuser to make the reflected light from the object uniform and expand the received field of view. A 1550nm bandpass filter is used to filter the reflected light from the object to reduce the noise caused by the ambient light.

[0015] The step S3 adopts: inputting the pulse output by the single photon detector and the synchronization pulse of the digital micromirror array into the single photon counter for synchronization, obtaining the timestamps of multiple pulses corresponding to each frame of the Fourier pattern, and then sorting them according to the number of arrival times to obtain a flight time histogram.

[0016] The step S4 adopts: obtaining the flight time corresponding to multiple peaks of the flight time histogram as the depth information of the object, and the depth can be calculated by the following formula:

[0017] l=v*t / 2

[0018] Where l is the distance from the object surface to the detector, v is the speed of light, and t is the flight time. The light intensity I reflected by an object at depth l illuminated by a single fringe image is:

[0019]

[0020] Among them, P irepresents the number of photons recorded at the ith flight time. The value range of k is (peak-k / 2, peak+k / 2], where peak is the histogram index corresponding to the peak value. The Fourier spectrum of the object can be obtained by the following formula:

[0021] F(f x ,f y )=[I0(x,y)-I π (x,y)]+j·[I π / 2 (x,y)-I 3π / 2 (x,y)]

[0022] Where I0, I π / 2 , I π , I 3π / 2 It represents the four-step phase shift stripes loaded by the digital micromirror array. Finally, the image O of the object is obtained by inverse Fourier transform:

[0023] O=IFFT(F)

[0024] Here, IFFT represents two-dimensional inverse Fourier transform.

[0025] In particular, by selecting other peak values ​​and recalculating the light intensity, step S4 is repeated to obtain the images of objects at different depths in association.

[0026] A single-photon three-dimensional imaging system based on Fourier correlation imaging includes the following modules:

[0027] M1: Fourier fringe projection module, which generates a series of fringe patterns of different frequencies by irradiating a pulsed laser onto a digital micromirror array, and then projects them onto the object using a projection lens.

[0028] M2: Light receiving module. The object illuminated by a series of stripe light fields emits diffuse reflected light, which is homogenized by the diffuser and filtered out by the filter to remove noise light in other bands before being collected by the single-photon detector.

[0029] M3: Post-processing module, consisting of TCSPC single photon counter and computing terminal.

[0030] The M1 adopts: inputting the synchronization signal of the digital micromirror array into the single photon counter as a reference for switching the next frame of stripes. In particular, at this time, the distance l' from the digital micromirror array to the projection lens, the focal length f of the projection lens, and the distance l from the object to the lens satisfy the formula:

[0031]

[0032] Therefore, when imaging at close range, it is necessary to estimate the approximate distance between the object and the system in advance and adjust the position of the projection lens. For imaging at long distance, the object is approximately at infinity, and l' is equal to the focal length f.

[0033] The M2 adopts: the dead time and detection efficiency (PDE) of the single-photon detector need to be set according to the frequency of the pulsed laser to ensure that at most one photon event is recorded per pulse, otherwise it will lead to the problem of recording multiple timestamps in a single pulse cycle, and the exact flight time cannot be obtained.

[0034] The M3 adopts: firstly, the collected tens of thousands of light pulse flight times are divided according to different projection fringe patterns, and then a histogram is performed to obtain the peak number of photons, which is the light intensity I reflected by the object:

[0035]

[0036] Among them, P i Represents the number of photons recorded at the ith flight time. The value range of k is (peak-k / 2,peak+k / 2], where peak is the histogram index value corresponding to the peak value.

[0037] The image of the object can be obtained by calculating the peak light intensity corresponding to different fringe patterns. x ,f y ) can be obtained by the following formula:

[0038] F(f x ,f y )=[I0(x,y)-I π (x,y)]+j·[I π / 2 (x,y)-I 3π / 2 (x,y)]

[0039] Where I0, I π / 2 , I π , I 3π / 2 It represents the four-step phase shift stripes loaded by the digital micromirror array. Finally, the image O of the object is obtained by inverse Fourier transform:

[0040] O=IFFT(F)

[0041] Here, IFFT represents two-dimensional inverse Fourier transform.

[0042] Compared with the prior art, the present invention has the following beneficial effects:

[0043] (1) The present invention uses a digital micromirror array to replace the scanning galvanometer in the single-photon radar, which greatly simplifies the complexity of the system and reduces the local noise introduced by the complex system;

[0044] (2) The present invention uses four-step phase-shifted Fourier fringes as the modulated light field, and can remove the interference of background noise through differential operation to improve the imaging quality;

[0045] (3) The present invention divides the flight time histogram into different regions according to the peak values, so as to obtain information about the objects at different depths. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 This is a schematic diagram of the single-photon three-dimensional imaging device of the present invention.

[0047] Figure 2 It is a flight time histogram corresponding to a single fringe pattern obtained by the present invention.

[0048] Figure 3 Schematic diagram of the target to be imaged.

[0049] Figure 4 Schematic diagram of single-photon imaging results at different depths.

[0050] Figure 5 Schematic diagram of single-photon three-dimensional imaging results. DETAILED DESCRIPTION

[0051] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments.

[0052] like Figure 1 As shown, a single-photon three-dimensional imaging method based on Fourier correlation imaging includes the following steps:

[0053] Step S1: The laser pulse irradiates the digital micromirror array to generate a series of Fourier fringe patterns;

[0054] Step S2: Project the stripes obtained in step 1 onto the object, and homogenize and filter the reflected light through a diffuser and a filter;

[0055] Step S3: using a single photon detector and a counter to obtain a flight time histogram corresponding to each fringe;

[0056] Step S4: Slice the multiple peaks of the histogram to obtain the number of photons as the light intensity value for correlation reconstruction to obtain images at different depths.

[0057] Specifically, step S1 uses: a 1550nm pulse light source in the atmospheric window, which has the advantages of strong penetration and low background noise in the case of long-distance detection. The laser pulse width is 100ps, the repetition frequency is 8MHz, and the beam is expanded by a fiber coupler and a collimating beam expander; the digital micromirror array encodes the light beam through pre-loaded Fourier fringes to obtain a striped light field. Furthermore, the number of Fourier fringes is determined according to the resolution of the image to be imaged.

[0058] Specifically, the step S2 adopts: the object to be detected is as follows: Figure 2 As shown, the two cubes are located at different depths. The diffuser makes the reflected light from the object uniform and expands the received field of view. A 1550nm bandpass filter is used to filter the reflected light from the object to reduce the noise caused by ambient light.

[0059] Specifically, step S3 adopts: inputting the pulse output by the single photon detector and the synchronization pulse of the digital micromirror array into the single photon counter for synchronization, obtaining the timestamps of multiple pulses corresponding to each frame of the Fourier pattern, and then sorting them according to the number of arrival times to obtain a flight time histogram. Figure 3 This is a flight time histogram collected when a single frame Fourier pattern is irradiated onto an object. Two peaks appear in it, indicating Figure 2 The positions of two objects at different depths.

[0060] Specifically, the step S4 adopts: obtaining the flight time corresponding to multiple peaks of the flight time histogram as the depth information of the object, and the depth can be calculated by the following formula:

[0061] l=v*t / 2

[0062] Where l is the distance from the object surface to the detector, v is the speed of light, and t is the flight time. The light intensity I reflected by an object at depth l illuminated by a single fringe image is:

[0063]

[0064] Among them, P i represents the number of photons recorded at the ith flight time. The value range of k is (peak-k / 2, peak+k / 2], where peak is the histogram index corresponding to the peak value. The Fourier spectrum of the object can be obtained by the following formula:

[0065] F(f x ,f y )=[I0(x,y)-I π (x,y)]+j·[I π / 2 (x,y)-I 3π / 2 (x,y)]

[0066] Where I0, I π / 2 , I π , I 3π / 2 It represents the four-step phase shift stripes loaded by the digital micromirror array. Finally, the image O of the object is obtained by inverse Fourier transform:

[0067] O=IFFT(F)

[0068] Here, IFFT represents two-dimensional inverse Fourier transform.

[0069] The image of the object obtained by adding the number of photons in the entire histogram as the light intensity and performing correlation operation is as follows: Figure 4 As shown in (a). Since the information of the two peaks is used at the same time, objects at different depths are reconstructed into the same image. However, the information between different objects will interfere with each other, resulting in poor reconstruction quality of the cube on the left. Figure 3 The image obtained by calculating the number of photons of peak1 is Figure 4 (b) Select Figure 3 The image obtained by calculating the number of photons of peak2 is Figure 4 (c). Objects at different depths are reconstructed separately. The depth distance between the two objects can be calculated based on the time difference between the two peaks, and the three-dimensional image of the object can be obtained, such as Figure 5 shown.

[0070] The single-photon three-dimensional imaging system based on Fourier correlation imaging is as follows: Figure 1 As shown, it includes the following modules:

[0071] M1: Fourier fringe projection module, which generates a series of fringe patterns of different frequencies by irradiating a pulsed laser onto a digital micromirror array, and then projects them onto the object using a projection lens.

[0072] M2: Light receiving module. The object illuminated by a series of stripe light fields emits diffuse reflected light, which is homogenized by the diffuser and filtered out by the filter to remove noise light in other bands before being collected by the single-photon detector.

[0073] M3: Post-processing module, consisting of TCSPC single photon counter and computing terminal.

[0074] The M1 adopts: a series of Fourier fringe patterns are pre-generated by a computer and then loaded onto a digital micromirror array, with a resolution of 512×512. Finally, the fringe pattern is projected onto an object through a projection lens. In particular, the synchronization signal of the digital micromirror array is input into a single-photon counter as a reference for switching the next frame of fringe. At this time, the distance l' from the digital micromirror array to the projection lens, the focal length f of the projection lens, and the distance l from the object to the lens satisfy the formula:

[0075]

[0076] Therefore, when imaging at close range, it is necessary to estimate the approximate distance between the object and the system in advance and adjust the position of the projection lens. For imaging at long distance, the object is approximately at infinity, and l' is equal to the focal length f.

[0077] The M2 adopts: the diffuse reflected light of the object is homogenized and filtered through a diffuser and a 1550nm filter, and then the light is converged by a converging lens onto the optical fiber probe of the single-photon detector. The dead time and detection efficiency (PDE) of the single-photon detector need to be set according to the frequency of the pulsed laser to ensure that at most one photon event is recorded per pulse, otherwise it will cause the problem of recording multiple timestamps within a single pulse cycle, and the exact flight time cannot be obtained. Since the single-photon detector can detect one photon and has a high sensitivity, the present invention can be applied to situations where only very few photons are returned over a long distance.

[0078] The M3 adopts: the TCSPC single photon counter obtains the timestamp of the laser pulse, and then inputs it into the computing terminal for processing and correlation imaging. First, the tens of thousands of light pulse flight times collected are divided according to different projection fringe patterns, and then the histogram is counted to obtain the peak number of photons, which is the light intensity I reflected by the object:

[0079]

[0080] Among them, P i Represents the number of photons recorded at the ith flight time. The value range of k is (peak-k / 2,peak+k / 2], where peak is the histogram index value corresponding to the peak value.

[0081] The image of the object can be obtained by calculating the peak light intensity corresponding to different fringe patterns. x ,f y ) can be obtained by the following formula:

[0082] F(f x ,f y )=[I0(x,y)-Iπ (x,y)]+j·[I π / 2 (x,y)-I 3π / 2 (x,y)]

[0083] Where I0, I π / 2 , I π , I 3π / 2 It represents the four-step phase shift stripes loaded by the digital micromirror array. Finally, the image O of the object is obtained by inverse Fourier transform:

[0084] O=IFFT(F)

[0085] Here, IFFT represents two-dimensional inverse Fourier transform.

[0086] The single-photon three-dimensional imaging system and method based on Fourier correlation imaging adopted by the present invention applies four-step phase-shifted Fourier fringes to single-photon imaging for the first time, and effectively suppresses the noise in the detection light intensity by differential reconstruction. It solves the problem of traditional radar scanning detection and simplifies the complexity of the imaging system. The reflectivity information and depth information of the object can be obtained at the same time to obtain the depth image of the object. The detection sensitivity at the single-photon level makes the present invention have great advantages in long-distance imaging.

Claims

1. A single-photon three-dimensional imaging method and system based on Fourier correlation imaging, characterized in that: The following steps are involved: Step S1: a series of Fourier fringes are generated after the laser pulse is irradiated onto the digital micromirror array; Step S2: Project the stripes obtained in step 1 onto the object, and homogenize and filter the reflected light through a diffuser and a filter; Step S3: using a single photon detector and a counter to obtain a flight time histogram corresponding to each fringe; Step S4: Slice the multiple peaks of the histogram to obtain the number of photons as the light intensity value for correlation reconstruction to obtain images at different depths.

2. The single-photon three-dimensional imaging method and system based on Fourier correlation imaging according to claim 1, characterized in that: The step S1 adopts: the pulse light source is expanded through the optical fiber coupler and the collimating beam expander; the digital micromirror array encodes the light beam through the pre-loaded Fourier fringes to obtain the fringing pattern.

3. The single-photon three-dimensional imaging method and system based on Fourier correlation imaging according to claim 1, characterized in that: The step S2 adopts the following method: the reflected light from the stripe pattern irradiating the object is homogenized by a diffuser, and then a part of the background noise is removed by a filter.

4. The single-photon three-dimensional imaging method and system based on Fourier correlation imaging according to claim 1, characterized in that: The step S3 adopts: inputting the pulse output by the single-photon detector and the synchronization pulse of the digital micromirror array into the single-photon counter for synchronization, obtaining the timestamps of multiple pulses corresponding to each frame of the Fourier pattern; and then sorting them according to the number of arrival times to obtain a flight time histogram.

5. The single-photon three-dimensional imaging method and system based on Fourier correlation imaging according to claim 1, characterized in that: The step S4 adopts: obtaining the flight time corresponding to multiple peaks of the flight time histogram as the depth information of the object, and the depth can be calculated by the following formula: l=v*t / 2 Where l is the distance from the object surface to the detector, v is the speed of light, and t is the flight time; the light intensity I reflected by an object at depth l illuminated by a single fringe image is: Among them, P i represents the number of photons recorded at the ith flight time; the value range of k is (peak-k / 2, peak+k / 2], where peak is the histogram index value corresponding to the peak value; the Fourier spectrum of the object can be obtained by the following formula: F(f x ,f y )=[I0(x,y)-I π (x,y)]+j·[I π / 2 (x,y)-I 3π / 2 (x,y)] Where I0, I π / 2 , I π , I 3π / 2 Represents the four-step phase-shifted stripes loaded by the digital micromirror array, and finally the image O of the object is obtained by inverse Fourier transform: O=IFFT(F) Here, IFFT represents two-dimensional inverse Fourier transform.

6. A single-photon three-dimensional imaging system based on Fourier correlation imaging, characterized in that: It includes the following modules: M1: Fourier fringe projection module, which generates a series of fringe patterns of different frequencies by irradiating a pulsed laser onto a digital micromirror array, and then projects them onto an object using a projection lens; M2: Light receiving module. The object illuminated by a series of stripe light fields emits diffuse reflected light, which is homogenized by a diffuser and filtered out by a filter to remove noise light of other bands before being collected by a single-photon detector. M3: Post-processing module, consisting of TCSPC single photon counter and computing terminal.

7. The single-photon three-dimensional imaging system based on Fourier correlation imaging according to claim 6, characterized in that: The M1 adopts: inputting the synchronization signal of the digital micromirror array into the single photon counter as a reference for switching the next frame of stripes; in particular, at this time, the distance l' from the digital micromirror array to the projection lens, the focal length f of the projection lens, and the distance l from the object to the lens satisfy the formula: Therefore, when imaging at close range, it is necessary to estimate the approximate distance from the object to the system in advance and adjust the position of the projection lens; for imaging at long distance, the object is approximately at infinity, and l' is equal to the focal length f.

8. The single-photon three-dimensional imaging system based on Fourier correlation imaging according to claim 6, characterized in that The M2 adopts: the dead time and detection efficiency (PDE) of the single photon detector need to be set according to the frequency of the pulsed laser to ensure that at most one photon event is recorded per pulse.

9. The single-photon three-dimensional imaging system based on Fourier correlation imaging according to claim 6, characterized in that The M3 adopts: firstly, the collected tens of thousands of light pulse flight times are divided according to different projection fringe patterns, and then a histogram is performed to obtain the peak number of photons, which is the light intensity I reflected by the object: Among them, P i represents the number of photons recorded in the ith flight time; the value range of k is (peak-k / 2, peak+k / 2], where peak is the histogram index value corresponding to the peak value; the image of the object can be obtained by calculating the peak light intensity corresponding to different fringe patterns; the Fourier spectrum of the object F(f x ,f y ) can be obtained by the following formula: F(f x ,f y )=[I0(x,y)-I π (x,y)]+j·[I π / 2 (x,y)-I 3π / 2 (x,y)] Where I0, I π / 2 , I π , I 3π / 2 Represents the four-step phase-shifted stripes loaded by the digital micromirror array; finally, the image O of the object is obtained by inverse Fourier transform: O=IFFT(F) Among them, IFFT represents two-dimensional inverse Fourier transform; by selecting other peaks and recalculating the light intensity, the images of objects located at different depths can be associated and obtained; by combining the flight times of different peaks, a three-dimensional image of the object can be obtained.

Images