A non-line-of-sight imaging system and method based on spectral and spatial-temporal dual coding

Abstract

The present invention relates to the field of photoelectric detection technology, and discloses a non-line-of-sight imaging system and method based on spectral and temporal dual coding. A wide-spectrum pulse laser emits laser light, which is processed to output a discrete pulse sequence. The discrete pulse sequence is processed to obtain linearly polarized light in the reflection direction, and the linearly polarized light in the reflection direction is dispersed to output multiple light beams. The multiple light beams are diffusely reflected on the surface of a relay wall and irradiate the surface of a hidden object, and then reflected back to the surface of the relay wall. The non-line-of-sight triple echo light beams that are diffusely reflected again on the surface of the relay wall are collected by a detector through a spatial dispersion element. The required imaging information is then calculated by a processing unit. The present invention introduces spectral and temporal dual coding, and uses time coding and space coding to simultaneously encode the spatial position information of the irradiation point and the photon flight time information in the time domain of the non-line-of-sight triple echo, thereby realizing non-line-of-sight imaging. The system has the advantages of fast and scanning-free operation, high integration, and high reconstruction accuracy.

Description

Technical Field

[0001] The present invention relates to the field of photoelectric detection technology, and in particular to a non-line-of-sight imaging system and method based on spectral and temporal dual coding. Background Art

[0002] Due to the obstruction of streets and buildings, it is difficult to effectively observe some targets and activity venues using direct imaging methods. Therefore, there is an urgent need for methods that can bypass obstructing objects and track and observe targets behind them. Non-line-of-sight imaging technology can reconstruct the surface morphology of hidden objects through diffuse reflection of relay surfaces, and has important application prospects in many fields such as autonomous driving, disaster relief, and medical diagnosis. However, due to the data acquisition time of the existing technology using galvanometer scanning point by point, the current imaging speed is mostly limited to hundreds of seconds for a single frame image, resulting in slow imaging. Summary of the Invention

[0003] The purpose of the present invention is to overcome the problem of slow imaging of existing non-line-of-sight imaging technology, and to provide a non-line-of-sight imaging system and method based on spectral and spatiotemporal dual coding.

[0004] In order to achieve the above-mentioned object, the present invention provides a non-line-of-sight imaging system based on spectral and spatiotemporal dual coding, comprising: a wide-spectrum pulse laser, an optical multiplexer, an optical fiber delay array, an optical amplifier, a laser polarization beam splitter, a spatial dispersion element, a relay wall, a detector, and a processing unit;

[0005] The laser output by the wide spectrum pulse laser passes through the optical multiplexer, the optical fiber delay array, and the optical multiplexer outputs a discrete pulse sequence with different central wavelengths; wherein the time interval between adjacent discrete pulses in the discrete pulse sequence is greater than the dead time of the detector;

[0006] The discrete pulse sequence passes through the optical amplifier and the laser polarization beam splitter to obtain linearly polarized light in the reflection direction. The linearly polarized light in the reflection direction is dispersed by the spatial dispersion element, and multiple light beams corresponding to the discrete pulse sequence are output to illuminate the surface of the relay wall to form two-dimensional orthogonal discrete illumination; wherein the multiple light beams are output in a manner that satisfies constructive interference conditions;

[0007] The relay wall surface undergoes diffuse reflection based on the two-dimensional orthogonal discrete illumination. Part of the diffusely reflected light is illuminated by the surface of the hidden object when blocked by an obstacle, and is reflected again by the surface of the hidden object back to the relay wall surface. The non-viewing zone tertiary echo light beam after the diffuse reflection again on the relay wall surface passes through the spatial dispersion element and is collected by the detector.

[0008] The processing unit calculates and obtains the required imaging information according to the detection signal corresponding to the non-viewing zone triple echo light beam collected by the detector.

[0009] As an implementable embodiment, it also includes: a light stop, a multimode optical fiber, a first reflector, a second reflector, a third reflector and a lens; the discrete pulse sequence will also obtain linear polarized light in the transmission direction after passing through the laser polarization beam splitter, and the linear polarized light in the transmission direction is collected by the light stop; the laser output by the wide-spectrum pulse laser is emitted to the optical multiplexer through the multimode optical fiber; the discrete pulse sequence is incident on the laser polarization beam splitter after changing the optical path through the first reflector and the second reflector, and the linear polarized light in the reflection direction is incident on the spatial dispersion element after changing the optical path through the third reflector for dispersion, and the non-field of view tertiary echo light beam after being diffusely reflected again on the surface of the relay wall returns to the spatial dispersion element in sequence and is then focused by the lens and collected by the detector.

[0010] As an implementation method, the optical amplifier is a fiber amplifier, the optical multiplexer is an arrayed waveguide grating, and the detector is a single-photon avalanche diode detector; the fiber delay array is a single-mode optical fiber with different lengths, wherein the arrayed waveguide grating decomposes the laser into discrete spectral channels corresponding to the number of single-mode optical fibers.

[0011] As an implementation method, the process in which the broad spectrum pulse laser outputs laser light and sequentially passes through the optical multiplexer, the optical fiber delay array, and the optical multiplexer outputs a discrete pulse sequence with different central wavelengths is specifically as follows:

[0012] The wide pulse laser emits laser light to the arrayed waveguide grating; the arrayed waveguide grating decomposes the laser light into multiple discrete spectral channels, and the laser light of the multiple discrete spectral channels is respectively delayed and transmitted to the arrayed waveguide grating through corresponding single-mode optical fibers with different lengths; the arrayed waveguide grating multiplexes the laser light of the multiple discrete spectral channels into discrete pulse sequences with different central wavelengths and time intervals between adjacent discrete pulses.

[0013] As an implementation method, the spatial dispersion element includes a cylindrical lens, a virtual image phase array, and a diffraction grating;

[0014] The cylindrical lens focuses the linearly polarized light in the reflection direction onto the virtual image phase array, performs spatial dispersion in the incident plane, and forms multiple light beams corresponding to discrete pulse sequences. The multiple light beams interfere with each other so as to be output in a manner that satisfies the constructive interference condition. The multiple light beams that satisfy the constructive interference condition are separated by intervals within the free spectral range and have the same output angle in the y-axis direction. The diffraction grating diffracts and expands the multiple light beams with the same output angle in the x-axis direction, and illuminates the surface of the relay wall to form two-dimensional orthogonal discrete illumination.

[0015] As an implementable embodiment, the broadband pulse laser is a broadband pulse laser with picosecond resolution, and the processing unit includes a counting module, which analyzes and processes the detection signal collected by the detector and the picosecond resolution synchronization signal of the broadband pulse laser to obtain the picosecond resolution photon number-flight time data corresponding to the discrete pulse sequence; wherein the flight time is the round-trip flight time of the photon between the relay wall and the surface of the hidden object; the gating window of the detector and the picosecond resolution synchronization signal of the broadband pulse laser are precisely delayed by an adjustable picosecond delay device to match the time range of the non-field-of-sight triple echo beam that needs to be collected.

[0016] As an implementation method, the processing unit also includes a reconstruction module, which reconstructs and restores the reflectivity information of the surface of the hidden object based on the Wiener filter deconvolution non-line-of-sight reconstruction algorithm of the cone transformation and the obtained photon number-flight time data and other related parameters, thereby realizing the non-line-of-sight reconstruction of the hidden object.

[0017] As an implementation method, the interval of the free spectral range is:

[0018]

[0019] Where c is the speed of light, T is the cavity thickness of the virtual image phase array, n is the cavity refractive index of the virtual image phase array, θ i is the cavity tilt angle of the virtual image phase array, θ in is the incident angle of the light beam in the virtual image phase array, θ y, is the exit angle of the light beam in the virtual image phase array cavity.

[0020] As an implementation method, the process of reconstructing and recovering the reflectivity information of the hidden object surface based on the Wiener filter deconvolution non-viewing area reconstruction algorithm with cone transformation and the obtained photon number-time of flight data and other related parameters specifically includes:

[0021] A standard forward model is established under confocal conditions; wherein the formula of the standard forward model is:

[0022]

[0023] Where τ represents the photon count-time data corresponding to each scanning point on the repeater surface, (x′, y′) is the position of the scanning point on the relay wall surface, (x, y, z) represents the spatial position of the hidden object surface, r is the distance between the spatial position of the hidden object surface and the corresponding scanning point on the relay wall surface, ρ represents the reflectivity of the hidden object surface, δ represents the Dirac function, Ω represents the three-dimensional space where the hidden object is located, t represents the flight time, and c represents the speed of light.

[0024] The standard forward model is converted into a three-dimensional convolution form by using a cone transformation to obtain a forward model in a three-dimensional convolution form. The cone transformation formula is:

[0025] v=(tc / 2) 2 , where represents z in the new coordinate system, and then v represents (tc / 2) in the new coordinate system 2 ;

[0026] The forward model formula of the three-dimensional convolution form is:

[0027] R t {τ}(x′,y′,v)=∫∫∫ Ω R z {ρ}(x,y,u)h(x′-x,y′-y,vu)dxdydu; where, R t {τ} represents the photon number-flight time data in the form of a three-dimensional convolution, R z {ρ} represents the reflectivity of the hidden object surface in the form of 3D convolution;

[0028] get:

[0029] Based on the Wiener filter three-dimensional deconvolution, a non-viewing reconstruction formula is established, and R t {τ}, R z {ρ} and other related parameters are brought into the non-viewing area reconstruction formula to reconstruct and restore the reflectivity information of the hidden object surface; wherein the non-viewing area reconstruction formula is:

[0030] Where F represents the three-dimensional Fourier transform, ρ represents the surface reflectivity information of the hidden object, represents the focus matrix containing the Fourier coefficients of the 3D convolution kernel, and α represents the signal-to-noise ratio.

[0031] Accordingly, the present invention also provides a non-line-of-sight imaging method based on spectral and spatiotemporal dual coding, comprising the following steps:

[0032] Outputting laser light and processing the laser light to obtain a discrete pulse sequence with different central wavelengths; wherein the time interval between adjacent discrete pulses in the discrete pulse sequence is greater than the dead time of a detector used for subsequent detection;

[0033] Amplifying and splitting the discrete pulse sequence to obtain linearly polarized light in the reflection direction, dispersing the linearly polarized light in the reflection direction, and outputting multiple light beams corresponding to the discrete pulse sequence, which are irradiated on the surface of the relay wall to form two-dimensional orthogonal discrete illumination; wherein the multiple light beams are output in a manner that satisfies constructive interference conditions;

[0034] The relay wall surface undergoes diffuse reflection based on the two-dimensional orthogonal discrete illumination. When blocked by an obstacle, part of the diffusely reflected light is irradiated on the surface of the hidden object, and is reflected again on the surface of the hidden object back to the relay wall surface. After diffuse reflection again on the relay wall surface, the non-viewing zone tertiary echo light beams are collected by the detector after being focused.

[0035] The required imaging information is calculated based on the detection signal corresponding to the non-viewing zone triple echo light beam collected by the detector.

[0036] Beneficial effects of the present invention: The present invention discloses a non-line-of-sight imaging system and method based on spectral-temporal dual coding. A wide-spectrum pulse laser emits laser light, which is processed to output a discrete pulse sequence. The discrete pulse sequence is processed to obtain linearly polarized light in the reflection direction, which is dispersed to output multiple light beams. The multiple light beams are diffusely reflected on the surface of a relay wall and illuminate the surface of a hidden object. They are then reflected back to the surface of the relay wall. The non-line-of-sight triple echo light beams that are diffusely reflected again on the surface of the relay wall are collected by a detector through a spatial dispersion element. The processing unit calculates the required imaging information based on the detection signal collected by the detector. The present invention introduces spectral-temporal dual coding into non-line-of-sight imaging based on the dispersion characteristics of the wide-spectrum laser, utilizes wavelength division multiplexing and discrete time delay methods to time-code the wide-spectrum pulse laser, and performs illumination spatial coding and regulation based on a spectral-spatial dispersion model. The spatial position information of the illumination point and the photon flight time information are simultaneously encoded in the time domain of the non-line-of-sight triple echoes, thereby realizing non-scanning snapshot non-line-of-sight imaging under detection by a single single-photon avalanche diode detector. Compared with imaging methods based on galvanometer spatial scanning and single-photon avalanche diode detector arrays, it has the advantages of fast and scanning-free, high integration and high reconstruction accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 Schematic diagram of the non-line-of-sight imaging system based on spectral and spatiotemporal dual coding. DETAILED DESCRIPTION

[0038] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0039] This embodiment provides a technical solution: a non-line-of-sight imaging system based on spectral and spatiotemporal dual coding, comprising: a broadband pulse laser, an optical multiplexer, a fiber delay array, an optical amplifier, a laser polarization beam splitter, a spatial dispersion element, a relay wall, a detector, and a processing unit;

[0040] The laser output by the wide spectrum pulse laser passes through the optical multiplexer, the optical fiber delay array, and the optical multiplexer outputs a discrete pulse sequence with different central wavelengths; wherein the time interval between adjacent discrete pulses in the discrete pulse sequence is greater than the dead time of the detector;

[0041] The discrete pulse sequence passes through the optical amplifier and the laser polarization beam splitter to obtain linearly polarized light in the reflection direction. The linearly polarized light in the reflection direction is dispersed by the spatial dispersion element, and multiple light beams corresponding to the discrete pulse sequence are output to illuminate the surface of the relay wall to form two-dimensional orthogonal discrete illumination; wherein the multiple light beams are output in a manner that satisfies constructive interference conditions;

[0042] The relay wall surface undergoes diffuse reflection based on the two-dimensional orthogonal discrete illumination. Part of the diffusely reflected light is illuminated by the surface of the hidden object when blocked by an obstacle, and is reflected again by the surface of the hidden object back to the relay wall surface. The non-viewing zone tertiary echo light beam after the diffuse reflection again on the relay wall surface passes through the spatial dispersion element and is collected by the detector.

[0043] The processing unit calculates and obtains the required imaging information according to the detection signal corresponding to the non-viewing zone triple echo light beam collected by the detector.

[0044] The optical amplifier may be a fiber amplifier, specifically an erbium-doped fiber amplifier; the optical multiplexer may be an arrayed waveguide grating (AWG); the detector may be a single-photon avalanche diode (SPAD) detector or other fast photoelectric response detector; the fiber delay array may be a single-mode optical fiber with different lengths, wherein the arrayed waveguide grating decomposes the laser into discrete spectral channels corresponding to the number of the single-mode optical fibers, such as Figure 1 As shown, the optical fiber delay array 4 includes 4 single-mode optical fibers, but this number does not represent the number of single-mode optical fibers actually used during reconstruction. The number of single-mode optical fibers actually used needs to be determined according to the hidden objects that need to be reconstructed.

[0045] The process in which the broad spectrum pulse laser outputs laser light and sequentially passes through the optical multiplexer, the optical fiber delay array, and the optical multiplexer outputs a discrete pulse sequence with different central wavelengths is specifically as follows:

[0046] The wide pulse laser emits laser light to the arrayed waveguide grating; the arrayed waveguide grating decomposes the laser light into multiple discrete spectral channels, and the laser light of the multiple discrete spectral channels is respectively delayed and transmitted to the arrayed waveguide grating through corresponding single-mode optical fibers with different lengths; the arrayed waveguide grating multiplexes the laser light of the multiple discrete spectral channels into discrete pulse sequences with different central wavelengths and time intervals between adjacent discrete pulses.

[0047] That is to say, after the laser is decomposed by the arrayed waveguide grating, the length of the single-mode optical fiber of each discrete spectral channel can be adjusted so that the pulse laser of each discrete spectral channel is an equally spaced sequence in time, and the time interval between discrete pulses of adjacent wavelengths is greater than the dead path time of the single-photon avalanche diode, so as to avoid the influence of the dead path time of the single-photon avalanche diode in the subsequent detection process.

[0048] In this embodiment, the spatial dispersion element includes a cylindrical lens, a virtual image phase array, and a diffraction grating. However, in other embodiments, it may also be other components, which is not limited in this embodiment.

[0049] Specifically, the cylindrical lens focuses the linearly polarized light in the reflection direction into the virtual image phase array, performs spatial dispersion in the incident plane, and forms multiple light beams corresponding to discrete pulse sequences. The multiple light beams interfere with each other so as to be output in a manner that satisfies the constructive interference condition. The multiple light beams that satisfy the constructive interference condition are separated by intervals within the free spectral range and have the same output angle in the y-axis direction. The diffraction grating diffracts and expands the multiple light beams with the same output angle in the x-axis direction, and illuminates the surface of the relay wall to form two-dimensional orthogonal discrete illumination.

[0050] The interval of the free spectral range satisfies the following formula:

[0051] That is, interval

[0052] Where c is the speed of light, T is the cavity thickness of the virtual image phase array, n is the cavity refractive index of the virtual image phase array, θ i is the cavity tilt angle of the virtual image phase array, θ in is the incident angle of the light beam in the virtual image phase array, θ y, is the exit angle of the light beam in the virtual image phase array cavity.

[0053] It can be seen that the dispersion characteristics in the y-axis direction depend on the cavity thickness, cavity inclination angle, surface reflectivity, incident light properties, etc. of the virtual image phase array. Therefore, in order to make the light beam output in a manner that satisfies the constructive interference condition, before using the spatial dispersion element composed of cylindrical lens-virtual image phase array-diffraction grating for two-dimensional spatial expansion, the structural type, structural size, structural material properties, etc. of the virtual image phase array can be determined according to the spatial dispersion characteristics of the virtual image phase array to optimize the spatial accuracy of the two-dimensional orthogonal discrete illumination light field.

[0054] As a result, in this embodiment, the dispersion intensity distribution of the virtual image phase array conforms to the Airy-Lorentz function distribution under the Gaussian envelope. By regulating the size of the free spectral range, the phenomenon of spectral spatial aliasing is effectively avoided; further combined with the diffraction effect of the diffraction grating in the x-direction, a two-dimensional orthogonal dispersion distribution is formed.

[0055] This embodiment provides a non-line-of-sight imaging system based on spectral and temporal dual coding, wherein the dual coding refers to temporal coding and spatial coding respectively;

[0056] The time encoding process is as follows: the laser light emitted by a high-time-resolution, wide-spectrum pulsed laser is collimated and incident on an arrayed waveguide grating (AWG) based on a lightwave circuit. The AWG decomposes the wide-spectrum pulsed laser light into discrete spectral channels with fixed wavelength intervals (e.g., approximately 0.4 nm) and covering a certain wavelength band (e.g., one band is 1530 nm-1570 nm). The number of discrete spectral channels can be 100 or more, and the actual number can be determined based on the hidden object to be reconstructed. The laser light from different discrete spectral channels is precisely time-delayed through single-mode optical fibers of different lengths and demultiplexed into a series of discrete pulse sequences with different central wavelengths using the AWG. To avoid the influence of detector dead time during subsequent detection, the time interval between discrete pulses of adjacent wavelengths should be greater than the detector dead time.

[0057] The spatial encoding process is as follows: the discrete pulse sequence is first amplified by a fiber amplifier to provide high instantaneous intensity for discrete illumination points; the discrete pulse sequence is then expanded by a spatial dispersion element composed of a cylindrical lens, a virtual image phase array, and a diffraction grating. The cylindrical lens focuses the discrete pulse sequence into the virtual image phase array cavity and performs spatial dispersion in the incident plane of the virtual image phase array cavity; due to the multi-beam interference corresponding to the discrete pulse sequence, the transmission spectrum of the virtual image phase array shows multiple resonance peaks, which are separated by intervals in the free spectral range and have the same output angle along the y-axis direction; the diffraction grating diffracts and expands the transmitted light beams with the same output angle in the x-axis direction, thereby realizing two-dimensional orthogonal discrete illumination of the relay wall surface.

[0058] Since the different central wavelengths of the discrete pulse sequence formed by time encoding correspond to different positions on the surface of the hidden object in two-dimensional orthogonal discrete illumination, combined with the mapping relationship between wavelength information and discrete pulse timing, the spatial position information of the illumination point on the relay wall surface and the corresponding photon flight time information are ultimately encoded simultaneously in the time domain of the non-line-of-sight triple echo, achieving reconstruction.

[0059] Among them, the non-line-of-sight triple echo beam refers to the laser emitted by the wide-spectrum pulse laser, which passes through the optical multiplexer, optical fiber delay array, optical amplifier, laser polarization beam splitter, and spatial dispersion element, and is diffusely reflected on the surface of the relay wall before being transmitted to the surface of the hidden object in the non-line-of-sight range, and then reflected again on the surface of the hidden object and returned to the surface of the relay wall, and then diffusely reflected again on the surface of the relay wall before returning to the echo beam of the spatial dispersion element.

[0060] In this embodiment, the non-line-of-sight imaging system based on spectral and spatiotemporal dual coding further includes: a light stop; the discrete pulse sequence will also obtain linear polarized light in the transmission direction after passing through the laser polarization beam splitter, and the linear polarized light in the transmission direction is collected by the light stop.

[0061] In this embodiment, the non-line-of-sight imaging system based on spectral and temporal dual coding also includes: a multimode optical fiber, a first reflector, a second reflector, a third reflector and a lens; the laser output by the wide-spectrum pulse laser is emitted to the optical multiplexer through the multimode optical fiber; the discrete pulse sequence is incident on the laser polarization beam splitter after changing the optical path through the first reflector and the second reflector, and the linear polarized light in the reflection direction is incident on the spatial dispersion element for dispersion after changing the optical path through the second reflector. After being diffusely reflected again on the surface of the relay wall, the non-line-of-sight tertiary echo light beam returns to the spatial dispersion element in sequence and is then focused by the lens and collected by the detector.

[0062] The processing unit calculates the required imaging information based on the detection signal corresponding to the non-field of view triple echo light beam collected by the detector, and obtains the imaging information as follows: the number of photons obtained by processing the detection signal - the flight time data; wherein the flight time is the round-trip flight time of the photons between the relay wall and the surface of the hidden object.

[0063] Furthermore, existing non-line-of-sight imaging technology is also limited by the time accuracy of the photoelectric detection element and the reconstruction accuracy of the non-line-of-sight algorithm. The currently achievable spatial resolution of the reconstructed image is mostly limited to about 3-5 cm. Therefore, in order to solve the above problems, the broadband pulse laser in this embodiment is a broadband pulse laser with picosecond resolution, and the processing unit includes a counting module. The counting module analyzes and processes the detection signal collected by the detector and the picosecond resolution synchronization signal of the broadband pulse laser to obtain the picosecond resolution photon number-flight time data corresponding to the discrete pulse sequence; wherein the flight time is the round-trip flight time of the photon between the relay wall and the surface of the hidden object; the gated window of the detector and the picosecond resolution synchronization signal of the broadband pulse laser are precisely delayed by an adjustable picosecond delay device to match the time range of the non-line-of-sight triple echo beam to be collected;

[0064] It should be noted that the technology for obtaining the photon number-flight time data based on the detection signal of the corresponding non-line-of-sight triple echo light beam in this embodiment is an existing technology and will not be described in detail in this embodiment.

[0065] Specifically, in this embodiment, the picosecond-resolution synchronization signal of the wide-spectrum pulse laser can be precisely delayed by an adjustable picosecond delay device to ensure that the operating time range of the time-to-amplitude converter of the counting module matches the time range of the detection signal to be collected; at the same time, in order to avoid the influence of the dead-path time and after-pulse effect caused by the direct echo from the surface of the non-line-of-sight imaging relay wall, an ultrafast gated detector (usually above 10 MHz) can be used for high-frequency gated detection, and the gating window can also be precisely delayed by an adjustable picosecond delay device so that it is located in the non-line-of-sight triple echo beam interval; thereby avoiding the erroneous collection of the direct reflected light directly reflected from the surface of the relay wall and no longer passing through the hidden object, i.e., the primary reflected light, and the secondary reflected light reflected from the surface of the hidden object and no longer passing through the relay wall;

[0066] The primary reflected light refers to the laser light emitted by the wide-spectrum pulse laser, which passes through the optical multiplexer, optical fiber delay array, optical amplifier, laser polarization beam splitter, and spatial dispersion element, and then is reflected on the surface of the relay wall. The reflected light then passes directly through the spatial dispersion element without being reflected on the surface of the hidden object and is then collected by the detector. The secondary reflected light refers to the laser light emitted by the wide-spectrum pulse laser, which passes through the optical multiplexer, optical fiber delay array, optical amplifier, laser polarization beam splitter, and spatial dispersion element, and then is reflected on the surface of the relay wall and irradiated on the surface of the hidden object. The reflected light then passes directly through the spatial dispersion element without being reflected on the relay wall and is then collected by the detector.

[0067] After the non-line-of-sight triple echoes are collected by the detector, the counting module simultaneously obtains the picosecond-resolution synchronization signal of the wide-spectrum pulse laser and the detection signal of the single-photon detector, and processes the data to obtain a photon number distribution curve with picosecond resolution, that is, an extremely short time interval. The photon number distribution curve includes photon number information, wavelength information that can be corresponding to spatial position, and photon flight time information. Finally, the detection signal is processed and combined with the non-line-of-sight reconstruction algorithm to reconstruct and restore the surface reflectivity information of the hidden object.

[0068] It should be noted that the resolution of the detector itself is related to its own bandwidth and generally has a resolution of only 50-100 nanoseconds. If the picosecond resolution synchronization signal of the wide-spectrum pulse laser is not obtained, a very high resolution cannot be obtained. Therefore, this embodiment obtains the picosecond resolution synchronization signal of the wide-spectrum pulse laser for processing to obtain a picosecond resolution photon number distribution curve; so that the final reconstructed image has high accuracy and high resolution.

[0069] The embodiment of the present invention is based on two-dimensional spatial dispersion coding, which corresponds wavelength information to the spatial position information of the irradiation point, and can realize scanless snapshot non-line-of-sight imaging, with an imaging speed much faster than traditional galvanometer scanning; through wavelength division decomposition, wavelength division multiplexing and discrete delay time coding, the photon flight time information corresponding to different wavelengths is encoded in the same laser pulse time sequence, and the detection process can be realized using a single single-photon detector, avoiding the use of expensive single-photon detector arrays, with higher integration and lower cost; and it combines spectral spatiotemporal coding with photon flight time imaging to realize snapshot non-line-of-sight imaging.

[0070] Specifically, such as Figure 1 As shown, the non-line-of-sight imaging system in this embodiment includes a broadband pulse laser 1, a multimode optical fiber 2, an arrayed waveguide grating 3, a fiber delay array 4, an erbium-doped fiber amplifier 5, a first reflector 6, a second reflector 7, a laser polarization beam splitter 8, a light stop 9, a third reflector 10, a cylindrical lens 11, a virtual image phase array 12, a diffraction grating 13, a relay wall 14, a blocking wall 15, a hidden object 16, a lens 17, a single-photon avalanche diode detector 18, and a processing unit 19;

[0071] The laser light emitted by the broadband pulse laser 1 is collimated and transmitted to the arrayed waveguide grating 3 via the multimode optical fiber 2. The arrayed waveguide grating 3 decomposes the laser light into discrete spectral channels with fixed wavelength intervals and covering a certain wavelength band. The laser light of different discrete spectral channels is precisely delayed by the optical fiber delay array 4 and then demultiplexed into a series of discrete pulse sequences with different central wavelengths by the arrayed waveguide grating 3.

[0072] After being amplified by the erbium-doped fiber amplifier 5, the modulated discrete pulse sequence is transmitted to the laser polarization beam splitter 8 through the first reflector 6 and the second reflector 7, and is divided by the laser polarization beam splitter 8 into linear polarization in the transmission direction and linear polarization in the reflection direction; the linear polarization in the transmission direction is collected by the light stop 9, and the linear polarization in the reflection direction is reflected by the third reflector 10, and then passes through the spatial dispersion element composed of the cylindrical lens 11-virtual image phase array 12-diffraction grating 13 in sequence. The cylindrical lens 11 focuses the pulsed laser into the cavity of the virtual image phase array 12 and performs spatial dispersion in the incident plane. Due to multi-beam interference, the transmission spectrum of the virtual image phase array 12 shows multiple resonance peaks. These resonance peaks are separated by intervals in the free spectral range and have the same output angle along the y-axis direction. The diffraction grating 13 diffracts and expands the transmitted light beams with the same output angle in the x-axis direction, forming two-dimensional orthogonal discrete illumination on the surface of the relay wall 14.

[0073] After diffuse reflection on the surface of the relay wall 14, the two-dimensional orthogonal discrete illumination beam is transmitted to the surface of the hidden object 16 blocked by the blocking wall 15. After being reflected again on the surface of the hidden object 16, it returns to the surface of the relay wall 14. After being diffusely reflected again on the surface of the relay wall 14, the non-line-of-sight tertiary echo beam passes through the spatial dispersion element composed of the diffraction grating 13, the virtual image phase array 12, and the cylindrical lens 11 again. The echo photons pass through the laser polarization beam splitter 8 and are focused by the lens 17 before being collected by the single-photon avalanche diode detector 18 and the processing unit 19. The echo photons pass through the laser polarization beam splitter 8 only to make the system more compact. In other embodiments, the beam splitter may not be used. Thus, a photon number distribution curve of an extremely short time interval is obtained. Finally, the detection signal is data processed and, combined with the non-line-of-sight reconstruction algorithm, the surface reflectivity information of the hidden object is reconstructed and restored.

[0074] This embodiment uses fast photoelectric response instruments such as single-photon avalanche diode detectors to record spectral characteristics and spatial dispersion light intensity distribution in real time, determines the influence of virtual image phase array cavity thickness, cavity inclination, surface reflectivity, etc. on spatial dispersion light intensity distribution, combines theoretical calculations, software simulation and experimental data, establishes a precise mapping relationship between spatial illumination point distribution and laser pulse timing, and establishes a spectral space-time dual coding model; based on the spectral space coding model, determines the specific structural parameters of the two-dimensional spatial dispersion element of diffraction grating-virtual image phase array-cylindrical lens, and realizes a two-dimensional orthogonal discrete spatial illumination light field.

[0075] This embodiment is based on a spectral and spatiotemporal dual coding model to achieve spatiotemporal decoding of the precise two-dimensional spatial position of the irradiation point, and decodes the detected laser pulse timing signal into time-of-flight signals corresponding to different spatial position points.

[0076] Furthermore, the processing unit also includes a reconstruction module, which reconstructs and restores the reflectivity information of the surface of the hidden object based on the Wiener filter deconvolution non-line-of-sight reconstruction algorithm of the cone transformation and the obtained photon number-flight time data and other related parameters, thereby realizing the non-line-of-sight reconstruction of the hidden object.

[0077] The reconstruction algorithm of this embodiment is performed based on an orthogonal grid, so a two-dimensional spatial orthogonal scanning illumination of an orthogonal grid structure is formed in this embodiment, which facilitates subsequent algorithm reconstruction.

[0078] The process of reconstructing and recovering the reflectivity information of the surface of the hidden object based on the Wiener filter deconvolution non-viewing area reconstruction algorithm and the obtained photon number-time of flight data and other related parameters specifically includes:

[0079] A standard forward model is established under confocal conditions; wherein the formula of the standard forward model is:

[0080]

[0081] Where τ represents the photon count corresponding to each scanning point on the repeater surface—the time-of-flight data; (x′, y′) represents the position of the scanning point on the relay wall surface; (x, y, z) represents the spatial position of the hidden object surface; r represents the distance between the spatial position of the hidden object surface and the corresponding scanning point on the relay wall surface; ρ represents the reflectivity of the hidden object surface; δ represents the Dirac function; Ω represents the three-dimensional space where the hidden object is located; t represents the time of flight; and c represents the speed of light. These data can be directly obtained through measurement or other methods.

[0082] The standard forward model is converted into a three-dimensional convolution form by using a cone transformation to obtain a forward model in a three-dimensional convolution form. The cone transformation formula is:

[0083] v=(tc / 2) 2 , where represents z in the new coordinate system, and then v represents (tc / 2) in the new coordinate system 2 ;

[0084] The forward model formula of the three-dimensional convolution form is:

[0085] R t {τ}(x′,y′,v)=∫∫∫ Ω R z {ρ}(x,y,u)h(x′-x,y′-y,vu)dxdydu; where, R t {τ} represents the photon number-flight time data in the form of a three-dimensional convolution, R z{ρ} represents the reflectivity of the hidden object surface in the form of 3D convolution;

[0086] Calculation yields: h(x′-x,y′-y,vu)=δ((x′-x) 2 +(y′-y) 2 +uv);

[0087] Based on the Wiener filter three-dimensional deconvolution, a non-viewing reconstruction formula is established, and R t {τ}, R z {ρ} and other related parameters are brought into the non-viewing reconstruction formula to reconstruct and restore the reflectivity information of the hidden object surface, thereby realizing the non-viewing reconstruction of the hidden object. The non-viewing reconstruction formula is:

[0088] Where F represents the three-dimensional Fourier transform, ρ represents the surface reflectivity information of the hidden object, represents the focus matrix containing the Fourier coefficients of the three-dimensional convolution kernel, and α represents the signal-to-noise ratio. These data can be obtained directly.

[0089] The present invention discloses a non-line-of-sight imaging system based on spectral and temporal dual coding. Spectral and temporal dual coding is introduced into non-line-of-sight imaging based on the dispersion characteristics of broadband lasers. The broadband pulse laser is time-coded using wavelength division multiplexing and discrete time delay methods. The illumination space is coded and regulated based on a spectral spatial dispersion model. The spatial position information of the irradiation point and the photon flight time information are simultaneously encoded in the time domain of the non-line-of-sight triple echo, thereby realizing scanless snapshot non-line-of-sight imaging under detection by a single single-photon avalanche diode detector. Compared with imaging methods based on galvanometer spatial scanning and single-photon avalanche diode detector arrays, the system has the advantages of fast and scanless operation, high integration, and high reconstruction accuracy.

[0090] Based on the same inventive concept, the present invention also provides a non-line-of-sight imaging method based on spectral and spatiotemporal dual coding, comprising the following steps:

[0091] Outputting laser light and processing the laser light to obtain a discrete pulse sequence with different central wavelengths; wherein the time interval between adjacent discrete pulses in the discrete pulse sequence is greater than the dead time of a detector used for subsequent detection;

[0092] Amplifying and splitting the discrete pulse sequence to obtain linearly polarized light in the reflection direction, dispersing the linearly polarized light in the reflection direction, and outputting multiple light beams corresponding to the discrete pulse sequence, which are irradiated on the surface of the relay wall to form two-dimensional orthogonal discrete illumination; wherein the multiple light beams are output in a manner that satisfies constructive interference conditions;

[0093] The relay wall surface undergoes diffuse reflection based on the two-dimensional orthogonal discrete illumination. When blocked by an obstacle, part of the diffusely reflected light is irradiated on the surface of the hidden object, and is reflected again on the surface of the hidden object back to the relay wall surface. After diffuse reflection again on the relay wall surface, the non-viewing zone tertiary echo light beams are collected by the detector after being focused.

[0094] The required imaging information is calculated based on the detection signal corresponding to the non-viewing zone triple echo light beam collected by the detector.

[0095] Although the present invention has been disclosed above in terms of preferred embodiments, this is not intended to limit the present invention. Any person skilled in the art may make possible changes and modifications to the technical solutions of the present invention by utilizing the methods and technical contents disclosed above without departing from the spirit and scope of the present invention. Therefore, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solutions of the present invention shall fall within the scope of protection of the technical solutions of the present invention.

Claims

1. A non-line-of-sight imaging system based on spectral and spatiotemporal dual coding, characterized in that: include: Broad-spectrum pulse lasers, optical amplifiers, laser polarization beam splitters, spatial dispersion elements, relay walls, detectors, processing units, optical multiplexers, and fiber delay arrays; The laser output from the wide-spectrum pulse laser passes through the first part of the optical multiplexer, the optical fiber delay array, and the second part of the optical multiplexer in sequence to output a discrete pulse sequence with different central wavelengths; wherein the time interval between adjacent discrete pulses in the discrete pulse sequence is greater than the dead time of the detector; The discrete pulse sequence passes through the optical amplifier and the laser polarization beam splitter to obtain linearly polarized light in the reflection direction. The linearly polarized light in the reflection direction is dispersed by the spatial dispersion element, and multiple light beams corresponding to the discrete pulse sequence are output to illuminate the surface of the relay wall to form two-dimensional orthogonal discrete illumination; wherein the multiple light beams are output in a manner that satisfies constructive interference conditions; The relay wall surface undergoes diffuse reflection based on the two-dimensional orthogonal discrete illumination. Part of the diffusely reflected light is illuminated by the surface of the hidden object when blocked by an obstacle, and is reflected again by the surface of the hidden object back to the relay wall surface. The non-viewing zone tertiary echo light beam after the diffuse reflection again on the relay wall surface passes through the spatial dispersion element and is collected by the detector. The processing unit calculates and obtains the required imaging information according to the detection signal corresponding to the non-viewing zone triple echo light beam collected by the detector.

2. The non-line-of-sight imaging system based on spectral-temporal dual coding according to claim 1, characterized in that: Also includes: A light stopper, a multimode optical fiber, a first reflector, a second reflector, a third reflector and a lens; the discrete pulse sequence will also obtain linear polarization in the transmission direction after passing through the laser polarization beam splitter, and the linear polarization in the transmission direction is collected by the light stopper; the laser output by the wide-spectrum pulse laser is emitted to the optical multiplexer through the multimode optical fiber; the discrete pulse sequence is incident on the laser polarization beam splitter after changing the optical path through the first reflector and the second reflector, and the linear polarization in the reflection direction is incident on the spatial dispersion element after changing the optical path through the third reflector for dispersion, and the non-field of view tertiary echo light beams after being diffusely reflected again on the surface of the relay wall return to the spatial dispersion element in sequence and are then focused by the lens and collected by the detector.

3. The non-line-of-sight imaging system based on spectral-temporal dual coding according to claim 1, characterized in that: The optical amplifier is a fiber amplifier, the optical multiplexer is an arrayed waveguide grating, and the detector is a single-photon avalanche diode detector; the fiber delay array is a single-mode optical fiber with different lengths, wherein the arrayed waveguide grating decomposes the laser to obtain a number of discrete spectral channels corresponding to the number of the single-mode optical fibers.

4. The non-line-of-sight imaging system based on spectral-temporal dual coding according to claim 3, characterized in that: The process in which the broad spectrum pulse laser outputs laser light and sequentially passes through the optical multiplexer, the optical fiber delay array, and the optical multiplexer outputs a discrete pulse sequence with different central wavelengths is specifically as follows: The wide-spectrum pulse laser emits laser light to the arrayed waveguide grating; the arrayed waveguide grating decomposes the laser light into multiple discrete spectral channels, and the laser light of the multiple discrete spectral channels is respectively delayed and transmitted to the arrayed waveguide grating through corresponding single-mode optical fibers with different lengths; the arrayed waveguide grating multiplexes the laser light of the multiple discrete spectral channels into discrete pulse sequences with different central wavelengths and time intervals between adjacent discrete pulses.

5. The non-line-of-sight imaging system based on spectral-temporal dual coding according to claim 1, characterized in that: The spatial dispersion element includes a cylindrical lens, a virtual image phase array and a diffraction grating; The cylindrical lens focuses the linearly polarized light in the reflection direction onto the virtual image phase array, performs spatial dispersion in the incident plane, and forms multiple light beams corresponding to discrete pulse sequences. The multiple light beams interfere with each other so as to be output in a manner that satisfies the constructive interference condition. The multiple light beams that satisfy the constructive interference condition are separated by intervals within the free spectral range and have the same output angle in the y-axis direction. The diffraction grating diffracts and expands the multiple light beams with the same output angle in the x-axis direction, and illuminates the surface of the relay wall to form two-dimensional orthogonal discrete illumination.

6. The non-line-of-sight imaging system based on spectral-temporal dual coding according to claim 1, characterized in that: The broadband pulse laser is a broadband pulse laser with picosecond resolution. The processing unit includes a counting module, which analyzes and processes the detection signal collected by the detector and the picosecond resolution synchronization signal of the broadband pulse laser to obtain picosecond resolution photon number-flight time data corresponding to the discrete pulse sequence; wherein the flight time is the round-trip flight time of the photon between the relay wall and the surface of the hidden object; the gating window of the detector and the picosecond resolution synchronization signal of the broadband pulse laser are precisely delayed by an adjustable picosecond delay device to match the time range of the non-field of view triple echo beam to be collected.

7. The non-line-of-sight imaging system based on spectral-temporal dual coding according to claim 6, characterized in that: The processing unit also includes a reconstruction module, which reconstructs and restores the reflectivity information of the surface of the hidden object based on the Wiener filter deconvolution non-line-of-sight reconstruction algorithm of the cone transformation and the obtained photon number-flight time data and other related parameters, thereby realizing non-line-of-sight reconstruction of the hidden object.

8. The non-line-of-sight imaging system based on spectral-temporal dual coding according to claim 5, characterized in that: The free spectral range is: ， Where c is the speed of light, T is the cavity thickness of the virtual image phase array, n is the cavity refractive index of the virtual image phase array, is the cavity tilt angle of the virtual image phase array, is the incident angle of the light beam in the virtual image phase array, is the exit angle of the light beam in the virtual image phase array cavity.

9. The non-line-of-sight imaging system based on spectral-temporal dual coding according to claim 7, characterized in that: The process of reconstructing and recovering the reflectivity information of the surface of the hidden object based on the Wiener filter deconvolution non-viewing area reconstruction algorithm and the obtained photon number-time of flight data and other related parameters specifically includes: A standard forward model is established under confocal conditions; wherein the formula of the standard forward model is: ； Where, Indicates the number of photons corresponding to each scanning point on the repeater surface - flight time data, ) is the scanning point position on the relay wall surface, ) represents the spatial position point on the surface of the hidden object, r is the distance between the spatial position point on the surface of the hidden object and the corresponding scanning point on the surface of the relay wall, represents the reflectivity of the hidden object surface, represents the Dirac function, represents the three-dimensional space where the hidden object is located, t represents the flight time, and c represents the speed of light; The standard forward model is converted into a three-dimensional convolution form by using a cone transformation to obtain a forward model in a three-dimensional convolution form. The cone transformation formula is: , where represents z in the new coordinate system, and then we can deduce that Indicates the new coordinate system ; The forward model formula of the three-dimensional convolution form is: Where, Represents the photon number-time of flight data in the form of a three-dimensional convolution, Represents the reflectivity of the hidden object surface in the form of a 3D convolution; get: , ; Based on the Wiener filter three-dimensional deconvolution, a non-viewing reconstruction formula is established. 、 and other relevant parameters are brought into the non-viewing area reconstruction formula to reconstruct and restore the reflectivity information of the hidden object surface; wherein the non-viewing area reconstruction formula is: ; Where F represents the three-dimensional Fourier transform, Represents the surface reflectivity information of the hidden object, represents the focus matrix containing the Fourier coefficients of the three-dimensional convolution kernel, Represents the signal-to-noise ratio.

10. A non-line-of-sight imaging method based on spectral-temporal dual coding, implemented based on the non-line-of-sight imaging system based on spectral-temporal dual coding according to claim 1, characterized in that: The following steps are involved: Outputting laser light and processing the laser light to obtain a discrete pulse sequence with different central wavelengths; wherein the time interval between adjacent discrete pulses in the discrete pulse sequence is greater than the dead time of a detector used for subsequent detection; Amplifying and splitting the discrete pulse sequence to obtain linearly polarized light in the reflection direction, dispersing the linearly polarized light in the reflection direction, and outputting multiple light beams corresponding to the discrete pulse sequence, which are irradiated on the surface of the relay wall to form two-dimensional orthogonal discrete illumination; wherein the multiple light beams are output in a manner that satisfies constructive interference conditions; The relay wall surface undergoes diffuse reflection based on the two-dimensional orthogonal discrete illumination. When blocked by an obstacle, part of the diffusely reflected light is irradiated on the surface of the hidden object, and is reflected again on the surface of the hidden object back to the relay wall surface. After diffuse reflection again on the relay wall surface, the non-viewing zone tertiary echo light beams are collected by the detector after being focused. The required imaging information is calculated based on the detection signal corresponding to the non-viewing zone triple echo light beam collected by the detector.

Images