Non-line-of-sight imaging method and device

Abstract

The present invention provides a non-line-of-sight imaging method and device, which can be applied to the field of laser three-dimensional imaging technology. The method includes: performing multi-point scanning on a relay surface by controlling the emission angle of a pulsed laser, and calculating the flight time of pulsed laser photons based on the echo signal of the relay surface obtained by the multi-point scanning; measuring and calibrating the relay surface based on the emission angle of the pulsed laser and the flight time of the pulsed laser photons to obtain three-dimensional information of the relay surface; adaptively generating multiple new scanning points based on the three-dimensional information of the relay surface, and obtaining the gating information of each new scanning point using the peak information of each new scanning point and a preset delay value; scanning the relay surface point by point using the gating information of multiple new scanning points and the new emission angle of the pulsed laser to obtain the echo signal of the hidden space to be detected; processing the echo information of the hidden space to be detected using a preset non-line-of-sight imaging algorithm to obtain the reconstruction result of the hidden space to be detected.

Description

Technical Field

[0001] The present invention relates to the technical field of laser three-dimensional imaging, and in particular to a non-line-of-sight imaging method and device. Background Art

[0002] The field of view of traditional optical imaging technology is usually limited to the line of sight of the human eye or the direct observation area of ​​the imaging device, so a large amount of visual information outside the field of view cannot be captured. However, the emergence of non-line-of-sight imaging technology has broken through this limitation, making it possible to detect targets in the blind spot of the field of view. By utilizing the diffuse reflection of the relay surface, optical non-line-of-sight imaging can image objects hidden from sight and achieve "corner imaging." This technological breakthrough not only breaks the boundaries of traditional optical imaging, but also significantly expands human perception capabilities. In the future, non-line-of-sight imaging technology is expected to show great application potential in fields such as autonomous driving, disaster relief, and medical diagnosis, and promote technological innovation and development in related industries.

[0003] However, the existing non-line-of-sight imaging technology solutions have the following technical problems: they can only observe the non-line-of-sight area close to the relay surface (about 1-2m); the effective range is relatively limited; the imaging time is long; most of them are laboratory demonstration systems, which are not conducive to transportation and use in various complex outdoor environments. Summary of the invention

[0004] In view of the above problems, the present invention provides a non-line-of-sight imaging method and device, which are used to solve at least one of the above technical problems.

[0005] According to a first aspect of the present invention, there is provided a non-line-of-sight imaging method, comprising:

[0006] The relay surface is scanned at multiple points by controlling the emission angle of the pulsed laser, and the flight time of the pulsed laser photons is calculated based on the echo signals of the relay surface obtained by the multi-point scanning;

[0007] The relay surface is measured and calibrated based on the emission angle of the pulsed laser and the flight time of the pulsed laser photons to obtain the three-dimensional information of the relay surface;

[0008] Adaptively generate multiple new scanning points based on the three-dimensional information of the relay surface, and use the peak information and preset delay value of each new scanning point to obtain the gating information of each new scanning point;

[0009] The gate information of multiple new scanning points and the new emission angle of the pulsed laser are used to scan the relay surface point by point to obtain the echo signal of the hidden space to be detected;

[0010] The preset non-line-of-sight imaging algorithm is used to process the echo information of the hidden space to be detected, and the reconstruction result of the hidden space to be detected is obtained.

[0011] According to an embodiment of the present invention, the three-dimensional information of the relay surface obtained by measuring and calibrating the relay surface based on the emission angle of the pulsed laser and the flight time of the pulsed laser photons includes:

[0012] A three-dimensional coordinate system is constructed by taking a control point for controlling the emission angle of the pulsed laser as an origin, and a plurality of rotation angles corresponding to the emission angle of the pulsed laser are determined;

[0013] Based on the preset coordinate calculation formula, the coordinate information of the scanning point in the three-dimensional coordinate system is calculated using multiple rotation angles and the flight time of the pulsed laser photon;

[0014] The relay surface is fitted using the coordinate information of the scanning points to obtain the three-dimensional information of the relay surface.

[0015] According to an embodiment of the present invention, the above preset coordinate formula is as shown in formulas (1) to (3):

[0016] (1),

[0017] (2),

[0018] (3),

[0019] in, Represents a three-dimensional coordinate system The rotation angle in the axis direction, Represents a three-dimensional coordinate system The rotation angle in the axis direction, represents the flight time of the pulsed laser photon, Represents a three-dimensional coordinate system The position in the axis direction, Represents a three-dimensional coordinate system The position in the axis direction, Represents a three-dimensional coordinate system Position along the axis.

[0020] According to an embodiment of the present invention, the preset delay value is set by the following operations:

[0021] Calculating the center distance between the center position of the hidden space to be detected and the center position of the relay surface;

[0022] The preset delay value is set to twice the flight time of the pulsed laser photon corresponding to the center distance.

[0023] According to an embodiment of the present invention, the above-mentioned preset non-line-of-sight imaging algorithm includes a phasor field non-line-of-sight imaging algorithm and a spectrum filtering non-line-of-sight imaging algorithm.

[0024] A second aspect of the present invention provides a non-line-of-sight imaging device for executing the above-mentioned non-line-of-sight imaging method, the device comprising an optical transceiver system and an electrical processing system;

[0025] Wherein, the optical transceiver system includes a laser, a first collimator, a first galvanometer, a first beam expander, a second collimator, a second galvanometer, and a second beam expander;

[0026] The electrical processing system includes a gate control circuit, a gate control detector, a time-to-digital converter and an industrial computer.

[0027] According to an embodiment of the present invention, the above-mentioned laser is used to emit a pulse laser with a preset pulse width and simultaneously output a reference signal with the same frequency as the pulse laser;

[0028] Wherein, the gating circuit receives a reference signal and generates a gating signal synchronized with the reference signal;

[0029] The time-to-digital converter receives a reference signal and uses it as a start signal for the flight time of the pulsed laser photons.

[0030] According to an embodiment of the present invention, the reference signal is implemented by the following operations:

[0031] A light signal is separated from the seed part of the laser light source, and the separated light signal is converted into an electrical signal by connecting to a photoelectric tube;

[0032] The electrical signal is shaped by passing through the discrimination board, and the shaped square wave signal is used as the reference signal.

[0033] According to an embodiment of the present invention, the emission angle of the pulsed laser emitted by the laser is controlled by controlling the rotation angle of the first galvanometer and the second galvanometer;

[0034] The first galvanometer and the second galvanometer include two-dimensional galvanometers based on a micro-electromechanical system.

[0035] According to an embodiment of the present invention, the gated detector comprises a gated single photon detector based on a single photon avalanche diode;

[0036] Wherein, the gate control circuit includes a first input signal and a second input signal;

[0037] The gate control circuit uses the first input signal as a reference signal to generate a gate control signal with the same frequency, variable delay and pulse width, and controls the gate opening time of the gate detector based on the gate control signal;

[0038] The gate control circuit adaptively transforms the signal delay based on the pixel signal represented by the second input signal, thereby adaptively adjusting the gate opening time of the gate control detector.

[0039] The non-line-of-sight imaging method and device provided by the present invention, due to the use of an adaptive gated detection scheme, greatly expands the distance from the system to the wall and the target to the wall. Compared with traditional non-line-of-sight imaging equipment that can only image targets within a range of 1-2m from the relay surface, it has the advantage of a long imaging distance; at the same time, due to the use of adaptive gated detection means, the echo signal of the relay surface, sunlight and other environmental noise can be effectively isolated, and the dark counts caused by the detector can be reduced, thereby effectively improving the acquisition rate and signal-to-noise ratio of the non-line-of-sight signal and improving the imaging speed; in addition, the device for executing the non-line-of-sight imaging method provided by the present invention adopts a wall calibration algorithm, so it can be easily moved to different positions and quickly complete preparatory work such as the setting of the scanning area and the gated delay. The device for executing the non-line-of-sight imaging method provided by the present invention adopts an integrated optical design, which has high stability and greatly facilitates the transportation of the device. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The above contents and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:

[0041] Figure 1 is a schematic diagram of the principle of non-line-of-sight imaging according to an embodiment of the present invention;

[0042] Figure 2 is a flow chart of a non-line-of-sight imaging method according to an embodiment of the present invention;

[0043] Figure 3 4 is a schematic diagram of the structure of a non-line-of-sight imaging device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0044] Below, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present invention. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of embodiments of the present invention. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of concepts of the present invention.

[0045] The terms used herein are only for describing specific embodiments and are not intended to limit the present invention. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0046] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0047] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0048] Figure 1 4 is a schematic diagram of the principle of non-line-of-sight imaging according to an embodiment of the present invention.

[0049] The basic process of non-line-of-sight imaging is as follows Figure 1 As shown: First, the non-line-of-sight imaging device actively emits laser light to the relay surface (such as the wall, the ground, etc.) (R1 in the figure), and the light is diffusely reflected on the relay surface to illuminate the hidden target object (R2 in the figure); then, the light is diffusely reflected again on the surface of the target object and returns to the relay surface (R3 in the figure); finally, the light is scattered again by the relay surface and returns to the detection end of the device (R4 in the figure). By performing high-sensitivity detection, correlation analysis and signal processing on the light signal received by the detection end, the non-line-of-sight imaging technology can effectively reconstruct the image information of the hidden object.

[0050] Non-line-of-sight imaging technology has important application value in many fields. Since the concept of non-line-of-sight imaging was proposed, more and more non-line-of-sight imaging technologies have been born, such as Light-Cone Transform (LCT), Frequency-wavenumber migration (FK Migration, FK), Virtual Wave Phase Field (Phasor Field, PF), Fermat Path (FermatPath, FP) and other iconic non-line-of-sight imaging technologies. These non-line-of-sight imaging technologies have significantly improved the three-dimensional reconstruction capabilities of non-line-of-sight imaging: Compared with the previous BP and other methods that can only restore simple targets such as numbers and letters, these imaging methods can reconstruct complex scenes containing multiple objects with high quality, preliminarily demonstrating the practical application potential of this technology. Since then, relevant research in this field has begun to turn to the development of practical technologies and devices, such as long-distance non-line-of-sight technologies and devices, fast non-line-of-sight technologies and devices, etc.

[0051] From the perspective of practical application, the current non-line-of-sight imaging technology and systems have the following main problems: only the non-line-of-sight area close to the relay surface (about 1-2m) can be observed. However, in applications such as emergency rescue, the non-line-of-sight space that needs to be observed can be as large as a room (more than 3m from the relay surface); the working range is relatively limited, and most systems or devices need to be placed at a position close to the relay surface (about 3-5m) for use, which cannot meet practical needs such as remote observation; the imaging time is long, often in the order of minutes, which cannot meet the real-time requirements of various practical applications; most of them are laboratory demonstration systems or devices, which are large in size, low in integration, poor in anti-interference, and require manual calibration before use, which is not conducive to transportation and use in various complex outdoor environments.

[0052] In response to the above technical problems, the present invention proposes a non-line-of-sight imaging technology based on an adaptive gated single-photon detection scheme. This technology first completely obtains the three-dimensional information of the relay surface through a self-calibration process; then uses the wall information to adaptively select the door opening time of the gated single-photon detector, thereby improving the collection efficiency of non-line-of-sight signals and effectively isolating interference such as ambient light. On the basis of this technology, the present invention also cooperates to design an integrated non-line-of-sight imaging device with a high level of engineering. Compared with existing imaging devices, the non-line-of-sight imaging device provided by the present invention can effectively expand the non-line-of-sight observation range and effective distance, improve the imaging speed, and is highly portable and easy to operate, and has a high practical value.

[0053] Figure 2 is a flow chart of a non-line-of-sight imaging method according to an embodiment of the present invention.

[0054] like Figure 2 As shown, the non-line-of-sight imaging method includes operations S210 to S250.

[0055] In operation S210, a relay surface is multi-point scanned by controlling an emission angle of a pulsed laser, and a flight time of pulsed laser photons is calculated based on an echo signal of the relay surface obtained by the multi-point scanning.

[0056] In operation S220, the relay surface is measured and calibrated based on the emission angle of the pulsed laser and the flight time of the pulsed laser photons to obtain three-dimensional information of the relay surface.

[0057] According to an embodiment of the present invention, the above-mentioned measurement and calibration of the relay surface based on the emission angle of the pulsed laser and the flight time of the pulsed laser photons to obtain the three-dimensional information of the relay surface includes: constructing a three-dimensional coordinate system with the control point that controls the emission angle of the pulsed laser as the origin, and determining multiple rotation angles corresponding to the emission angle of the pulsed laser; based on a preset coordinate calculation formula, calculating the coordinate information of the scanning point in the three-dimensional coordinate system using multiple rotation angles and the flight time of the pulsed laser photons; and fitting the relay surface using the coordinate information of the scanning point to obtain the three-dimensional information of the relay surface.

[0058] According to an embodiment of the present invention, the above preset coordinate formula is as shown in formulas (1) to (3):

[0059] (1),

[0060] (2),

[0061] (3),

[0062] in, Represents a three-dimensional coordinate system The rotation angle in the axis direction, Represents a three-dimensional coordinate system The rotation angle in the axis direction, represents the flight time of the pulsed laser photon, Represents a three-dimensional coordinate system The position in the axis direction, Represents a three-dimensional coordinate system The position in the axis direction, Represents a three-dimensional coordinate system Position along the axis.

[0063] In operation S230, a plurality of new scanning points are adaptively generated based on the three-dimensional information of the relay surface, and gating information of each new scanning point is obtained by using peak information and a preset delay value of each new scanning point.

[0064] According to an embodiment of the present invention, the above-mentioned preset delay value is set by the following operations: calculating the center distance between the center position of the hidden space to be detected and the center position of the relay surface; setting the preset delay value to twice the flight time of the pulsed laser photon corresponding to the center distance.

[0065] The above preset delay value is exemplary, and those skilled in the art can flexibly set the above preset delay value according to actual scenarios.

[0066] In operation S240, the relay surface is scanned point by point using the gating information of the multiple new scanning points and the new emission angle of the pulsed laser to obtain an echo signal of the hidden space to be detected.

[0067] In operation S250, the echo information of the hidden space to be detected is processed by using a preset non-line-of-sight imaging algorithm to obtain a reconstruction result of the hidden space to be detected.

[0068] According to an embodiment of the present invention, the above-mentioned preset non-line-of-sight imaging algorithm includes a phasor field non-line-of-sight imaging algorithm and a spectrum filtering non-line-of-sight imaging algorithm.

[0069] Those skilled in the art may adopt other non-line-of-sight imaging algorithms according to application scenarios.

[0070] Figure 3 4 is a schematic diagram of the structure of a non-line-of-sight imaging device according to an embodiment of the present invention.

[0071] like Figure 3 As shown, the non-line-of-sight imaging device for executing the above-mentioned non-line-of-sight imaging method includes an optical transceiver system and an electrical processing system; wherein, the optical transceiver system includes a laser, a first collimator, a first galvanometer, a first beam expander, a second collimator, a second galvanometer and a second beam expander; wherein, the electrical processing system includes a gating circuit, a gated detector, a time-to-digital converter and an industrial computer.

[0072] The non-line-of-sight imaging method provided by the present invention, due to the use of an adaptive gated detection scheme, greatly expands the distance from the imaging device to the wall and the target to the wall. Compared with the traditional non-line-of-sight imaging equipment that can only image targets within a range of 1-2m from the relay surface, it has the advantage of a long imaging distance; at the same time, due to the use of adaptive gated detection means, the echo signal of the relay surface, sunlight and other environmental noise can be effectively isolated, and the dark counts caused by the detector can be reduced, which effectively improves the acquisition rate and signal-to-noise ratio of the non-line-of-sight signal and improves the imaging speed. In addition, the device for executing the non-line-of-sight imaging method provided by the present invention can be easily moved to different positions due to the use of a wall calibration algorithm, and can quickly complete the preparation work such as the setting of the scanning area and gated delay. The device for executing the non-line-of-sight imaging method provided by the present invention adopts an integrated optical design, which has high stability and greatly facilitates the transportation of the device.

[0073] According to an embodiment of the present invention, the above-mentioned laser is used to emit a pulse laser with a preset pulse width and simultaneously output a reference signal with the same frequency as the pulse laser; wherein, the gating circuit receives the reference signal and generates a gating signal synchronized with the reference signal; wherein, the time-to-digital converter receives the reference signal and serves as a start signal for the flight time of the pulse laser photons.

[0074] According to an embodiment of the present invention, the above-mentioned reference signal is realized by the following operations: a light signal is separated from the seed part of the light source of the laser, and the separated light signal is converted into an electrical signal by connecting it to a photoelectric tube; the electrical signal is shaped by a discriminator, and the shaped square wave signal is used as a reference signal.

[0075] According to an embodiment of the present invention, the emission angle of the pulsed laser emitted by the laser is controlled by controlling the rotation angle of the first galvanometer and the second galvanometer; wherein the first galvanometer and the second galvanometer include two-dimensional galvanometers based on a micro-electromechanical system.

[0076] According to an embodiment of the present invention, the above-mentioned gated detector includes a gated single-photon detector based on a single-photon avalanche diode; wherein the gating circuit includes a first input signal and a second input signal; wherein the gating circuit uses the first input signal as a reference signal to generate a gating signal with the same frequency, variable delay and pulse width, and controls the gate opening time of the gated detector based on the gating signal; wherein the gating circuit adaptively transforms the signal delay based on the pixel signal represented by the second input signal to achieve adaptive adjustment of the gate opening time of the gated detector.

[0077] The non-line-of-sight imaging method is further described in detail below through specific implementations and in combination with the non-line-of-sight imaging device provided by the present invention.

[0078] In terms of non-line-of-sight imaging detection, the technical solution proposed in the present invention mainly includes three steps: wall measurement calibration, adaptive gating setting and non-line-of-sight imaging. In the first step, the non-line-of-sight imaging device emits a pulsed laser into the scene and receives the echo signal from the relay surface. The gated single-photon detector of the non-line-of-sight imaging device receives and records the flight time of the photon; the non-line-of-sight imaging device controls the galvanometer to scan multiple points on the wall, and the three-dimensional information of the wall can be obtained by calculation. In the second step, using the three-dimensional information of the wall, a new set of scanning points can be adaptively generated, and the gated detector opening time corresponding to each scanning point can be adaptively adjusted to isolate the signal of the relay surface and only detect the signal of the hidden space. In the third step, the non-line-of-sight imaging device emits a pulsed laser into the scene and receives the echo signal from the hidden space. The gated single-photon detector receives and records the flight time of the photon. The galvanometer is controlled to scan the entire relay surface to obtain the overall information of the hidden space. The scene of the hidden space can be reconstructed using the non-line-of-sight reconstruction algorithm.

[0079] In terms of wall measurement calibration, the non-line-of-sight imaging device transmits pulsed lasers into the scene through the transmitting galvanometer, and receives the echo signal from the relay surface through the receiving galvanometer. The gated single-photon detector receives and records the flight time of the photon. The galvanometer is controlled to scan multiple points on the wall to obtain N groups of galvanometer rotation angles. and the corresponding flight time information ( ). With the center of the galvanometer as the origin, a coordinate system is established, and the position of each scanning point in the coordinate system can be calculated, as shown in formulas (1) to (3):

[0080] (1),

[0081] (2),

[0082] (3),

[0083] in, Represents a three-dimensional coordinate system The rotation angle in the axis direction, Represents a three-dimensional coordinate system The rotation angle in the axis direction, represents the flight time of the pulsed laser photon, Represents a three-dimensional coordinate system The position in the axis direction, Represents a three-dimensional coordinate system The position in the axis direction, Represents a three-dimensional coordinate system Position along the axis.

[0084] Since the scanning plane should be a plane, the coordinates of each point should be in the same plane, as shown in formula (4):

[0085] (4).

[0086] Therefore, you can use Scanning points , , Perform plane fitting and obtain the above , , , that is, the three-dimensional information of the relay surface relative to the imaging device is obtained.

[0087] Among them, in terms of adaptive gating settings, based on the known three-dimensional information of the wall, a new set of scanning points can be adaptively generated for non-viewing imaging, and the primary peak position D corresponding to each scanning point can be calculated. A delay is added to this position information to obtain the gating signal corresponding to each scanning point. The delay value is generally set to twice the photon flight time corresponding to the distance from the center of the area to be detected to the center of the wall.

[0088] Among them, in terms of non-line-of-sight imaging, the pulsed laser is emitted into the scene again, the relay surface is scanned point by point, and the echo signal from the hidden space is received; the echo signal is processed using the existing non-line-of-sight imaging algorithm to reconstruct the non-line-of-sight hidden space.

[0089] The specific components and functions of the non-line-of-sight imaging device provided by the present invention are further described in detail below through specific implementation methods.

[0090] like Figure 3 As shown, the present invention adopts an integrated optical design, and the optical transceiver system adopts an integrated structure fixed beam expander and collimator, with a simple optical path, small size, easy installation and adjustment, and high stability. The galvanometer adopts a two-dimensional MEMS galvanometer, and the mirror diameter is only a few millimeters. With a small-aperture beam expansion system, the volume of the optical path is greatly reduced, and it has a resonant frequency of hundreds of hertz, which can achieve fast beam scanning.

[0091] In terms of the gated detection scheme, the present invention uses a gated single-photon detector based on a single-photon avalanche diode (SPAD) combined with a gated circuit. The gated circuit has two input signals. The first is a reference signal with the same frequency as the laser emission. The gated circuit uses this signal as a reference to generate a signal with the same frequency, variable delay and pulse width, which is used to control the gate opening time of the gated detector. The second signal is the pixel signal output by the galvanometer. The gated circuit stores the delay information corresponding to all scanning points. During the imaging process, the gated circuit will receive the pixel signal from the galvanometer and transform the corresponding signal delay to achieve adaptive adjustment of the detector opening time.

[0092] All of the above components are installed in a small structure to form a proprietary non-line-of-sight imaging device, which can be operated after connecting to power and network cables.

[0093] The non-line-of-sight imaging device provided by the present invention has the following advantages compared to the prior art: using a gated detector combined with pixel-adaptive gated detection technology effectively improves the signal-to-noise ratio and imaging speed (or reduces the time) of the non-line-of-sight imaging device; adopts a miniaturized and integrated system design, combined with a self-calibration method, the device can be easily transported to different locations, quickly complete pre-imaging preparations, and start working.

[0094] The non-field of view imaging method and device provided by the present invention are further described in detail below through specific experiments.

[0095] In the laser emission part, a pulse laser with a pulse width of less than or equal to 20ps can be used as the light source, and the laser is required to output a reference signal of the same frequency. One implementation method is to separate a light signal from the seed part of the light source, connect this signal to a photoelectric tube to convert it into an electrical signal, and then pass it through a discriminator for shaping to form a square wave signal as a reference signal. The reference signal is divided into two paths, one is connected to the TDC as the start signal of the flight time recording, and the other is connected to the gate circuit board to generate a synchronous gate signal.

[0096] The gate control circuit board has two main functions: (1) generating a square wave signal with the same frequency as the laser, connecting it to the gated detector to control the detector's switch. The delay and pulse width of the square wave signal relative to the input reference signal can be adjusted. Setting a suitable delay allows the detector to open when a valid photon signal returns and close at other times; (2) receiving the pixel signal from the galvanometer and adaptively changing the delay of the gate control signal.

[0097] The detector can be a multi-mode gated detector based on a single-photon avalanche diode (SPAD). The multi-mode receiving method can improve the receiving efficiency. The detector needs to have a faster gate opening time (≤1ns) to effectively achieve the isolation of the relay surface signal. Other single-photon detectors with gated modes or free-running single-photon detectors used in combination with high-speed optical switches can also be selected.

[0098] The signal generated by the detector will be connected to the time digital converter (TDC) as the end signal of the flight time. TDC requires a time accuracy of less than 16ps and a time jitter (full width at half maximum) of less than 20ps. For example, Swabian Instruments' TimeTagger Ultra has a time accuracy of 1ps and a time jitter full width at half maximum of 18ps.

[0099] The light receiving and transmitting circuit can use a 7.33x beam expander with a collimator to achieve high resolution beyond 40m. The beam expander and the focal length of the collimator can also be adjusted according to the scene distance. The galvanometer can use a two-dimensional MEMS galvanometer to obtain a smaller size and faster scanning speed. At the same time, the galvanometer driver has the function of pixel signal output, and other galvanometers with similar functions can also be used.

[0100] The non-line-of-sight imaging device also includes a USB hub and an industrial computer; all the components are installed in a small structure to form a small non-line-of-sight imaging device. The device can work after being connected to a power source and a network cable.

[0101] After the non-line-of-sight imaging device is built, the device can be used for non-line-of-sight detection. The first step is to set a set of galvanometer scanning angles to scan the relay surface, obtain the photon flight time from the relay surface, and calculate the three-dimensional information of the wall. At this time, the scanning angle is generally selected at equal intervals. Since the wall has a certain inclination angle relative to the imaging device, the photon flight time returned from different scanning points has a certain difference. In order to detect a single echo at all scanning points, the gated detector here can open a wider door or use the free running mode. The second step is to use the three-dimensional information of the wall to adaptively set the galvanometer scanning angle and the gating signal. The third step is to scan the relay surface again to obtain the photon flight time information from the hidden space, and use the non-line-of-sight imaging algorithm to reconstruct the image.

[0102] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0103] It will be appreciated by those skilled in the art that the features described in the various embodiments of the present invention may be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in the present invention. In particular, without departing from the spirit and teachings of the present invention, the features described in the various embodiments of the present invention may be combined and / or combined in various ways. All of these combinations and / or combinations fall within the scope of the present invention.

[0104] The embodiments of the present invention are described above. However, these embodiments are only for the purpose of illustration, and are not intended to limit the scope of the present invention. Although each embodiment is described above, it does not mean that the measures in each embodiment cannot be used in combination advantageously. Without departing from the scope of the present invention, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present invention.

Claims

1. A non-line-of-sight imaging method, characterized in that: The method comprises: The relay surface is scanned at multiple points by controlling the emission angle of the pulsed laser, and the flight time of the pulsed laser photons is calculated based on the echo signals of the relay surface obtained by the multi-point scanning; Measuring and calibrating the relay surface based on the emission angle of the pulsed laser and the flight time of the pulsed laser photons to obtain three-dimensional information of the relay surface; Adaptively generate a plurality of new scanning points based on the three-dimensional information of the relay surface, and obtain gating information of each new scanning point using peak information and a preset delay value of each new scanning point; Scanning the relay surface point by point using the gating information of the multiple new scanning points and the new emission angle of the pulsed laser to obtain an echo signal of the hidden space to be detected; The echo information of the hidden space to be detected is processed by using a preset non-line-of-sight imaging algorithm to obtain a reconstruction result of the hidden space to be detected.

2. The method according to claim 1, characterized in that The relay surface is measured and calibrated based on the emission angle of the pulsed laser and the flight time of the pulsed laser photons to obtain the three-dimensional information of the relay surface, including: A three-dimensional coordinate system is constructed by taking a control point for controlling the emission angle of the pulsed laser as an origin, and a plurality of rotation angles corresponding to the emission angle of the pulsed laser are determined; Based on a preset coordinate calculation formula, the coordinate information of the scanning point in the three-dimensional coordinate system is calculated using the plurality of rotation angles and the flight time of the pulsed laser photons; The relay surface is fitted using the coordinate information of the scanning points to obtain three-dimensional information of the relay surface.

3. The method according to claim 2, characterized in that The preset coordinate calculation formula is shown in formulas (1) to (3): （1）， （2）， （3）， in, Indicates the three-dimensional coordinate system The rotation angle in the axis direction, Indicates the three-dimensional coordinate system The rotation angle in the axis direction, represents the flight time of the pulsed laser photon, Indicates the three-dimensional coordinate system The position in the axis direction, Indicates the three-dimensional coordinate system The position in the axis direction, Indicates the three-dimensional coordinate system Position along the axis.

4. The method according to claim 1, characterized in that: The preset delay value is set by the following operations: Calculating the center distance between the center position of the hidden space to be detected and the center position of the relay surface; The preset delay value is set to twice the flight time of the pulsed laser photon corresponding to the center distance.

5. The method according to claim 1, characterized in that The preset non-line-of-sight imaging algorithm includes a phasor field non-line-of-sight imaging algorithm and a spectrum filtering non-line-of-sight imaging algorithm.

6. A non-line-of-sight imaging device for executing the method according to any one of claims 1 to 5, characterized in that: The device includes an optical transceiver system and an electrical processing system; Wherein, the optical transceiver system comprises a laser, a first collimator, a first galvanometer, a first beam expander, a second collimator, a second galvanometer and a second beam expander; Wherein, the electrical processing system includes a gate control circuit, a gate control detector, a time-to-digital converter and an industrial computer.

7. The device according to claim 6, characterized in that The laser is used to emit a pulse laser with a preset pulse width and simultaneously output a reference signal with the same frequency as the pulse laser; Wherein, the gating circuit receives the reference signal and generates a gating signal synchronized with the reference signal; The time-to-digital converter receives the reference signal and uses it as a start signal for the pulse laser photon flight time.

8. The device according to claim 7, characterized in that The reference signal is realized by the following operations: A light signal is separated from the seed part of the light source of the laser, and the separated light signal is converted into an electrical signal by connecting to a photoelectric tube; The electrical signal is shaped by a discriminator, and the shaped square wave signal is used as the reference signal.

9. The device according to claim 6, characterized in that Controlling the emission angle of the pulsed laser emitted by the laser by controlling the rotation angles of the first galvanometer and the second galvanometer; Wherein, the first galvanometer and the second galvanometer include two-dimensional galvanometers based on a micro-electromechanical system.

10. The device according to claim 6, characterized in that The gated detector comprises a gated single photon detector based on a single photon avalanche diode; Wherein, the gate control circuit includes a first input signal and a second input signal; Wherein, the gate control circuit uses the first input signal as a reference signal to generate a gate control signal with the same frequency, variable delay and pulse width, and controls the gate opening time of the gated detector based on the gate control signal; The gate control circuit adaptively transforms the signal delay based on the pixel signal represented by the second input signal to achieve adaptive adjustment of the gate opening time of the gate control detector.

Images