Non-vision field imaging method and device based on quasi-Fresnel transformation

Abstract

The invention provides a non-vision-field imaging method and device based on quasi-Fresnel transformation, and belongs to the technical field of optical non-vision-field imaging. The method comprises the following steps: collecting photon echo data when non-vision-field imaging is carried out on a non-vision-field target as a detection signal; calculating a convolution size parameter of the detection signal based on quasi-Fresnel transform; calculating a two-dimensional complex representation of the detection signal based on the convolution size parameter; calculating a two-dimensional complex representation of the non-view target based on the two-dimensional complex representation of the detection signal; and calculating reflectivity and depth information of the non-vision target based on the two-dimensional complex representation of the non-vision target so as to realize imaging of the non-vision target. According to the method, the two-dimensional features of the surface of the non-vision-field target are fully utilized, the non-vision-field target can be accurately and efficiently reconstructed, and the method has the advantages of being high in calculation speed, small in used memory, low in data dependence and accurate in imaging and positioning.

Description

Technical Field

[0001] The present invention belongs to the technical field of optical non-line-of-sight imaging, and in particular relates to a non-line-of-sight imaging method and device based on quasi-Fresnel transformation. Background Art

[0002] Non-line-of-sight imaging technology uses multiple reflected light signals to reconstruct targets outside the field of view, and has potential application value in the fields of medical imaging, autonomous driving, and remote sensing. The typical non-line-of-sight imaging method is to use laser pulses emitted by a laser to scan the intermediate wall surface, and the detector receives the light signal after multiple reflections and reconstructs the target outside the field of view based on it. When the irradiation point of the laser on the intermediate wall surface coincides with the receiving point of the detector on the intermediate wall surface, this detection mode is called confocal detection mode. Because this mode is convenient for developing fast algorithms, it has received widespread attention. At present, the main methods of confocal detection mode non-line-of-sight imaging include back projection method, light cone transformation method, wave migration method, and phasor field method. However, these algorithms all use three-dimensional voxels to represent non-line-of-sight targets, without considering the two-dimensional characteristics of the object surface, resulting in slow calculation speed and large memory usage. This problem seriously hinders the goal of achieving high-resolution real-time imaging on low-performance devices. Summary of the invention

[0003] The purpose of the present invention is to overcome the shortcomings of the prior art and to propose a non-line-of-sight imaging method and device based on quasi-Fresnel transform. The present invention has the advantages of fast calculation speed, less memory usage, low data dependence, and accurate imaging and positioning.

[0004] The first aspect of the present invention provides a non-line-of-sight imaging method based on quasi-Fresnel transform, comprising:

[0005] Collecting photon echo data when performing non-line-of-sight imaging on non-line-of-sight targets as detection signals;

[0006] Based on the quasi-Fresnel transform, calculating the convolution size parameter of the detection signal;

[0007] Based on the convolution size parameter, calculating a two-dimensional complex representation of the detection signal;

[0008] Calculating a two-dimensional complex representation of the non-viewing target based on the two-dimensional complex representation of the detection signal;

[0009] Based on the two-dimensional complex representation of the non-line-of-sight target, the reflectivity and depth information of the non-line-of-sight target are calculated to achieve imaging of the non-line-of-sight target.

[0010] In a specific implementation of the present invention, the calculating of the convolution size parameter of the detection signal includes:

[0011] When imaging in the non-viewing area, the detection area on the intermediate wall is a rectangle, and the lengths of the sides of the rectangle in the horizontal and vertical directions are l x and l y , let τ(x, t) be the detection signal of point x on the intermediate wall at time t, and the discretization of the signal is a size N x ×N y ×N t A three-dimensional array, where N x and N y Respectively represent the number of scanning points in the horizontal and vertical directions on the intermediate wall, N t Represents the number of detections in the time dimension;

[0012] Then use the following formula to calculate the convolution size parameter:

[0013]

[0014] Where l represents the calculated side length, l = max(l x , l y ), max(·) represents the function of taking the maximum value, N is the number of scanning points corresponding to the dimension l, π represents pi, c is the speed of light, n represents the maximum number of cycles allowed to differ between the phase after time discretization and the true phase, δ t represents the one-dimensional resolution of time t.

[0015] In a specific implementation of the present invention, the two-dimensional complex representation of the detection signal is calculated in any of the following ways:

[0016] Method 1:

[0017] Calculate the two-dimensional complex representation φ(x; s) of the detection signal of point x on the intermediate wall under the convolution size parameter s. The expression is as follows:

[0018]

[0019] Among them, i satisfies represents an imaginary unit;

[0020] Method 2:

[0021] Calculate the two-dimensional complex representation φ(x; s) of the detection signal of point x on the intermediate wall under the convolution size parameter s. The expression is as follows:

[0022]

[0023] Among them, T k represents the time when the kth photon event is recorded, and K represents the total number of photon events recorded.

[0024] In a specific implementation of the present invention, the calculating the two-dimensional complex representation of the non-viewing target includes:

[0025] Let the projection point of any point of the non-viewing target on the intermediate wall be x, then the calculation expression of the two-dimensional complex representation ψ(x; s) of the non-viewing target point under the convolution size parameter s is as follows:

[0026]

[0027] in, represents a two-dimensional plane, is a point on a two-dimensional plane.

[0028] In a specific implementation of the present invention, the calculating the reflectivity and depth information of the non-viewing target includes:

[0029] Calculate the reflectivity a(x) corresponding to the projection point x of the non-viewing target on the intermediate wall:

[0030] a(x)=|ψ(x;s)| (5)

[0031] Among them, |·| represents the modulus length of the complex number;

[0032] Let the projection point of any point of the non-viewing target on the intermediate wall be x, then the distance from the point of the non-viewing target to the intermediate wall is the depth d(x) of the point, and the calculation expression is as follows:

[0033]

[0034] In a specific implementation of the present invention, it also includes:

[0035] A pixel-level adaptive low-pass Wiener filter is applied to the reflectivity a(x) to obtain the updated reflectivity.

[0036] In a specific implementation of the present invention, it also includes:

[0037] Update depth information, including:

[0038] The current convolution size parameter s is recorded as s1, and a new convolution size parameter is selected and recorded as s2;

[0039] Let the number of cycles m be an integer, and the calculation expression is as follows:

[0040]

[0041] Among them, round(·) represents the rounding function, arg(z) is the argument of the complex number z;

[0042] Based on the result of formula (7), the updated depth information is obtained:

[0043]

[0044] A second aspect of the present invention provides a non-line-of-sight imaging device based on quasi-Fresnel transform, comprising:

[0045] A detection signal acquisition module is used to collect photon echo data when performing non-line-of-sight imaging on a non-line-of-sight target as a detection signal;

[0046] A convolution size parameter calculation module, used to calculate the convolution size parameter of the detection signal based on the quasi-Fresnel transform;

[0047] A detection signal two-dimensional complex representation calculation module, used to calculate the two-dimensional complex representation of the detection signal based on the convolution size parameter;

[0048] A non-line-of-sight target two-dimensional complex representation calculation module, used to calculate the two-dimensional complex representation of the non-line-of-sight target based on the two-dimensional complex representation of the detection signal;

[0049] An imaging module is used to calculate the reflectivity and depth information of the non-line-of-sight target based on the two-dimensional complex representation of the non-line-of-sight target, so as to realize imaging of the non-line-of-sight target.

[0050] A third aspect of the present invention provides an electronic device, including:

[0051] at least one processor; and a memory communicatively coupled to the at least one processor;

[0052] The memory stores instructions that can be executed by the at least one processor, and the instructions are configured to execute the above-mentioned non-line-of-sight imaging method based on quasi-Fresnel transform.

[0053] A fourth aspect of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the above-mentioned non-line-of-sight imaging method based on quasi-Fresnel transform.

[0054] Features and beneficial effects of the present invention:

[0055] The present invention realizes a two-dimensional representation of input data and non-line-of-sight target information by convolving data with a convolution kernel similar to Fresnel scattering, and then quickly reconstructs the reflectivity and depth information of the non-line-of-sight target through a two-dimensional deconvolution operation, and can image the non-line-of-sight target more accurately and efficiently. The present invention makes full use of the two-dimensional features of the surface of the non-line-of-sight target, makes up for the deficiencies of existing imaging technologies in terms of computational time and spatial complexity, can accurately and efficiently reconstruct the non-line-of-sight target, and has the advantages of fast computational speed, less memory usage, low data dependence, and accurate imaging and positioning.

[0056] Compared with existing imaging technologies, the calculation speed of the present invention can be increased by dozens of times, and the memory usage is reduced to nearly one thousandth of the original. In addition, the boundaries of the reconstruction results obtained by the present invention are clearer and the depth information is more accurate. Compared with the light cone transformation method, the present invention requires less data while achieving the same reconstruction effect.

[0057] The present invention makes it possible to achieve high frame rate and high resolution real-time non-line-of-sight imaging on low-configuration devices such as mobile phones and embedded devices. This technology improves the data acquisition and transmission methods of hardware in non-line-of-sight imaging, and has potential application value in the fields of autonomous driving, disaster relief, landform detection, security and anti-terrorism, military confrontation, remote sensing imaging, medical imaging, etc. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 It is an overall flow chart of a non-line-of-sight imaging method based on quasi-Fresnel transform according to an embodiment of the present invention.

[0059] Figure 2 It is a schematic diagram of the reflectivity and depth information reconstruction results of a specific embodiment of the present invention. DETAILED DESCRIPTION

[0060] The present invention proposes a non-line-of-sight imaging method and device based on quasi-Fresnel transform, which is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0061] The first aspect of the present invention provides a non-line-of-sight imaging method based on quasi-Fresnel transform, comprising:

[0062] Collecting photon echo data when performing non-line-of-sight imaging on non-line-of-sight targets as detection signals;

[0063] Based on the quasi-Fresnel transform, calculating the convolution size parameter of the detection signal;

[0064] Based on the convolution size parameter, calculating a two-dimensional complex representation of the detection signal;

[0065] Calculating a two-dimensional complex representation of the non-viewing target based on the two-dimensional complex representation of the detection signal;

[0066] Based on the two-dimensional complex representation of the non-line-of-sight target, the reflectivity and depth information of the non-line-of-sight target are calculated to achieve imaging of the non-line-of-sight target.

[0067] In a specific embodiment of the present invention, the non-line-of-sight imaging method based on quasi-Fresnel transform has the following overall process: Figure 1 As shown, the following steps are included:

[0068] 1) In non-line-of-sight imaging, a laser is used to emit laser light to the intermediate wall, and a detector is used to collect photon echo data as a detection signal.

[0069] In this embodiment, the laser emits a detection laser to the intermediate wall surface, and the point irradiated on the intermediate surface is called the irradiation point. The photon enters the non-viewing area through diffuse reflection, and is reflected back to the intermediate wall surface on the non-viewing target surface. At this time, a photon receiving device (i.e., a detector) is used to detect the photon echo intensity information with high time resolution at a certain point on the intermediate wall surface, and this point is called a receiving point. When all the irradiation points are the same as the receiving points, it is called a confocal detection mode; otherwise it is called a non-confocal detection mode. The situation of the non-confocal detection mode can be approximately converted into data processing of the confocal detection mode by translating the data. This embodiment has no special requirements for the detection mode, and in a specific embodiment of the present invention, it is a confocal detection mode. This embodiment has no special requirements for both the laser and the detector. In a specific embodiment of the present invention, NKTPhotonics OneFive KATANA 05 HP is used as a laser with a wavelength of 532nm, a pulse width of 35ps, and a repetition frequency of 10MHz. The detection module is composed of a single-photon avalanche photodiode (Micro Photon Devices PDM seriesSPAD), a time-correlated single-photon counter (PicoQuant PicoHarp 300), a two-dimensional scanning galvanometer and other devices. Among them, the single-photon avalanche photodiode is used as a detector, and its response band is the visible light band, and it has a gating function. The time-correlated single-photon counter is used as a counting unit, and its time resolution is 32ps, which together with the detector constitutes a detection module. This embodiment has no special requirements for the target of non-field of view imaging.

[0070] Furthermore, in this embodiment, the detection area on the intermediate wall is a rectangle, and its side lengths in the horizontal and vertical directions are l and x and l y , let τ(x, t) be the detection signal of point x on the intermediate wall at time t, and the discretization of the signal is a size N x ×N y ×N t A three-dimensional array, where Nx and N y Represents the number of scanning points in the horizontal and vertical directions on the intermediate wall, N t represents the number of detections in the time dimension, and the resolution of one dimension of time t is δ t In general, the detection area is a square, i.e. l ​​= l x = l y , where l represents the calculated side length; and N = N x =N y , where N represents the number of calculation scanning points. Otherwise, when l x ≠l y When the calculated side length is l=max(l x , l y ), N is taken as the number of scanning points in the dimension corresponding to l, where max(·,·) represents the function of taking the maximum value. In a specific embodiment of the present invention, the detection area is a square, the calculated side length l is the side length of the square 2m, and the time resolution δ t is taken as 32ps, the number of calculated scanning points N is taken as 512, N t Take it as 250.

[0071] 2) Based on the data collected in step 1), the convolution size parameters of the detection signal are calculated.

[0072] This embodiment comprehensively considers the theoretical optimality and the limitation of time resolution, and calculates the convolution size parameter s in the following way:

[0073]

[0074] Wherein, π represents the circumference of a circle, c represents the speed of light, and n represents the maximum number of cycles that the phase after time discretization is allowed to differ from the real phase, and the value range is usually between 0.05 and 0.1. In a specific embodiment of the present invention, n=0.1.

[0075] 3) Based on the convolution size parameter obtained in step 2), a two-dimensional complex representation of the detection signal is calculated.

[0076] In this embodiment, the two-dimensional complex representation φ(x; s) of the detection signal of a point x on the intermediate wall under the convolution size parameter s is calculated in the following manner:

[0077]

[0078] Among them, i satisfies Represents an imaginary unit.

[0079] It can be seen from formula (2) that in this embodiment, it is only necessary to obtain the weighted sum of the three-dimensional measurement data along the time dimension. This step can be directly implemented on the photon receiving device, further improving the time and space efficiency.

[0080] Therefore, further,

[0081] This embodiment uses the following formula to directly calculate the corresponding two-dimensional complex representation φ(x; s) of the detection signal of the point x on the intermediate wall under the convolution size parameter s as a hardware replacement for formula (2) (if the hardware does not support it, the calculation is still performed using formula (2)):

[0082]

[0083] Among them, T k represents the time when the kth photon event is recorded, and X represents the total number of recorded photon events.

[0084] This embodiment can use the photon events recorded by the detector to directly obtain the two-dimensional complex representation of the detection signal through formula (3), without the need to obtain a three-dimensional array τ(x, t), further improving the calculation and storage efficiency.

[0085] 4) Based on the result of step 3), calculate the two-dimensional complex representation of the non-viewing target.

[0086] In this embodiment, the projection point of any point of the non-viewing target on the intermediate wall is denoted as x, and the two-dimensional complex number representation ψ(x; s) of the point of the non-viewing target under the convolution size parameter s is calculated from φ(x; s), and the expression is as follows:

[0087]

[0088] in, represents a two-dimensional plane, is a point on a two-dimensional plane.

[0089] Formula (4) is a two-dimensional convolution, and in a specific embodiment of the present invention, fast Fourier transform is used to efficiently implement it. By representing the non-viewing target and the detection signal with two-dimensional data and establishing a two-dimensional mapping between the two, the method described in this embodiment can make full use of the two-dimensional characteristics of the non-viewing target surface to achieve efficient reconstruction.

[0090] 5) Based on the result of step 4), calculate the reflectivity and depth information of the non-viewing area target.

[0091] In this embodiment, the reflectivity a(x) corresponding to the projection point x of the non-viewing target on the intermediate wall is calculated in the following manner:

[0092] a(x)=|ψ(x;s)| (5)

[0093] Here, |·| represents the modulus length of the complex number.

[0094] Let the projection point of any point of the non-viewing target on the intermediate wall be x, then the distance from the point of the non-viewing target to the intermediate wall is the depth d(x) of the point, and the calculation expression is as follows:

[0095]

[0096] Among them, the partial derivative It can be calculated by the difference method, that is, by selecting two similar parameters s to calculate formula (2) and (4) and then taking the difference. In a specific embodiment of the present invention, s is selected as 0.05 and 0.05+10 respectively. -7 Estimate the depth information.

[0097] In this embodiment, the reflectivity a(x) calculated by equation (5) and the depth d(x) calculated by equation (6) constitute the final non-line-of-sight imaging result. The reflectivity map and depth map of the non-line-of-sight target can be obtained using a(x) and d(x), respectively.

[0098] Furthermore, the method described in this embodiment can also fine-tune the parameter s selected in step 2) by observing the imaging result, and then repeat steps 3) to 5) to achieve a better imaging effect. In this embodiment, the fine-tuning range of s can be ±10% of the result calculated by formula (1).

[0099] Furthermore, the method described in this embodiment can also improve the imaging quality by improving the reflectivity and depth information, as follows:

[0100] 6) Improve the result obtained in step 5) to improve the quality of reflectivity and depth information respectively; the specific steps are as follows:

[0101] 6-1) Improve reflectivity quality.

[0102] In a specific embodiment of the present invention, a pixel-level adaptive low-pass Wiener filter is used on the reflectivity a(x) obtained in step 5) to reduce the background noise of the reconstructed image.

[0103] 6-2) Improve the quality of depth information.

[0104] In this embodiment, the current convolution size parameter s is recorded as s1, and a new convolution size parameter is selected and recorded as s2.

[0105] Define the number of cycles m as an integer, and the calculation expression is as follows:

[0106]

[0107] Wherein, round(·) represents the rounding function, and arg(z) is the argument of the complex number z. It should be noted that s2 is generally selected to be slightly larger than s1, with a difference of 1%-2%. In a specific embodiment of the present invention, s1 is 0.05 and s2 is 0.05+10. -3 .

[0108] Based on the result of formula (7), the updated depth information is obtained:

[0109]

[0110] We get more accurate depth information at point x

[0111] Furthermore, in this embodiment, another new parameter s2 may be selected to repeat step 6-2) to further improve the quality of the depth information, wherein the difference between the newly selected parameter s2 and s1 is larger, for example, 20% larger than s1.

[0112] Figure 2 The reconstruction results of reflectivity and depth information after adopting the method described in this embodiment in a specific embodiment of the present invention are shown. Figure 2 In the figure, from left to right are the original image of the non-viewing target, the enhanced reflectivity reconstruction result, and the depth information reconstruction result. This embodiment uses only the data of the first 250 time intervals to obtain a relatively ideal reconstruction result. Compared with the existing method, this embodiment uses half the amount of data to reconstruct a clear result, and the calculation time and memory usage are reduced by about 78 times and 800 times respectively.

[0113] A second aspect of the present invention provides a non-line-of-sight imaging device based on quasi-Fresnel transform, comprising:

[0114] A detection signal acquisition module is used to collect photon echo data when performing non-line-of-sight imaging on a non-line-of-sight target as a detection signal;

[0115] A convolution size parameter calculation module, used to calculate the convolution size parameter of the detection signal based on the quasi-Fresnel transform;

[0116] A detection signal two-dimensional complex representation calculation module, used to calculate the two-dimensional complex representation of the detection signal based on the convolution size parameter;

[0117] A non-line-of-sight target two-dimensional complex representation calculation module, used to calculate the two-dimensional complex representation of the non-line-of-sight target based on the two-dimensional complex representation of the detection signal;

[0118] An imaging module is used to calculate the reflectivity and depth information of the non-line-of-sight target based on the two-dimensional complex representation of the non-line-of-sight target, so as to realize imaging of the non-line-of-sight target.

[0119] It should be noted that the aforementioned explanation of an embodiment of a non-line-of-sight imaging method based on quasi-Fresnel transform is also applicable to a non-line-of-sight imaging device based on quasi-Fresnel transform in this embodiment, and will not be repeated here. According to an embodiment of the present invention, a non-line-of-sight imaging device based on quasi-Fresnel transform is proposed, by collecting photon echo data when performing non-line-of-sight imaging on a non-line-of-sight target as a detection signal; based on the quasi-Fresnel transform, the convolution size parameter of the detection signal is calculated; based on the convolution size parameter, the two-dimensional complex representation of the detection signal is calculated; based on the two-dimensional complex representation of the detection signal, the two-dimensional complex representation of the non-line-of-sight target is calculated; based on the two-dimensional complex representation of the non-line-of-sight target, the reflectivity and depth information of the non-line-of-sight target are calculated to achieve imaging of the non-line-of-sight target.

[0120] In a specific implementation of the present invention, the calculating of the convolution size parameter of the detection signal includes:

[0121] When imaging in the non-viewing area, the detection area on the intermediate wall is a rectangle, and the lengths of the sides of the rectangle in the horizontal and vertical directions are l x and l y , let τ(x, t) be the detection signal of point x on the intermediate wall at time t, and the discretization of the signal is a size N x ×N y ×N t A three-dimensional array, where N x and N y Respectively represent the number of scanning points in the horizontal and vertical directions on the intermediate wall, N t Represents the number of detections in the time dimension;

[0122] Then use the following formula to calculate the convolution size parameter:

[0123]

[0124] Where l represents the calculated side length, l = max(l x , l y ), max(·) represents the function of taking the maximum value, N is the number of scanning points corresponding to the dimension l, π represents pi, c is the speed of light, n represents the maximum number of cycles allowed to differ between the phase after time discretization and the true phase, δ t represents the one-dimensional resolution of time t.

[0125] In a specific implementation of the present invention, the two-dimensional complex representation of the detection signal is calculated in any of the following ways:

[0126] Method 1:

[0127] Calculate the two-dimensional complex representation φ(x; s) of the detection signal of point x on the intermediate wall under the convolution size parameter s. The expression is as follows:

[0128]

[0129] Among them, i satisfies represents an imaginary unit;

[0130] Method 2:

[0131] Calculate the two-dimensional complex representation φ(x; s) of the detection signal of point x on the intermediate wall under the convolution size parameter s. The expression is as follows:

[0132]

[0133] Among them, T k represents the time when the kth photon event is recorded, and K represents the total number of photon events recorded.

[0134] In a specific implementation of the present invention, the calculating the two-dimensional complex representation of the non-viewing target includes:

[0135] Let the projection point of any point of the non-viewing target on the intermediate wall be x, then the calculation expression of the two-dimensional complex representation ψ(x; s) of the non-viewing target point under the convolution size parameter s is as follows:

[0136]

[0137] in, represents a two-dimensional plane, is a point on a two-dimensional plane.

[0138] In a specific implementation of the present invention, the calculating the reflectivity and depth information of the non-viewing target includes:

[0139] Calculate the reflectivity a(x) corresponding to the projection point x of the non-viewing target on the intermediate wall:

[0140] a(x)=|ψ(x;s)| (5)

[0141] Among them, |·| represents the modulus length of the complex number;

[0142] Let the projection point of any point of the non-viewing target on the intermediate wall be x, then the distance from the point of the non-viewing target to the intermediate wall is the depth d(x) of the point, and the calculation expression is as follows:

[0143]

[0144] In a specific implementation of the present invention, it also includes:

[0145] A pixel-level adaptive low-pass Wiener filter is applied to the reflectivity a(x) to obtain the updated reflectivity.

[0146] In a specific implementation of the present invention, it also includes:

[0147] Update depth information, including:

[0148] The current convolution size parameter s is recorded as s1, and a new convolution size parameter is selected and recorded as s2;

[0149] Let the number of cycles m be an integer, and the calculation expression is as follows:

[0150]

[0151] Among them, round(·) represents the rounding function, arg(z) is the argument of the complex number z;

[0152] Based on the result of formula (7), the updated depth information is obtained:

[0153]

[0154] This can fully utilize the two-dimensional features of the non-line-of-sight target surface, make up for the shortcomings of existing imaging technology in terms of computational time and spatial complexity, and reconstruct the non-line-of-sight target accurately and efficiently.

[0155] To implement the above embodiment, a third aspect of the present invention provides an electronic device, including:

[0156] at least one processor; and a memory communicatively coupled to the at least one processor;

[0157] The memory stores instructions that can be executed by the at least one processor, and the instructions are configured to execute the above-mentioned non-line-of-sight imaging method based on quasi-Fresnel transform.

[0158] To implement the above-mentioned embodiment, a fourth aspect of the present invention proposes a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the above-mentioned non-line-of-sight imaging method based on quasi-Fresnel transform.

[0159] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0160] The computer-readable medium may be included in the electronic device, or may exist independently without being installed in the electronic device. The computer-readable medium carries one or more programs, and when the one or more programs are executed by the electronic device, the electronic device executes a non-line-of-sight imaging method based on quasi-Fresnel transform in the above embodiment.

[0161] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0162] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0163] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0164] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0165] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute the instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purpose of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or otherwise processing in a suitable manner if necessary, and then stored in a computer memory.

[0166] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0167] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0168] In addition, each functional unit in each embodiment of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0169] The storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present application. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A non-line-of-sight imaging method based on quasi-Fresnel transform, characterized in that: include: Collecting photon echo data when performing non-line-of-sight imaging on non-line-of-sight targets as detection signals; Based on the quasi-Fresnel transform, calculating the convolution size parameter of the detection signal; Based on the convolution size parameter, calculating a two-dimensional complex representation of the detection signal; Calculating a two-dimensional complex representation of the non-viewing target based on the two-dimensional complex representation of the detection signal; Based on the two-dimensional complex representation of the non-line-of-sight target, the reflectivity and depth information of the non-line-of-sight target are calculated to achieve imaging of the non-line-of-sight target.

2. The method according to claim 1, characterized in that The calculating the convolution size parameter of the detection signal includes: When imaging in the non-viewing area, the detection area on the intermediate wall is a rectangle, and the lengths of the sides of the rectangle in the horizontal and vertical directions are l x and l y , let τ(x, t) be the detection signal of point x on the intermediate wall at time t, and the discretization of the signal is a size N x ×N y ×N t A three-dimensional array, where N x and N y Respectively represent the number of scanning points in the horizontal and vertical directions on the intermediate wall, N t Represents the number of detections in the time dimension; Then use the following formula to calculate the convolution size parameter: Where l represents the calculated side length, l = max(l x , l y ), max(·) represents the function of taking the maximum value, N is the number of scanning points corresponding to the dimension l, π represents pi, c is the speed of light, n represents the maximum number of cycles allowed to differ between the phase after time discretization and the true phase, δ t represents the one-dimensional resolution of time t.

3. The method according to claim 2, characterized in that The two-dimensional complex number representation of the detection signal is calculated in any of the following ways: Method 1: Calculate the two-dimensional complex representation φ(x; s) of the detection signal of point x on the intermediate wall under the convolution size parameter s. The expression is as follows: Among them, i satisfies represents an imaginary unit; Method 2: Calculate the two-dimensional complex representation φ(x; s) of the detection signal of point x on the intermediate wall under the convolution size parameter s. The expression is as follows: Among them, T k represents the time when the kth photon event is recorded, and K represents the total number of photon events recorded.

4. The method according to claim 3, characterized in that The calculating the two-dimensional complex representation of the non-viewing target comprises: Let the projection point of any point of the non-viewing target on the intermediate wall be x, then the calculation expression of the two-dimensional complex representation ψ(x; s) of the non-viewing target point under the convolution size parameter s is as follows: in, represents a two-dimensional plane, is a point on a two-dimensional plane.

5. The method according to claim 4, characterized in that The calculating the reflectivity and depth information of the non-viewing target includes: Calculate the reflectivity a(x) corresponding to the projection point x of the non-viewing target on the intermediate wall: a(x)=|ψ(x;s)| (5) Among them, |·| represents the modulus length of the complex number; Let the projection point of any point of the non-viewing target on the intermediate wall be x, then the distance from the point of the non-viewing target to the intermediate wall is the depth d(x) of the point, and the calculation expression is as follows:

6. The method according to claim 5, characterized in that Also includes: A pixel-level adaptive low-pass Wiener filter is applied to the reflectivity a(x) to obtain the updated reflectivity.

7. The method according to claim 5, characterized in that Also includes: Update depth information, including: The current convolution size parameter s is recorded as s1, and a new convolution size parameter is selected and recorded as s2; Let the number of cycles m be an integer, and the calculation expression is as follows: Among them, round(·) represents the rounding function, arg(z) is the argument of the complex number z; Based on the result of formula (7), the updated depth information is obtained:

8. A non-line-of-sight imaging device based on quasi-Fresnel transform, characterized in that: include: A detection signal acquisition module is used to collect photon echo data when performing non-line-of-sight imaging on a non-line-of-sight target as a detection signal; A convolution size parameter calculation module, used to calculate the convolution size parameter of the detection signal based on the quasi-Fresnel transform; A detection signal two-dimensional complex representation calculation module, used to calculate the two-dimensional complex representation of the detection signal based on the convolution size parameter; A non-line-of-sight target two-dimensional complex representation calculation module, used to calculate the two-dimensional complex representation of the non-line-of-sight target based on the two-dimensional complex representation of the detection signal; An imaging module is used to calculate the reflectivity and depth information of the non-line-of-sight target based on the two-dimensional complex representation of the non-line-of-sight target, so as to realize imaging of the non-line-of-sight target.

9. An electronic device, characterized in that: include: at least one processor; and, a memory communicatively coupled to the at least one processor; The memory stores instructions executable by the at least one processor, and the instructions are configured to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the method according to any one of claims 1 to 7.

Images