Time-of-Flight ranging method, Time-of-Flight ranging device, electronic device and computer-readable storage medium

Abstract

The present application discloses a time-of-flight ranging method, device, electronic device and storage medium. The method includes: obtaining a first histogram with a time bin length of a first preset duration, and performing a rough measurement operation on the first histogram to obtain a rough time of flight; obtaining a second histogram with a time bin length of a second preset duration, and performing a fine measurement operation on the second histogram to obtain a fine time of flight; the second preset duration is less than the first preset duration; determining the time of flight of the laser pulse based on the rough time of flight and the fine time of flight, and determining the distance of the external object according to the time of flight, so as to extract an accurate time of flight by using the rough time of flight and the fine time of flight, making the ranging result more accurate, and since this solution does not require calculating and correcting the histogram, a large amount of computing resources can be saved, and the ranging efficiency can be improved.

Description

Technical Field

[0001] This application relates to the field of optical detection technology, and particularly relates to a time-of-flight ranging method, a time-of-flight ranging device, an electronic device, and a computer-readable storage medium. Background Art

[0002] Time-of-flight technology (ToF, Time of Flight) is widely used as a ranging solution with long distance, high precision, and low power consumption. Currently, ToF technology mainly includes direct time of flight (dToF) and indirect time of flight (iTOF). Among them, the dToF scheme needs to emit and receive laser pulses, and measure the flight time of the laser pulses reflected by an object to achieve ranging. Usually, the dToF scheme uses a single photon avalanche diode (SPAD) to detect the photons of the reflected laser pulses, counts the time when the photons are received through a time-to-digital convertor (TDC), outputs a histogram, and then obtains the flight time of the received reflected laser pulses by finding the time bin with the highest count in the histogram or by calculating the centroid of the entire signal peak in the histogram.

[0003] However, in actual use, due to the presence of ambient light, the photon count in the histogram will be mixed with noise counts generated by photons of ambient light, which easily causes the pile-up effect. The later the detected time is, the lower the actual photon count in the corresponding time bin is compared with the ideal photon count value, and finally, the problem of being unable to correctly find the signal peak occurs. Summary of the Invention

[0004] The purpose of this application is to propose a time-of-flight ranging method, a time-of-flight ranging device, an electronic device, and a computer-readable storage medium for the deficiencies of the above-mentioned prior art, and this purpose is achieved through the following technical solutions.

[0005] The first aspect of this application proposes a time-of-flight ranging method, and the method includes:

[0006] Obtain a first histogram with a time bin length of a first preset duration, and perform a rough measurement operation on the first histogram to obtain a rough measurement flight time;

[0007] Obtain a second histogram with a time bin length of a second preset duration, and perform a fine measurement operation on the second histogram to obtain a fine measurement flight time; the second preset duration is less than the first preset duration;

[0008] Determine the flight time of the laser pulse based on the roughly measured flight time and the finely measured flight time, and determine the distance of the external object according to the flight time;

[0009] Among them, the first histogram is obtained by controlling the time-to-digital converter (TDC) with a lower time detection accuracy, and the second histogram is obtained by controlling the TDC with a higher time detection accuracy.

[0010] The second aspect of the present application proposes a time-of-flight ranging device, including

[0011] A transmitting module configured to emit a sensing optical signal to the measurement scene for three-dimensional detection of an external object in the measurement scene;

[0012] A receiving module configured to sense the optical signal from the measurement scene and output a corresponding optical induction signal;

[0013] A processing circuit, coupled to the transmitting module and the receiving module, configured to process the optical induction signal generated corresponding to the photons received by the receiving module to obtain three-dimensional information of the external object. The processing circuit includes:

[0014] A timing module configured to accumulate and count according to the corresponding optical induction signal output when the receiving module senses the optical signal within the corresponding time bin;

[0015] A statistical module configured to statistically count the optical induction signal counts accumulated in each corresponding time bin for multiple sensing within one detection frame to generate a corresponding photon count histogram;

[0016] A flight time determination module configured to determine the flight time of the sensing optical signal reflected by the external object according to the time stamp of the time bin corresponding to the signal peak in the photon count histogram; and

[0017] A distance determination module configured to obtain the distance information of the external object according to the determined flight time;

[0018] Wherein, the timing module includes a first timing unit and a second timing unit. The first timing unit is configured to count the light induction signals with a time bin length of a first preset duration. The second timing unit is configured to count the light induction signals with a time bin length of a second preset duration. The first preset duration is less than the second preset duration. The time-of-flight determination module includes a time-of-flight rough measurement unit, a time-of-flight fine measurement unit, and a determination unit. The time-of-flight rough measurement unit is configured to perform a rough measurement operation on the first histogram with a time bin length of the first preset duration to obtain a roughly measured time of flight. The time-of-flight fine measurement unit is configured to perform a fine measurement operation on the second histogram with a time bin length of the second preset duration to obtain a finely measured time of flight. The determination unit is used to determine the time of flight of the laser pulse based on the roughly measured time of flight and the finely measured time of flight.

[0019] A third aspect of the present application provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the steps of the method described in the first aspect above are implemented.

[0020] A fourth aspect of the present application provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, the steps of the method described in the first aspect above are implemented.

[0021] Based on the time-of-flight ranging method and device described in the first and second aspects above, the present application has at least the following beneficial effects or advantages:

[0022] First, by controlling the TDC to obtain a histogram with a relatively long time bin length (i.e., the time length of one bin) with a lower time detection accuracy, and performing a rough measurement operation on the histogram to obtain a roughly measured time of flight in the time of flight. Then, control the TDC to obtain a histogram with a relatively short time bin length with a higher time detection accuracy, and perform a fine measurement operation on the histogram to find an accurate finely measured time of flight in the time of flight. Since the acquisition granularity of the histogram obtained the second time is small, the determined finely measured time of flight is more accurate. Therefore, an accurate time of flight can be obtained by using the roughly measured time of flight and the finely measured time of flight, making the ranging result more accurate. And since this solution does not require calculating and correcting the histogram, a large amount of computing resources can be saved, and the ranging efficiency can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The drawings described herein are used to provide a further understanding of the present application, and constitute a part of the present application. The schematic embodiments of the present application and their descriptions are used to explain the present application, and do not constitute an improper limitation of the present application. In the drawings:

[0024] Figure 1 Schematic diagram of the hardware structure of an electronic device shown according to an exemplary embodiment of the present application;

[0025] Figure 2 Schematic diagram of the hardware structure of a time-of-flight ranging device shown according to an exemplary embodiment of the present application;

[0026] Figure 3 Schematic diagram of a statistical histogram shown according to an exemplary embodiment of the present application;

[0027] Figure 4 is Figure 2 Schematic diagram of the structure of an exemplary embodiment of the timing module described in;

[0028] Figure 5 Schematic diagram of the photon count histogram of the time-of-flight ranging device in the case of pile-up effect according to an exemplary embodiment of the present application;

[0029] Figure 6 Flowchart of the embodiment of a time-of-flight ranging method shown according to an exemplary embodiment of the present application;

[0030] Figure 7 According to the present application Figure 6 The first histogram for coarse measurement operation shown in the illustrated embodiment;

[0031] Figure 8 According to the present application Figure 6 Schematic diagram of the structure of the first network model shown in the illustrated embodiment;

[0032] Figure 9 According to the present application Figure 7 Schematic diagram of the structure of a residual block shown in the illustrated embodiment;

[0033] Figure 10 According to the present application Figure 8 Schematic diagram of the structure of the attention layer shown in the illustrated embodiment;

[0034] Figure 11 According to the present application Figure 6 The second histogram for fine measurement operation shown in the illustrated embodiment;

[0035] Figure 12 Schematic diagram of the structure of a time-of-flight ranging device shown according to an exemplary embodiment of the present application;

[0036] Figure 13 Schematic diagram of the structure of another time-of-flight ranging device shown according to an exemplary embodiment of the present application;

[0037] Figure 14This is a schematic structural diagram of another time-of-flight ranging device shown according to an exemplary embodiment of the present application;

[0038] Figure 15 This is a schematic structural diagram of a storage medium shown according to an exemplary embodiment of the present application. Detailed implementation manners

[0039] Here, the exemplary embodiments will be described in detail, and the examples are shown in the drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementation manners described in the following exemplary embodiments do not represent all implementation manners consistent with the present application. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0040] The terms used in the present application are only for the purpose of describing specific embodiments and are not intended to limit the present application. The singular forms "a", "the", and "said" used in the present application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0041] It should be understood that although the terms first, second, third, etc. may be used in the present application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "when" or "while" or "in response to determining".

[0042] As Figure 1 shown, the electronic device 1 described in the embodiment of the present application has a time-of-flight ranging function and can be used for three-dimensional information sensing or spatial distance measurement. For example, it can specifically be used for face recognition, gesture recognition, posture or motion recognition, autonomous driving, machine vision, object recognition, scene modeling, augmented reality (AR) / virtual reality (VR), ranging, proximity sensing, simultaneous localization and mapping (SLAM), or 3D mapping, etc. The electronic device 1 may include devices with a three-dimensional information sensing function requirement such as smartphones, tablets, computers, laptops, desktop computers, smart wearable devices, smart door locks, in-vehicle electronic devices, medical devices, and aviation devices.

[0043] The electronic device 1 can be a device based on the dToF technology. The dToF technology is a ranging method for measuring the distance of an external object in a scene based on the principle of Time-Correlated Single-Photon Counting (TCSPC). TCSPC can obtain the relevant three-dimensional information of the external object that reflects the sensed light signal by repeatedly transmitting and receiving the sensed light signal and statistically analyzing the reception time information of the sensed light signal returned by the external object.

[0044] The following describes an exemplary structure of the electronic device:

[0045] As Figure 1 shown, the electronic device 1 may include a processor 101 and a memory 102. The processor 101 is coupled to the memory 102. The processor 101 can be used to control the operation of the electronic device 1, such as a Central Processing Unit (CPU). The processor 101 can be an integrated circuit chip with signal processing capabilities, such as: a general-purpose processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an Image Signal Processor (ISP), a Field Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc.

[0046] The memory 102 is used to store computer programs and data generated during the operation. It can be a Random Access Memory (RAM), a Read-Only Memory (ROM), or other types of storage devices. Specifically, the memory 102 may include one or more computer-readable storage media, and the computer-readable storage media can be non-transitory. In some embodiments, the computer-readable storage media in the memory 102 is used to store at least one program code. The computer program stored in the memory 102 can be executed by the processor 101, and thus can control the operation of the electronic device 1 to implement related operations and functions.

[0047] In some embodiments, the electronic device 1 may further include a peripheral device interface 103 and peripheral devices 104. The processor 101, the memory 102, and the peripheral device interface 103 may be connected through a bus or signal lines. Each of the peripheral devices 104 may be connected to the peripheral device interface 103 through a bus, signal lines, or a circuit board. Specifically, the peripheral device 104 may include any one or more of a radio frequency circuit, a display screen, an audio circuit, a power supply, and a time-of-flight ranging device 20.

[0048] The peripheral device interface 103 may be used to connect the peripheral devices 104 to the processor 101 and the memory 102. Optionally, in some embodiments, the processor 101, the memory 102, and the peripheral device interface 103 may be integrated on the same chip or circuit board. Optionally, in some other embodiments, any one or two of the processor 101, the memory 102, and the peripheral device interface 103 may be implemented on a separate chip or circuit board, and the present application does not make specific limitations thereto.

[0049] As Figure 1 shown, in some embodiments, the electronic device 1 may include a time-of-flight ranging device 20, and the time-of-flight ranging device 20 may be coupled to the processor 101. Optionally, the time-of-flight ranging device 20 may implement a dToF function to detect an external object in a measurement scene to obtain three-dimensional information of the external object. The electronic device 1 is configured to implement corresponding functions according to the three-dimensional information of the external object obtained by the time-of-flight ranging device 20.

[0050] As Figure 2As shown, the time-of-flight ranging device 20 includes a transmitting module 201, a receiving module 202, and a processing circuit 203. The processing circuit 203 can be coupled to the processor 101 of the electronic device 1, and the transmitting module 201 and the receiving module 202 are respectively coupled to the processing circuit 203. The transmitting module 201 is configured to emit a sensing optical signal to the measurement scene to perform three-dimensional detection on an external object 30 in the measurement scene. A part of the sensing optical signal will be reflected by the external object 30 in the measurement scene and return. The reflected sensing optical signal carries the three-dimensional information of the external object 30, and a part of the reflected sensing optical signal can be sensed by the receiving module 202 to obtain the three-dimensional information of the external object 30. The receiving module 202 is configured to sense the optical signal from the measurement scene and output a corresponding optical induction signal. By analyzing the optical induction signal, the three-dimensional information detection of the external object 30 in the measurement scene can be realized. The three-dimensional information can be the distance information of the external object 30 and / or the depth information of the surface of the external object 30. It can be understood that the optical signal sensed by the receiving module 202 can be photons, for example, including photons of the sensing optical signal reflected by the external object 30 in the measurement scene and photons of the ambient light of the measurement scene. The receiving module 202 uses a SPAD as a photosensor to sense the photons returned from the measurement scene and output a corresponding optical induction signal.

[0051] Optionally, the sensing optical signal can be a laser pulse with a preset frequency. The transmitting module 201 periodically emits N laser pulses as the sensing optical signal at the preset frequency within one detection frame, that is, one detection frame can include N emission cycles of the sensing optical signal. The preset frequency can be, for example, 10 Hz - 100 MHz, and further can be 1000 Hz - 10 MHz. The present application does not make specific limitations on this.

[0052] Optionally, the sensing optical signal can be visible light, infrared light, or near-infrared light, and the wavelength range is, for example, 390 nanometers (nm) - 780 nm, 700 nm - 1400 nm, 800 nm - 1000 nm.

[0053] Optionally, the receiving module 202 may also sense the reflected optical signal through other optical sensors, such as a Charge-Coupled Device (CCD), a Complementary Metal Oxide Semiconductor (CMOS), an Avalanche Photon Diode (APD), a Silicon Photomultiplier (SiPM) with multiple SPADs connected in parallel, and / or other suitable optical sensors. Optionally, multiple optical sensors may be arranged in a regular array or in an irregular random manner.

[0054] Optionally, the receiving module 202 has a detection period corresponding to the emission period. For example, the optical sensor of the receiving module 202 has a start time and an end time that are consistent with the emission period. Each time the transmitting module 201 emits a light signal for sensing, the optical sensor of the receiving module 202 starts to sense the photons returned from the measurement scene.

[0055] The processing circuit 203 is configured to process the optical induction signal generated corresponding to the photons received by the receiving module 202 to obtain the three-dimensional information of the external object 30. In some embodiments, the processing circuit 203 may include a timing module 2031, a statistical module 2032, a time-of-flight determination module 2033, and a distance determination module 2034.

[0056] Among them, the timing module 2031 is configured to determine the time when the receiving module 202 senses the optical signal and outputs the corresponding optical induction signal, and accumulate the count in the corresponding time bin accordingly. The time bin is the time unit for the timing module 2031 to record the generation time of the optical induction signal, which can reflect the accuracy of the timing module in recording the time of the optical induction signal. The finer the time bin, the higher the accuracy of the recorded time.

[0057] Optionally, the timing module 2031 may include a Time-to-Digital Converter (TDC) and a counting memory, and the counting memory has a counting storage space allocated correspondingly according to time binning. Each time the receiving module 202 senses a photon, it outputs a corresponding optical induction signal. The timing module 2031 detects the generation time of the optical induction signal through the TDC and accumulatively increments by one in the counting storage space of the corresponding time bin accordingly. For example, within a detection frame of the time-of-flight ranging device, the transmitting module 201 may repeatedly transmit the sensing optical signal N times, and the receiving module 202 correspondingly performs N times of optical sensing. Specifically, each time of sensing may sense photons or may not sense photons, but each time a photon is sensed, the timing module 2031 accumulatively increments by one in the counting storage space of the corresponding time bin according to the time when the photon is sensed (i.e., the generation time of the optical induction signal). It should be understood that the length of a single time bin can be adjusted correspondingly by changing the time detection accuracy of the TDC. For example, the length of a single time bin is the shortest time unit that can be detected by the current time detection accuracy of the TDC. Thus, the timing module 2031 can divide the entire time-of-flight detection range into a corresponding number of time bins according to the time detection accuracy of the TDC to count the time distribution of the optical induction signals. It should be understood that the maximum time-of-flight value of the time-of-flight detection range is at least greater than the time-of-flight required for the farthest detection distance of the photon round-trip ranging device to ensure that the photons returned from the farthest detection distance can be counted.

[0058] The statistical module 2032 may be configured to count the optical induction signal counts accumulated in each corresponding time bin for N times of sensing in a detection frame to generate a corresponding photon count histogram, as Figure 3 shown. Among them, the abscissa of the photon count histogram represents the timestamps of each corresponding time bin, and the count can be performed in the time bin with the corresponding timestamp according to the output time of the optical induction signal detected by the TDC. The ordinate of the photon count histogram represents the count values of the optical induction signals accumulated in each corresponding time bin. Optionally, the statistical module 2032 may be a histogram circuit.

[0059] During the sensing process, a large number of photons of ambient light are also received and sensed by the receiving module 202, generating corresponding optical induction signal counts. Since the optical induction signal counts caused by ambient light are random, the probability that these photons of ambient light are sensed and leave counts in each time bin tends to be the same, constituting the noise background (Noise Level) of the measurement scenario. In a measurement scenario with stronger ambient light, the average level of the measured noise background is relatively high, and in a measurement scenario with weaker ambient light, the average level of the measured noise background is relatively low. On this basis, the optical induction signal counts corresponding to the sensed optical signals reflected from external objects within the measurement scenario are superimposed on the noise background, making the optical induction signal counts in the time bin corresponding to the sensing time of the sensed optical signal significantly higher than those in other time bins, thereby forming a prominent signal peak in the photon counting histogram. Thus, the time-of-flight determination module 2033 can obtain the time of flight of the relevant sensed optical signal reflected by the external object and sensed by the receiving module 202 based on the time difference between the time stamp t1 of the time bin corresponding to the peak value of the signal peak and the emission time t0 of the relevant sensed optical signal that generates the signal peak. The distance determination module 2034 can be configured to obtain the distance information between the external object 30 that reflects the relevant sensed optical signal and the time-of-flight ranging device 20 based on the time of flight of the relevant sensed optical signal determined from the photon counting histogram, for example: the line distance between the external object and the time-of-flight ranging device in the measurement scenario.

[0060] As Figure 4 shown, in some embodiments, the timing module 2031 further includes a first timing unit 610 and a second timing unit 620. The first timing unit 610 is configured to count and statistically analyze the time when the receiving module 202 senses an optical signal and outputs an optical induction signal with a time bin length of a first preset duration. The second timing unit 620 is configured to count and statistically analyze the time when the receiving module 202 senses an optical signal and outputs an optical induction signal with a time bin length of a second preset duration. Wherein, the second preset duration is less than the first preset duration, that is, the first timing unit 610 controls the TDC to record the optical signal sensing time of the receiving module 202 with a lower time detection accuracy, and the second timing unit 620 controls the TDC to record the optical signal sensing time of the receiving module 202 with a lower time detection accuracy.

[0061] Correspondingly, the statistical module 2032 obtains a first histogram with a time bin length of the first preset duration based on the counting and statistical analysis of the optical induction signal sensing time by the first timing unit 610. The statistical module 2032 obtains a second histogram with a time bin length of the second preset duration based on the counting and statistical analysis of the optical induction signal sensing time by the second timing unit 620.

[0062] It should be understood that the emission module 201 and the reception module 202 are arranged side by side adjacent to each other. The light-emitting surface of the emission module 201 and the light-incident surface of the reception module 202 both face the same side of the time-of-flight ranging device. The value range of the distance between the emission module 201 and the reception module 202 can be, for example, from 2 millimeters (mm) to 20 mm. Since the emission module 201 and the reception module 202 are relatively close to each other, although the emission path of the sensing light signal from the emission module 201 to the external object 30 and the return path after reflection from the external object to the reception module 202 are not exactly equal, both are much larger than the distance between the emission module 201 and the reception module 202 and can be regarded as approximately equal. Thus, the distance information between the external object 30 and the time-of-flight ranging device 20 can be calculated based on the product of half of the flight time t of the sensing light signal reflected by the external object 30 and the speed of light c.

[0063] If the reception module uses a SPAD as the photosensor device, there is a certain probability that the SPAD operating in the Geiger mode will be triggered by photons to generate an avalanche effect and output a photoinduction signal. After the avalanche, the SPAD needs to be reset to restore to the Geiger mode capable of sensing photons again, and the time period from the start of the avalanche of the SPAD to its restoration to the Geiger mode again is the dead time when photons cannot be sensed. Thus, in the case where the power of the sensing light signal emitted by the emission module 201 is too high, the reflectivity of the external object is relatively high, the ambient light intensity is relatively large, and / or the distance of the measured external object is relatively close, etc., in the relatively early period within a detection cycle, a large number of photons of the sensing light signal reflected by the external object will avalanche most of the SPADs of the reception module 202 and cannot sense and count the subsequent reflected sensing light signal photons within this detection cycle, resulting in the pile-up effect (Pile Up Effect) where the peak position of the signal peak in the obtained photon count histogram moves forward, that is, the finally obtained flight time is on the small side, and thus the measured distance value of the external object is also smaller than the actual value.

[0064] As Figure 5 shown, the dashed part in the photon count histogram is the photon count distribution of the signal peak without the pile-up effect, and the solid shaded part is the photon count distribution of the signal peak with the pile-up effect. After the pile-up effect occurs, the position of the peak time bin of the signal peak moves forward, resulting in the measured peak time bin of the signal peak being smaller than the actual peak time bin of the signal peak, and the flight time value of the sensing light signal obtained in this detection frame also moves forward correspondingly and is on the small side.

[0065] Therefore, before performing statistical analysis on the histogram, it is necessary to correct the count error for different time bins on the histogram to correct the generated stacking effect.

[0066] However, correcting the count of each time bin requires buffering the counts of several previous time bins for calculation, resulting in a large consumption of computing resources for count correction. Moreover, in most cases, the correction effect is not good, and even error correction cannot be achieved. For example, due to the read and write speed limitations of the digital circuit in the TDC, some photon counts are randomly lost, leading to a poor correction effect.

[0067] Based on this, the present application proposes a time-of-flight ranging method, that is, by controlling the time detection accuracy of the TDC, first obtaining a first histogram with a time bin length of a first preset duration at a lower time detection accuracy to perform a rough measurement operation based on the first histogram to obtain a rough time of flight, and then controlling the TDC to obtain a second histogram with a time bin length of a second preset duration at a higher time detection accuracy to perform a fine measurement operation based on the second histogram to obtain a fine time of flight. The second preset duration is less than the first preset duration. Finally, based on the rough time of flight and the fine time of flight, the time of flight of the laser pulse reflected back and sensed is determined, and the distance of the external object is determined according to the time of flight.

[0068] The technical effects that can be achieved based on the above description are as follows:

[0069] First, by controlling the TDC to obtain a histogram with a longer time bin length (i.e., the time length of one bin) at a lower time detection accuracy and performing a rough measurement operation on the histogram to obtain the rough time of flight in the time of flight, and then controlling the TDC to obtain a histogram with a shorter time bin length at a higher time detection accuracy and performing a fine measurement operation on the histogram to find the accurate fine time of flight in the time of flight. Since the acquisition granularity of the histogram obtained the second time is small, the determined fine time of flight is more accurate. Thus, the accurate time of flight can be obtained by using the rough time of flight and the fine time of flight, making the ranging result more accurate. And since this solution does not require calculation and correction of the histogram, a large amount of computing resources can be saved, and the ranging efficiency can be improved.

[0070] To enable those skilled in the art to better understand the solution of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present application.

[0071] Figure 6 It is a flowchart of an embodiment of a time-of-flight ranging method shown according to an exemplary embodiment of the present application. In the embodiment of the present application, the first histogram is obtained by controlling the TDC at a lower time detection accuracy, and the second histogram is obtained by controlling the TDC at a higher time detection accuracy. Refer to Figure 6 As shown, the ranging method includes the following steps:

[0072] Step 401: Obtain a first histogram with a time bin length of a first preset duration.

[0073] Among them, the time range of the first histogram is determined by the farthest distance that the laser pulse can measure (i.e., the longest flight time of the laser pulse). Therefore, the first histogram contains the photon counts of the reflected pulses in multiple time bins in the time series, and the length of a single time bin is the first preset duration.

[0074] See Figure 6 As shown, it is the first histogram for rough measurement operation. Among them, the time length of a single time bin (i.e., a single long bar in the figure) is 1.6 nanoseconds. There are a total of 25 time bins. From left to right, the time bin numbers are 1, 2, 3, 4... 25 in sequence, and the total time range is 25 * 1.6 = 40 nanoseconds. That is to say, the longest flight time of the laser pulse is 40 nanoseconds.

[0075] It should be noted that the longer the length of a single time bin is set, the higher the histogram acquisition efficiency is, but the lower the calculation accuracy of the flight time is. Therefore, a suitable time bin length needs to be set to acquire the histogram.

[0076] Step 402: Perform a rough measurement operation on the first histogram to obtain a rough measurement flight time.

[0077] Among them, the rough measurement operation refers to finding the bin number in the first histogram that may represent the flight time of the laser pulse.

[0078] In a possible implementation manner, since there is a large amount of noise in the first histogram, in order to avoid counting error correction required by the traditional flight time acquisition algorithm, this application uses a method based on a deep neural network to obtain the flight time, which not only saves computing resources but also has better effects.

[0079] The specific rough measurement process is as follows: Input the first histogram into the first network model, so that the first network model predicts the flight time of the first histogram and outputs the first bin number representing the flight time, thereby determining the rough measurement flight time by using the first bin number and the first preset duration.

[0080] In specific implementation, since the length of a single time bin is relatively long during the rough measurement process, a laser pulse is easily recorded at the junction of two time bins. Therefore, theoretically, there may be more than one time bin in the histogram representing the rough measurement flight time.

[0081] Based on this, in this embodiment, the first network model can output two adjacent first bin numbers. One is the bin number with the highest probability, and the other is the one with a relatively high probability among the two bin numbers adjacent to the bin number with the highest probability.

[0082] For the determination process of the rough measurement of the flight time, specifically, it is determined by using the bin number before the first bin number and the first preset duration. The time corresponding to the previous bin number belongs to the previous period of the total flight time. Assuming that the first bin numbers are 15 and 16 respectively, and the bin time length is 1.6 nanoseconds, then the rough measurement of the flight time T1 = 14 * 1.6 = 22.4 nanoseconds.

[0083] The structure and processing flow of the first network model will be described in detail below.

[0084] See Figure 7 In the shown model structure, the processing flow for the first histogram is as follows: The first histogram is converted into multi-channel data through a convolutional layer, and the multi-channel data is denoised multiple times through a residual network to obtain the denoised multi-channel data. Finally, the output layer performs global average pooling on the denoised multi-channel data in the channel dimension to obtain single-channel one-dimensional data, and the first bin number is output according to the single-channel one-dimensional data.

[0085] Among them, since the input histogram data is single-channel one-dimensional data, in order to better perform feature extraction, the histogram data is changed into multi-channel one-dimensional data through a convolutional layer. Specifically, a 3*1*1 convolutional kernel can be used in the convolutional layer to change the single-channel one-dimensional data into 3-channel one-dimensional data. In a deep network, as the number of layers increases, the problem of gradient degradation becomes more and more serious. Each residual block in the residual network is implemented in the form of a skip connection, that is, the input of the residual block is directly added to the output, which can alleviate the problem of gradient degradation of the network. Therefore, in this embodiment, the residual network is used for multiple denoising to facilitate accurately obtaining the bin positions that may represent the flight time.

[0086] Specifically, the two bin numbers corresponding to the higher value among the maximum value and the two values adjacent to the maximum value are selected and output from the single-channel one-dimensional data.

[0087] For example, as the above Figure 7 shown first histogram includes the counts of 25 time bins. Therefore, the 1*1*25 single-channel one-dimensional data obtained after the global average pooling operation by the output layer, and the 16th value in the 1*1*25 single-channel one-dimensional data is the largest. Among the 15th value and the 17th value adjacent to the 16th value, the 15th value is the largest. Therefore, the output layer outputs the bin numbers 15 and 16 corresponding to the 15th value and the 16th value, indicating that the flight time may exist in these two bins.

[0088] By observing Figure 7It can be seen that, in fact, the statistical count of the 3rd time bin is the highest. However, after being predicted by the network model, the output is the sequence number of the 15th time bin and the sequence number of the 16th time bin adjacent to it. This is because the peak position of the signal peak caused by the pile-up effect moves forward. That is, in the earlier period within the detection cycle, a large number of photons of the sensed optical signal reflected by external objects avalanche most of the SPADs and cannot sense and count the subsequent reflected sensed optical signal photons within this detection cycle. Therefore, by inputting the histogram into the trained network model for prediction, the misjudgment problem of the signal peak caused by the pile-up effect can be avoided.

[0089] In a specific optional embodiment, referring to Figure 8 as shown, the residual network includes four serially connected residual blocks. The output of the previous residual block is used as the input of the next residual block. Thus, four serially connected residual blocks can perform four serial denoising operations on multi-channel data, that is, each residual block performs one denoising process on the input multi-channel data.

[0090] Furthermore, referring to Figure 9 the structure of a single residual block as shown, which includes an attention layer, a soft threshold layer, and a splicing layer. The specific denoising process is as follows: the threshold on each channel is obtained through the attention layer based on the input multi-channel data. For each channel data in the input multi-channel data, the soft threshold denoising is performed on this channel data using the threshold on this channel. Finally, the input multi-channel data and the denoised data of each channel are spliced according to the channel dimension and output through the splicing layer.

[0091] By introducing an attention mechanism in each residual block to find the noise threshold on each channel and using the soft threshold operation method to set the noise in each channel data to zero, since the signal-to-noise ratio of time-of-flight data varies greatly in different environments, the soft threshold denoising has higher robustness and better effect compared with the fixed threshold denoising.

[0092] Among them, the soft threshold denoising formula is as follows:

[0093]

[0094] It can be seen from the above formula that the soft threshold denoising deletes the features with absolute value less than the threshold and shrinks the features with absolute value greater than the threshold towards zero, which belongs to a non-linear transformation. The derivative of its output y with respect to the input x is either 1 or 0, having the same property as the ReLu activation function. Therefore, the soft threshold operation as the activation function of the network model can avoid the risks of gradient dispersion and gradient explosion in the network model.

[0095] Regarding the process of obtaining the threshold on each channel through the attention layer, referring to Figure 10As shown, specifically, absolute value operation and global average pooling operation are performed on the input multi-channel data C*W*H to change the size of each channel of data to 1*1. After activation operation on the 1*1 data on each channel, dot product is performed with the 1*1 data on each channel to obtain the threshold value on each channel.

[0096] Step 403: Obtain a second histogram with a time bin length of a second preset duration, where the second preset duration is less than the first preset duration.

[0097] Among them, the second histogram also includes the photon counts of the reflected pulses in multiple time bins on the time series. Only the length of a single time bin is the second preset duration. In order to obtain the accurate flight time in the latter stage, the time bin length at this time is much shorter than that in the rough measurement, that is, the acquisition granularity of the second histogram becomes smaller. Therefore, the second preset duration can be the shortest time unit that can be detected by the time detection accuracy of the TDC.

[0098] It should be noted that the second histogram is the data for further fine measurement based on the rough measurement result. Therefore, the time range of the second histogram is related to the total length of the time bins in the rough measurement result. Therefore, when obtaining the second histogram, specifically, the acquisition time range of the histogram is determined according to the number of the first bin numbers and the first preset duration, and then the second histogram with a time bin length of the second preset duration and a time series of this acquisition time range is obtained.

[0099] See Figure 11 As shown, it is the second histogram for the fine measurement operation. Since there are 2 output first bin numbers in the above example and the length of each time bin is 1.6 nanoseconds, the time range of the second histogram is 1.6 * 2 = 3.2 nanoseconds, and the length value of each time bin is 0.1 nanosecond. Therefore, the second histogram has a total of 32 time bins. From left to right, the bin numbers are 1, 2, 3, 4... 32 in sequence. That is to say, during the fine measurement process, a finer fine measurement flight time needs to be selected from 0 - 3.2 ns.

[0100] Step 404: Perform a fine measurement operation on the second histogram to obtain the fine measurement flight time.

[0101] It should be noted that since there may still be a large amount of noise in the second histogram and it is not suitable to use the traditional flight time acquisition algorithm, the present application still uses the method based on the deep neural network to obtain the flight time, which not only saves computing resources but also has better effects.

[0102] The specific detailed measurement process is as follows: By inputting the second histogram into the second network model, the second network model performs time-of-flight prediction on the second histogram and outputs the second bin number representing the time of flight. Then, the detailed measurement time of flight is determined using the second bin number and the second preset duration.

[0103] Among them, during the detailed measurement process, the length of a single time bin is relatively short and is already close to the hardware limit of the TDC. Therefore, the time bin representing the detailed measurement time of flight is one, that is, the number of the second bin numbers is only 1.

[0104] Regarding the determination process of the detailed measurement time of flight, specifically, it is determined using the second bin number and the second preset duration. Assume the second bin number is 15 and the time bin length is 0.1 nanosecond. Then the detailed measurement time of flight T2 = 15 * 0.1 = 1.5 nanoseconds.

[0105] It should be further noted that the principle of using the second network model to perform time-of-flight prediction on the second histogram is the same as that of the first network model used in the above-mentioned rough measurement process. Therefore, the network structure of the second network model is the same as that of the first network model, but different network parameters need to be trained for the two network models respectively.

[0106] Therefore, for the prediction process of the second network model on the second histogram, please refer to the processing process of the above-mentioned first network model, and this application will not elaborate here.

[0107] Step 405: Determine the time of flight of the laser pulse based on the rough measurement time of flight and the detailed measurement time of flight, and determine the distance of the external object according to the time of flight.

[0108] Specifically, based on the rough measurement time of flight T1 and the detailed measurement time of flight T2 described above, the time of flight T of the laser pulse = T1 + T2.

[0109] Those skilled in the art can understand that the process of calculating the distance according to the time of flight can be implemented using related technologies, and this application does not specifically limit it. For example, the distance of the external object can be calculated according to the time of flight and the speed of light.

[0110] So far, the above Figure 6The ranging process shown first controls the TDC to obtain a histogram with a relatively long time bin length (i.e., the time length of one bin) with a lower time detection accuracy, and performs a rough measurement operation on the histogram to obtain a rough flight time in the flight time. Then, the TDC is controlled to obtain a histogram with a relatively short time bin length with a higher time detection accuracy, and a fine measurement operation is performed on the histogram to find the accurate fine measurement flight time in the flight time. Since the acquisition granularity of the histogram obtained the second time is small, the determined fine measurement flight time is more accurate. Thus, the accurate flight time can be obtained using the rough measurement flight time and the fine measurement flight time, making the ranging result more accurate. And since this solution does not require calculating and correcting the histogram, a large amount of computing resources can be saved, and the ranging efficiency can be improved.

[0111] Corresponding to the embodiments of the foregoing time-of-flight ranging method, the flight time determination module 2033 in the processing circuit 203 of the time-of-flight ranging device 20 provided in the embodiments of the present application further includes corresponding functional units.

[0112] Figure 12 It is a schematic diagram of the functional units of a flight time determination module shown according to an exemplary embodiment of the present application. The flight time determination module 2033 is used to execute the time-of-flight ranging method provided in any of the foregoing embodiments. As Figure 12 shown, the flight time determination module 2033 includes a rough flight time unit 710, a fine flight time unit 720, and a determination unit 730.

[0113] The rough flight time unit 710 is configured to perform a rough measurement operation on the first histogram with a time bin length of a first preset duration to obtain a rough flight time. The rough measurement operation refers to the rough flight time unit 710 determining the rough flight time according to the time stamp of the time bin corresponding to the peak value of the signal peak in the first histogram.

[0114] The fine flight time unit 720 is configured to perform a fine measurement operation on a second histogram with a time bin length of a second preset duration to obtain a fine flight time; the second preset duration is less than the first preset duration. The rough measurement operation refers to the rough flight time unit 710 determining the rough flight time according to the time stamp of the time bin corresponding to the peak value of the signal peak in the second histogram.

[0115] The determination unit 730 is configured to determine the flight time reflected back by the external object 30 based on the rough flight time and the fine flight time.

[0116] Among them, the first histogram is obtained by controlling the time-to-digital converter TDC with a lower time detection accuracy, and the second histogram is obtained by controlling the TDC with a higher time detection accuracy.

[0117] In an alternative implementation, as Figure 13 shown, the coarse time-of-flight measurement unit 710 includes:

[0118] A first model prediction unit 7101, configured to input the first histogram into a first network model, so that the first network model performs time-of-flight prediction on the first histogram and outputs a first bin number representing the time of flight;

[0119] A coarse measurement time determination unit 7102, configured to determine a coarse measurement time of flight by using the first bin number and the first preset duration.

[0120] In an alternative implementation, as Figure 14 shown, the fine time-of-flight measurement unit 720 includes:

[0121] A range determination unit 7201, configured to determine an acquisition time range of a second histogram according to the number of the first bin numbers and the first preset duration;

[0122] An acquisition unit 7202, configured to acquire a second histogram with a time bin length of a second preset duration and a time series of the acquisition time range.

[0123] In an alternative implementation, the first network model includes a convolutional layer, a residual network, and an output layer; the coarse time-of-flight measurement unit 710 is specifically configured to, in the process of performing time-of-flight prediction on the first histogram by the first network model and outputting a first bin number representing the time of flight, convert the first histogram into multi-channel data through the convolutional layer; perform multiple denoising processes on the multi-channel data through the residual network to obtain denoised multi-channel data; perform global average pooling operation on the denoised multi-channel data in the channel dimension through the output layer to obtain single-channel one-dimensional data, and output a first bin number according to the single-channel one-dimensional data.

[0124] In an alternative implementation, the residual network includes four cascaded residual blocks, and the output of the previous residual block is used as the input of the next residual block; the coarse time-of-flight measurement unit 710 is specifically configured to, in the process of performing multiple denoising processes on the multi-channel data through the residual network, perform four denoising processes on the multi-channel data through the four cascaded residual blocks; wherein, each residual block performs one denoising process on the input multi-channel data.

[0125] In an alternative implementation, the residual block includes an attention layer, a soft threshold layer, and a splicing layer; the ToF rough measurement unit 710 is specifically configured to, in the process of performing denoising processing on the input multi-channel data by each residual block once, obtain the threshold on each channel based on the input multi-channel data through the attention layer; perform soft threshold denoising on each channel data in the input multi-channel data through the soft threshold layer by using the threshold on this channel; and splice the input multi-channel data and the denoised data of each channel according to the channel dimension and output.

[0126] In an alternative implementation, the ToF rough measurement unit 710 is specifically configured to, in the process of obtaining the threshold on each channel based on the input multi-channel data through the attention layer, perform an absolute value operation and a global average pooling operation on the input multi-channel data through the attention layer to change the size of each channel data to 1*1, perform an activation operation on the 1*1 data on each channel, and then perform a dot product with the 1*1 data on each channel to obtain the threshold on each channel.

[0127] In an alternative implementation, as Figure 13 shown, the ToF fine measurement unit 720 further includes:

[0128] A second model prediction unit 7203, configured to input the second histogram into a second network model, so that the second network model performs ToF prediction on the second histogram and outputs a second bin number representing the ToF;

[0129] A fine measurement time determination unit 7204, configured to determine the fine measurement ToF by using the second bin number and the second preset duration.

[0130] The implementation processes of the functions and roles of each unit in the above device are specifically described in detail in the implementation processes of the corresponding steps in the above method, and will not be elaborated here.

[0131] For the device embodiment, since it basically corresponds to the method embodiment, the relevant parts can be referred to the partial description of the method embodiment. The device embodiments described above are only illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of the present application. Those of ordinary skill in the art can understand and implement it without creative efforts.

[0132] An embodiment of the present application also provides a computer-readable storage medium corresponding to the time-of-flight ranging method provided in the foregoing embodiment. Please refer to Figure 15 As shown, the computer-readable storage medium shown is an optical disc 30, on which a computer program (i.e., a program product) is stored. When the computer program is run by a processor, it will execute the time-of-flight ranging method provided in any of the foregoing embodiments.

[0133] It should be noted that examples of the computer-readable storage medium may also include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory, or other optical and magnetic storage media, which will not be elaborated here one by one.

[0134] The computer-readable storage medium provided in the above embodiment of the present application and the time-of-flight ranging method provided in the embodiment of the present application are based on the same inventive concept and have the same beneficial effects as the method adopted, run, or implemented by the application program stored therein.

[0135] Those skilled in the art will readily conceive of other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any variations, uses, or adaptations of the present application, which follow the general principles of the present application and include common general knowledge or conventional technical means in the technical field not disclosed in the present application. The specification and embodiments are only regarded as exemplary, and the true scope and spirit of the present application are pointed out by the following claims.

[0136] It should also be noted that the term "comprising", "including", or any other variation thereof is intended to cover a non-exclusive inclusion, such that a process, method, commodity, or device comprising a series of elements not only includes those elements but also includes other elements not expressly listed, or further includes elements inherent to such process, method, commodity, or device. Without further limitation, an element defined by the statement "comprising one..." does not exclude the presence of additional identical elements in the process, method, commodity, or device comprising the element.

[0137] The above are only the preferred embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present application shall be included within the scope of protection of the present application.

Claims

1. A time-of-flight ranging method, characterized in that, The method includes: Obtaining a first histogram with a time bin length of a first preset duration, and performing a rough measurement operation on the first histogram to obtain a rough measurement flight time; Obtaining a second histogram with a time bin length of a second preset duration, and performing a fine measurement operation on the second histogram to obtain a fine measurement flight time; the second preset duration is less than the first preset duration; Determining the flight time of a laser pulse based on the rough measurement flight time and the fine measurement flight time, and determining the distance of an external object according to the flight time; Wherein, the first histogram is obtained by controlling a time-to-digital converter (TDC) with a lower time detection accuracy, and the second histogram is obtained by controlling the TDC with a higher time detection accuracy; Wherein, performing a rough measurement operation on the first histogram to obtain a rough measurement flight time includes: Inputting the first histogram into a first network model, so that the first network model performs flight time prediction on the first histogram and outputs a first bin number representing the flight time; Determining the rough measurement flight time by using the first bin number and the first preset duration.

2. The time-of-flight ranging method according to claim 1, wherein Obtaining a second histogram with a time bin length of a second preset duration includes: Determining the acquisition time range of the histogram according to the number of the first bin numbers and the first preset duration; Obtaining a second histogram with a time bin length of a second preset duration and a time series within the acquisition time range.

3. The time-of-flight ranging method according to claim 1, characterized in that, The first network model includes a convolutional layer, a residual network, and an output layer; The first network model performing flight time prediction on the first histogram and outputting a first bin number representing the flight time includes: Converting the first histogram into multi-channel data through the convolutional layer; Performing multiple denoising processes on the multi-channel data through the residual network to obtain denoised multi-channel data; Performing global average pooling operation on the denoised multi-channel data in the channel dimension through the output layer to obtain single-channel one-dimensional data, and outputting the first bin number according to the single-channel one-dimensional data.

4. The time-of-flight ranging method according to claim 3, wherein The residual network includes four cascaded residual blocks, and the output of the previous residual block is used as the input of the next residual block; Performing multiple denoising processes on the multi-channel data through the residual network includes: Performing four denoising processes on the multi-channel data through the four cascaded residual blocks; Wherein, each residual block performs one denoising process on the input multi-channel data.

5. The time-of-flight ranging method according to claim 4, wherein The residual block includes an attention layer, a soft threshold layer, and a splicing layer; Each residual block performing one denoising process on the input multi-channel data includes: Obtaining a threshold on each channel based on the input multi-channel data through the attention layer; Performing soft threshold denoising on each channel data in the input multi-channel data by using the threshold on the channel through the soft threshold layer; Outputting by splicing the input multi-channel data and the denoised data of each channel in the channel dimension through the splicing layer.

6. The time-of-flight ranging method according to claim 5, wherein, Obtaining a threshold on each channel based on the input multi-channel data through the attention layer includes: Perform an absolute value operation and a global average pooling operation on the input multi-channel data through the attention layer to change the size of each channel of data to 1*1, and perform an activation operation on the 1*1 data on each channel and then perform a dot product with the 1*1 data on each channel to obtain a threshold value on each channel.

7. The time-of-flight ranging method according to claim 1, wherein, Perform a fine measurement operation on the second histogram to obtain the fine measurement flight time, including: Input the second histogram into a second network model, so that the second network model predicts the flight time of the second histogram and outputs a second bin number representing the flight time; Determine the fine measurement flight time by using the second bin number and the second preset duration.

8. A time-of-flight ranging device, characterized in that, Including: A transmitting module configured to emit a sensing optical signal to a measurement scene to perform three-dimensional detection on an external object in the measurement scene; A receiving module configured to sense an optical signal from the measurement scene and output a corresponding optical induction signal; A processing circuit coupled to the transmitting module and the receiving module, configured to process the optical induction signal generated corresponding to the photons received by the receiving module to obtain three-dimensional information of the external object. The processing circuit includes: A timing module configured to perform cumulative counting on the corresponding optical induction signal output when the receiving module senses an optical signal within a corresponding time bin; A statistical module configured to statistically count the cumulative optical induction signal counts in each corresponding time bin for multiple detections within a detection frame to generate a corresponding photon count histogram; A flight time determination module configured to determine the flight time of the sensing optical signal reflected by the external object according to the time stamp of the time bin corresponding to the signal peak in the photon count histogram; and A distance determination module configured to obtain distance information of the external object according to the determined flight time; Wherein, the timing module includes a first timing unit and a second timing unit. The first timing unit is configured to count the optical induction signal with a time bin length of a first preset duration, and the second timing unit is configured to count the optical induction signal with a time bin length of a second preset duration. The first preset duration is less than the second preset duration. The flight time determination module includes a coarse measurement unit for flight time, a fine measurement unit for flight time, and a determination unit. The coarse measurement unit for flight time is configured to perform a coarse measurement operation on a first histogram with a time bin length of a first preset duration to obtain a coarse measurement flight time. The fine measurement unit for flight time is configured to perform a fine measurement operation on a second histogram with a time bin length of a second preset duration to obtain a fine measurement flight time. The determination unit is used to determine the flight time of the laser pulse based on the coarse measurement flight time and the fine measurement flight time; Wherein, the coarse measurement unit for flight time includes: A first model prediction unit for inputting the first histogram into a first network model, so that the first network model predicts the flight time of the first histogram and outputs a first bin number representing the flight time; and A coarse measurement time determination unit for determining the coarse measurement flight time by using the first bin number and the first preset duration.

9. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the program, the steps of the method according to any one of claims 1-7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, the steps of the method according to any one of claims 1-7 are implemented.

Images