A time-delay integration type direct time-of-flight LiDAR sensor and its processing method

Abstract

This invention relates to a time-delay integration direct time-of-flight (DToF) LiDAR sensor and its processing method. Based on an area-array scanning operation, it completes quantization and readout within each short transit time, thus increasing the frame rate. The sensor's readout circuit employs a successive TDC (Time-of-Flight Difference) approach to approximate the histogram peak value, significantly reducing the amount of data generated by quantization. This reduces both the area overhead of the internal storage circuitry and the amount of data read out. Simultaneously, the system uses a coarse-quantization followed by a fine-quantization method, improving the quantization accuracy of the histogram TDC at a lower clock frequency. Benefiting from coarse quantization generating only one sign bit per stage, the TDI-DToF-LiDAR sensor exhibits a smaller data volume, higher frame rate, and smaller storage circuitry area overhead compared to traditional DToF-LiDAR sensors. Furthermore, the coarse-quantization + fine-quantization approach allows for high quantization accuracy without requiring a high-frequency clock input.

Description

Technical Field

[0001] This invention belongs to the field of lidar sensor technology, and in particular to a time-delay integration type direct time-of-flight LiDAR sensor and its processing method. Background Technology

[0002] In recent years, LiDAR (Light Detection and Ranging) sensors have been widely used in various fields such as user identification, AR / VR devices, and autonomous vehicles. Due to their larger detection range and stronger resistance to background light interference compared to Indirect Time-of-Flight (IToF) LiDAR sensors, Direct Time-of-Flight (DToF) LiDAR sensors based on Single Photon Avalanche Diodes (SPADs) have become an ideal choice for long-distance measurements in outdoor environments.

[0003] For most DToF-LiDAR sensors, when the SPAD receives a light signal, it generates an electrical signal. This signal is then processed by the pixel front-end circuitry and sent to the Time-to-Digital Converter (TDC) for quantization. The readout circuit then reads out the data containing the time information. By performing multiple measurements, a large amount of data can be obtained off-chip, allowing the actual distance to be calculated. However, this off-chip data processing method results in a large amount of data output from the readout circuit, limiting the frame rate. To reduce the data volume, histogram-based TDCs have been proposed. The traditional histogram-based TDC works as follows... Figure 1 As shown. When a SPAD pixel receives a reflected light signal, it generates a pulse to the TDC. After each measurement, the TDC increments the count value by 1 at the corresponding timestamp. The histogram is formed as follows. Figure 2 After multiple measurements, a histogram is generated over a certain time period. The time corresponding to the largest count value in the histogram is considered the actual flight time, and then only the corresponding time information needs to be read out. This on-chip histogram generation method greatly reduces the amount of output data, but it requires a large amount of storage circuitry to store the data obtained from each measurement, resulting in a large circuit area overhead and often insufficient accuracy.

[0004] To further reduce the data rate, a successive approximation-based histogram TDC was proposed, building upon the previous histogram TDC. The successive approximation-based TDC generates histograms as follows: Figure 3As shown, its operation differs from the traditional histogram TDC in that it does not require counting and storing data across multiple small timestamp ranges. Instead, it performs a multi-step successive approximation operation. The first step involves counting across two large timestamp ranges and comparing the count values ​​to determine the timestamp of the reflected light. The second step divides the refined time range into two smaller timestamps, and then counts are performed again at these smaller timestamps, further narrowing down the range of timestamps containing the reflected light. After multiple such steps, the precise time of the reflected light can be gradually approximated. Furthermore, compared to the traditional method, the successive approximation method does not require storing all measured data on-chip; instead, a portion of the data is discarded after each approximation step, eliminating the need for storage circuitry and thus reducing on-chip area consumption. Theoretically, the successive approximation TDC can infinitely improve time accuracy through multiple steps, but it is often limited by clock frequency and cannot achieve a very high level of accuracy. Moreover, because the successive approximation TDC involves multiple approximation steps, its frame rate is significantly lower than that of the traditional histogram TDC.

[0005] To further improve the accuracy and frame rate of DToF lidar, it is necessary to further improve the lidar's scanning method and readout circuit structure. Summary of the Invention

[0006] The purpose of this invention is to overcome the shortcomings of existing technologies and propose a time-delay integration type direct time-of-flight LiDAR sensor and processing method. Based on an area array scanning mode, it can complete quantization and readout once within each short transit time, thus improving the frame rate. The sensor's readout circuit uses a successive TDC approach to successively approximate the histogram peak value, which greatly reduces the amount of data generated by quantization, thereby reducing the area overhead of the internal storage circuit and significantly reducing the amount of data read out. Simultaneously, the system employs a coarse quantization plus fine quantization method, improving the quantization accuracy of the histogram TDC at a lower clock frequency.

[0007] The technical problem solved by this invention is achieved through the following technical solution:

[0008] A time-delay integration type direct time-of-flight LiDAR sensor is characterized by comprising an m×n pixel array, an m×n TDC array, a pixel control circuit, a readout control circuit, and a displacement register, wherein the m×n pixel array is respectively connected to the m×n TDC array, the pixel control circuit controls the m×n pixel array, the readout control circuit reads the quantized data of the m×n TDC array, and the output terminal of the m×n TDC array is connected to the displacement register.

[0009] Moreover, the m×n TDC array adopts the method of first m-1 stages of coarse quantization TDC plus the last stage of fine quantization TDC.

[0010] Furthermore, the coarse quantization TDC includes upper and lower counters, a storage circuit, and a window circuit, while the fine quantization TDC includes a counter.

[0011] Moreover, the sensor scans the target object in a column-oriented manner based on scanning, and each level of the TDC in the m×n TDC array is quantized during each transit time.

[0012] A processing method for a time-delay integration type direct time-of-flight LiDAR sensor, the specific implementation method of which is as follows:

[0013] The laser emits a square wave pulse, and at the same time, the pixels in the m×n pixel array scan the object starting from the first level. The corresponding TDC in the m×n TDC array quantizes the pulse of the reflected light.

[0014] For the upper and lower counters of the first-stage TDC, the counting time is the maximum range that can be measured without the need for window circuits. In the first half of this range, the upper and lower counters are switched to the upward counting mode, and the value of the counter is incremented by 1 for each pulse signal received. In the second half of this time range, the upper and lower counters are switched to the downward counting mode, and the value of the counter is decremented by 1 for each pulse received.

[0015] After multiple measurements, the sign bit generated by the upper and lower counters is finally stored in the storage circuit of the first-level TDC. Then, the window circuit of the first level generates the time range of the second-level counting by performing logical operations on the sign bit in the storage circuit.

[0016] When the second-level pixel scan detects an object, the second-level TDC counts within the time range given by the window circuit of the first level. Similarly, the counter counts upward in the upper half and downward in the lower half. The sign bit obtained from multiple measurements is stored in the storage circuit. Then, the window circuit generates the quantization time range of the third level based on this sign bit. After the quantization operation of the m-1 level TDC, the position of the reflected light is gradually approximated.

[0017] The advantages and positive effects of this invention are:

[0018] This invention is based on an area array scanning operation, which completes quantization and readout once within each short transit time, thus improving the frame rate. The sensor's readout circuit uses a successive TDC approach to successively approximate the histogram peak, which greatly reduces the amount of data generated by quantization, thereby reducing the area overhead of the internal storage circuit and significantly reducing the amount of data read out. Simultaneously, the system employs a coarse-quantization plus fine-quantization method, improving the quantization accuracy of the histogram TDC at a lower clock frequency. Thanks to the fact that each stage of coarse quantization generates only one sign bit, the TDI-DToF-LiDAR sensor achieves a smaller data volume, higher frame rate, and smaller storage circuit area overhead compared to traditional DToF-LiDAR sensors. Furthermore, the coarse-quantization + fine-quantization approach allows for high quantization accuracy without requiring a high-frequency clock input. Attached Figure Description

[0019] Figure 1 A flowchart illustrating the workflow of a conventional DToF-LiDAR sensor;

[0020] Figure 2 Generate histograms for existing traditional histogram TDCs;

[0021] Figure 3 This describes a histogram generation method for existing successive approximation histogram TDCs.

[0022] Figure 4 To develop a time-delay integration type direct time-of-flight LiDAR sensor architecture;

[0023] Figure 5 Schematic diagram of the working principle of the invention time-delay integration type direct flight time LiDAR sensor;

[0024] Figure 6 The invention provides a structural diagram of a 3-bit asynchronous + 5-bit synchronous up and down counter;

[0025] Figure 7 Diagram of a 5-bit synchronous counter for invention;

[0026] Figure 8 This is a schematic diagram of the connection relationship of the storage circuit in the invention;

[0027] Figure 9 The circuit structure diagram for the nth level window is shown. Detailed Implementation

[0028] The present invention will be further described in detail below with reference to the accompanying drawings.

[0029] A time-delay integration type direct time-of-flight LiDAR sensor, such as Figure 4As shown, it includes an m×n pixel array, an m×n TDC array, a pixel control circuit, a readout control circuit, and a shift register. The m×n pixel array is connected to the m×n TDC array. The pixel control circuit controls the m×n pixel array. The readout control circuit reads the quantized data from the m×n TDC array. The output of the m×n TDC array is connected to the shift register.

[0030] The pixel array consists of five parts: macropixels, quenching circuits, monostable circuits, OR trees, and coincidence detection circuits. Macropixels are composed of SPADs, each with a corresponding quenching circuit to ensure that the SPAD quickly ends its spontaneously maintained avalanche process and resumes Geiger mode after detecting a photon. The output of the quenching circuit is connected to the monostable circuit, which compresses the width of the output pulse from the SPAD's dead time to a short width. The coincidence detection circuit utilizes the spatiotemporal correlation of the SPADs to eliminate background noise and generate a reliable trigger signal to be passed to the TDC circuit for quantization.

[0031] The m×n TDC array employs a combination of m-1 coarse quantization TDC stages followed by a final fine quantization TDC stage. After fine quantization, the readout control circuit reads out all the quantized data. The coarse quantization TDC includes upper and lower counters, storage circuitry, and windowing circuitry, while the fine quantization TDC includes a counter (which only needs to count upwards). The sensor scans the target object in a column-wise manner, and during each transit time, each stage of the m×n TDC array undergoes quantization.

[0032] In this embodiment, a 6-level coarse quantization TDC plus a 1-level fine quantization TDC is used.

[0033] like Figure 6 As shown, taking an 8-bit up / down counter as an example, this uses a cascaded configuration of a 3-bit asynchronous counter and a 5-bit synchronous counter. The structure of the 5-bit synchronous counter is as follows: Figure 7 As shown. Both synchronous and asynchronous up and down counters consist of T flip-flops and a series of logic gates. EN is the enable signal; when EN=0, counting stops, and when EN=1, counting is enabled. SiPM is the pulse signal output by the coincidence detection circuit; when EN=1, the counter value increments (decrements) by one on each rising edge of SiPM. UP / DN are control signals that control the up and down counting of the up and down counters; they are inverted. When UP=1, the counter counts up; when DN=1, the counter counts down. Sign is the sign bit, which is the output of the most significant bit of the counter.

[0034] This invention employs an asynchronous + synchronous structure because the output of each T flip-flop in a synchronous counter performs a logical operation with the output of the preceding T flip-flop, inevitably resulting in a delay. If the signal has not yet reached the last T flip-flop before the next pulse arrives, a counting error will occur. Therefore, using an asynchronous structure can extend the time interval between two pulses.

[0035] For the storage circuit, the design uses a cascaded D flip-flop configuration, with each stage of the storage circuit connected as follows: Figure 8 As shown in the diagram. The sign bit generated after each counter stage completes its count is determined by Din. <0> Din <1> ...Din <5> The input, Memoryshift, is the shift signal. Since the first stage generation window requires 1 sign bit, the second stage requires 2, and so on, the sixth stage generation window requires 6 sign bits. The sign bit of each stage needs to be stored in the memory circuit, so each stage requires 1, 2, 3…6 D flip-flops respectively. Cascading the first flip-flop of each stage, the second flip-flop of each stage, and so on, forms the following... Figure 8 The diagram shows a chain of six shift registers. During each transit time, after the TDC counter finishes counting, it controls the Memory shift signal to shift the six shift register chains once. After five shifts, the six-bit value in the last stage of the storage circuit represents the coarse quantization time of flight. This six-bit value is output by the readout circuit as the coarse quantization time of flight. Furthermore, after each stage of the storage circuit completes its shift, the quantized value is passed to the next stage, allowing this stage to store other information during the next transit time. Thus, a different quantized value can be read out for each transit time.

[0036] For the aforementioned window circuit, the following method is adopted: Figure 9 The structure shown is as follows. Memory<0:n-1> represents the sign bit stored in the nth level of memory circuit, and CLK<0:n-1> is the additional clock signal that needs to be provided, where CLK... <0> CLK <1> ……CLK <n-1>The frequency increases in multiples of 2, and WIN is the output window. The next level of counting windows is obtained by performing an XOR operation between the sign bit in the storage circuit and different clock signals CLK, followed by a AND operation. In the final stage of coarse quantization, three sets of window circuits are needed because a window must be opened on each side of the central window.

[0037] After coarse quantization, a further fine quantization stage is performed. The fine quantization TDC consists of only three ordinary counters. Because the final window circuit of the coarse quantization stage generates three windows, the fine quantization counters will count in these three different intervals. Then, all the count values ​​from the three counters are output by the readout circuit. Finally, the fine quantization value is calculated off-chip.

[0038] Because time-delay integration direct time-of-flight LiDAR sensors are based on scanning, each stage of the time-delay integration (TDC) performs quantization within its own time range at different transit times. This means that within one transit time, the last stage of the TDC can output quantized data once. Taking a maximum measurement time of 320 ns and a transit time of 96 μs as an example, each stage of the TDC will complete at least 300 measurements within this time. Within 96 μs, each stage of the TDC can quantize different distances, and after quantization, it is passed to the next stage of the TDC, and then quantization continues for another 96 μs. This also means that within each 96 μs period, the last stage of the TDC will also complete quantization and output the data through the readout circuit. If 200 pixels are scanned per frame, the maximum frame rate will reach 50 frames per second, while traditional DToF-LiDAR frame rates often only reach 30 frames per second. This demonstrates that the sensor described in this patent can achieve a significant increase in frame rate. Regarding the output data volume, taking a single column as an example, because it uses level 6 coarse quantization, the coarse quantization data generated in each 96µs transit time is 6 bits. For fine quantization, since it is divided into three windows for counting and requires 300 measurements, the data in each of the three windows must be at least 9 bits, so fine quantization generates at least 27 bits of data. Therefore, the total data generated for a single column is only 33 bits. This significantly reduces the data volume compared to traditional methods.

[0039] A processing method for a time-delay integration type direct time-of-flight LiDAR sensor, such as... Figure 5 As shown, the specific implementation method for the processing is as follows:

[0040] The laser emits a square wave pulse, and simultaneously, pixels in an m×n pixel array scan the object starting from the first stage. The corresponding m×n TDC array quantizes the received reflected light pulses. For the upper and lower counters of the first-stage TDC, the counting time is the maximum measurable range, requiring no window circuit. In the first half of this range, the upper and lower counters switch to upward counting mode, incrementing the counter value by 1 for each received pulse signal. In the second half of this range, the upper and lower counters switch to downward counting mode, decrementing the counter value by 1 for each received pulse. After multiple measurements (counts), the sign bit generated by the upper and lower counters is stored in the storage circuit of the first-stage TDC. Then, the first-stage window circuit generates the second-stage counting time range—the counting window—by performing logical operations on the sign bit (the highest bit of the counter) in the storage circuit. When the second-level pixel scan detects an object, the second-level Time-Digital Detection (TDC) counts within the time range specified by the window circuit of the first level. Similarly, the counter counts upwards in the upper half and downwards in the lower half. The sign bits obtained from multiple measurements are stored in the storage circuit. Then, the window circuit generates the quantization time range for the third level based on this sign bit. After m-1 levels of TDC quantization, the position of the reflected light is approximated stepwise. Because each level of quantization is completed based on the result of the previous level, similar to the TDI method, it is called TDI-DToF-LiDAR. Through stepwise approximation, an m-1 bit binary code is obtained, representing the pulse position. This indicates that TDI-DToF-LiDAR has a smaller data volume than most traditional histogram TDCs.

[0041] However, since the window size is limited by the clock frequency, and the clock frequency usually cannot be very high, the successive approximation TDC can only obtain a relatively coarse result, i.e., it can only be used as coarse quantization. To solve this problem, this invention introduces fine quantization in the last stage. The fine quantization TDC consists only of ordinary counters. Let the timestamp range quantized to the last stage of coarse quantization be T. d T d The full width at half maximum (FWHM) is the same as that of the emitted laser. Fine quantization, based on the window obtained from the final coarse quantization stage (the central window), opens two windows to the left and right (window 1 and window 2). The fine quantization TDC counter then counts in these three windows respectively. At this point, the distribution of reflected light in the windows falls into two categories: first, window 2 has only background noise, while the central window and window 1 mainly distribute pulse waves; second, window 1 has only background noise, while the central window and window 2 mainly distribute pulse waves. These different scenarios lead to different formulas for calculating the total flight time. The formula for calculating the flight time in the first scenario is as follows:

[0042]

[0043] TOF total =TOF coarse -TOF fine

[0044] The calculation formula for the second case is as follows:

[0045]

[0046] TOF total =TOF coarse +TOF fine

[0047] Among them, TOF fine For finer time quantification, TOF coarse For coarse quantization, TOF total For the total flight time, T d For the timestamp range of the last stage of coarse quantization, CNT1 and CNT2 are the count values ​​of the counter in the left and right side windows of the center window, respectively. center This is the counter's count value in the center window.

[0048] By employing the aforementioned fine-grained windowing operation, the pulse position can be restored to the maximum extent, while also eliminating the influence of background light. For TDI-DToF-LiDAR, due to the scanning method, after each distance is quantized, the next distance is quantized immediately. Therefore, in the actual scanning process, the quantization of one distance and data readout are completed within each transit time. This is less time-consuming than the traditional TDC, which requires all quantizations to be included in one frame, resulting in a higher frame rate.

[0049] It should be emphasized that the embodiments described in this invention are illustrative rather than limiting. Therefore, this invention includes, but is not limited to, the embodiments described in the specific implementation. Any other implementations derived by those skilled in the art based on the technical solutions of this invention are also within the scope of protection of this invention.

Claims

1. A processing method for a time-delay integration type direct time-of-flight LiDAR sensor, characterized in that: The sensor used includes an m×n pixel array, an m×n TDC array, a pixel control circuit, a readout control circuit, and a displacement register. The m×n pixel array is connected to the m×n TDC array. The pixel control circuit controls the m×n pixel array. The readout control circuit reads the quantized data from the m×n TDC array. The output of the m×n TDC array is connected to the displacement register. The m×n TDC array adopts the method of first m-1 stages of coarse quantization TDC plus the last stage of fine quantization TDC; The coarse quantization TDC includes upper and lower counters, a storage circuit, and a window circuit, while the fine quantization TDC includes a counter. The sensor scans the target object in a column-oriented manner based on scanning. During each transit time, the TDC of each level in the m×n TDC array is quantized. The specific implementation method for processing is as follows: The laser emits a square wave pulse, and at the same time, the pixels in the m×n pixel array scan the object starting from the first level. The corresponding TDC in the m×n TDC array quantizes the pulse of the reflected light. For the upper and lower counters of the first-stage TDC, the counting time is the maximum range that can be measured without the need for window circuits. In the first half of this range, the upper and lower counters are switched to the upward counting mode, and the value of the counter is incremented by 1 for each pulse signal received. In the second half of this time range, the upper and lower counters are switched to the downward counting mode, and the value of the counter is decremented by 1 for each pulse received. After multiple measurements, the sign bit generated by the upper and lower counters is finally stored in the storage circuit of the first-level TDC. Then, the window circuit of the first level generates the time range of the second-level counting by performing logical operations on the sign bit in the storage circuit. When the second-level pixel scan detects an object, the second-level TDC counts within the time range given by the window circuit of the first level. Similarly, the counter counts upward in the upper half and downward in the lower half. The sign bit obtained from multiple measurements is stored in the storage circuit. Then, the window circuit generates the quantization time range of the third level based on this sign bit. After the quantization operation of the m-1 level TDC, the position of the reflected light is gradually approximated.

Images