Light detection and ranging device and method of operation thereof

Abstract

The invention relates to a light detection and ranging device and a method of operating the same. A light detection and ranging (LiDAR) device according to the present disclosure includes a transmitter configured to transmit a laser signal to a target and a receiver configured to receive a reflected laser signal returned from reflection by the target. The receiver includes a plurality of column circuits connected to the sensors in the receiver through N independent channels for each column of the pixel array corresponding to the sensors in the receiver, where N is a natural number equal to or greater than 2.

Description

Technical Field

[0001] Exemplary embodiments of the present disclosure relate to a light detection and ranging (LiDAR) device and an operating method thereof. Background Art

[0002] Recently, with the interest in autonomous and driverless vehicles, light detection and ranging (LiDAR) devices have attracted a lot of attention. LiDAR devices use laser signals to obtain information about distances to the surrounding environment. Because of the advantages of excellent accuracy and resolution and identifying objects in three dimensions, LiDAR devices are trending for applications in various fields (e.g., drones, airplanes, etc., and vehicles).

[0003] Generally, the entire array in a transceiver of a LiDAR device operates according to a pre-specified operation algorithm, and in this case, a histogram method is used as a noise processing method. The histogram method is a method of accumulating multiple measurement values (between frames) to obtain a single time-of-flight (ToF) result (one frame). Frames per second (fps), which represents the speed of frame updates per second, is one of the main performance metrics of a LiDAR device.

[0004] According to the prior art, as the resolution (number of pixels) of a receiver increases, the area that must be covered by one pixel of a transmitter increases. Therefore, there is a problem of wasting the area of the receiver during operation due to the limited number of readout integrated circuits (ROICs, one per column) including time-to-digital converters (counters).

[0005] The prior art of the present disclosure is disclosed in Korean Patent Application Publication No. 10-2023-0102817 (published on July 7, 2023 and titled "LiDAR Device and Operating Method Thereof"). Summary of the Invention

[0006] Various embodiments are directed to providing a LiDAR device and an operating method thereof that can obtain additional intensity data from wasted pixels in a receiver by utilizing circuits (counters) used in a time-to-digital converter (TDC).

[0007] The objectives to be solved by the present disclosure are not limited to the above objectives, and other objectives not mentioned herein will be clearly understood by those skilled in the art to which the present disclosure pertains from the following description.

[0008] A LiDAR device according to an embodiment of the present disclosure includes: a transmitter configured to emit a laser signal toward a target; and a receiver configured to receive a reflected laser signal that returns after being reflected by the target, wherein the receiver includes a plurality of column circuits, and the plurality of column circuits are connected to sensors in the receiver through N independent channels in each column of a pixel array corresponding to the sensors in the receiver, where N is a natural number equal to or greater than 2.

[0009] The receiver may further include a sensor array module configured such that the number of sensors in the receiver in each row of each column of the pixel array is N times the number of sensors in the transmitter.

[0010] The column circuits may alternately operate the sensors in the receiver for each row of the pixel array based on a control signal sent by a column scanner to perform signal processing on the response of the sensors in the receiver to the reflected signal.

[0011] When the sensors in the receiver are connected to independent first and second channels, the column circuits may generate time-of-flight (ToF) histogram data through the first channel and obtain intensity data representing the number of times the corresponding pixel has responded through the second channel.

[0012] The column circuits may obtain the intensity data through a circuit configuration based on a single-slope analog-to-digital converter (ADC).

[0013] The column circuits may include switches and operate such that the sensors in the receiver can be alternately connected to the first and second channels in units of frames through the switches.

[0014] The column circuits may further include a multiplexer (MUX) and a counter, and share the counter for the first and second channels through the multiplexer.

[0015] A method of operating a light detection and ranging (LiDAR) device according to an embodiment of the present disclosure includes: emitting a laser signal toward a target by a transmitter; receiving a reflected laser signal that returns from the target by a receiver; and performing signal processing on the response of sensors in the receiver to the reflected laser signal by a plurality of column circuits connected to the sensors in the receiver, wherein the column circuits are connected to the sensors in the receiver through N independent channels in each column of a pixel array corresponding to the sensors in the receiver, where N is a natural number equal to or greater than 2.

[0016] The receiver may further include a sensor array module configured such that the number of sensors in the receiver in each row of each column of the pixel array is N times the number of sensors in the transmitter.

[0017] When sensors in a receiver are connected to independent first and second channels, a column circuit can generate time-of-flight (ToF) histogram data through the first channel and obtain intensity data representing the number of times corresponding pixels have responded through the second channel.

[0018] Details of other examples are included in the detailed description and drawings of the present disclosure.

[0019] According to an embodiment of the present disclosure, a circuit (counter) used in a TDC can be used to obtain additional intensity data from pixels in a receiver, which allows for making full use of the pixels in the receiver to obtain an output without wasting pixels.

[0020] According to an embodiment of the present disclosure, without degrading the performance of the LiDAR device, the FPS can be increased by up to two times. Description of the Drawings

[0021] Figure 1 is a block diagram showing a light detection and ranging (LiDAR) device according to an embodiment of the present disclosure.

[0022] Figure 2 is showing Figure 1 a detailed configuration of the receiver.

[0023] Figure 3 is showing Figure 2 a detailed configuration of the column circuit.

[0024] Figure 4 and Figure 5 is showing Figure 2 an operating example of the column circuit.

[0025] Figure 6 is a flowchart showing a method of operating a LiDAR device according to an embodiment of the present disclosure. Detailed Description

[0026] Components described in the exemplary embodiments can be implemented by hardware components including, for example, at least one digital signal processor (DSP), a processor, a controller, an application specific integrated circuit (ASIC), programmable logic elements (such as an FPGA), other electronic devices, or combinations thereof. At least some of the functions or processes described in the exemplary embodiments can be implemented by software, and the software can be recorded on a recording medium. The components, functions, and processes described in the exemplary embodiments can be implemented by a combination of hardware and software.

[0027] The method according to the exemplary embodiments can be embodied as a computer-executable program and can be implemented in various recording media such as magnetic storage media, optical reading media, and digital storage media.

[0028] The different techniques described herein can be implemented as digital electronic circuits, or as computer hardware, firmware, software, or combinations thereof. The techniques can be implemented as a computer program product, i.e., a computer program tangibly embodied in an information carrier (e.g., in a machine-readable storage device (e.g., a computer-readable medium)) or a propagated signal for processing by, or to control the operation of, a data processing apparatus (e.g., a programmable processor, a computer, or multiple computers). The computer program can be written in any form of programming language, including compiled or interpreted languages, and can be deployed in any form, including as a stand-alone program or as modules, components, subroutines, or other units suitable for use in a computing environment. The computer program can be deployed to be executed on one computer or on multiple computers at a site or distributed across multiple sites and interconnected by a communication network.

[0029] For example, processors suitable for executing a computer program include any one or more processors of general and special purpose microprocessors and any kind of digital computer. Generally, a processor will receive instructions and data from a read-only memory or a random access memory or both. Elements of a computer may include at least one processor for executing instructions and one or more memory devices for storing instructions and data. Generally, a computer will also include one or more mass storage devices (e.g., magnetic disks, magneto-optical disks, or optical disks) or be coupled to receive data from, transfer data to, or perform both on one or more mass storage devices for storing data. Examples of information carriers suitable for embodying computer program instructions and data include semiconductor memory devices, such as magnetic media like hard disks, floppy disks, and magnetic tapes, optical media like compact disc read-only memory (CD-ROM), digital video disc (DVD), etc., and magneto-optical media like floppy optical discs, as well as read-only memory (ROM), random access memory (RAM), flash memory, erasable programmable ROM (EPROM), and electrically erasable programmable ROM (EEPROM) and any other known computer-readable media. The processor and the memory can be supplemented by, or integrated into, special purpose logic circuitry.

[0030] The processor can run an operating system (OS) and one or more software applications running on the OS. The processor device can also access, store, manipulate, process, and create data in response to the execution of the software. For simplicity, the description of the processor device is used in the singular; however, those skilled in the art will understand that the processor device can include multiple processing elements and / or multiple types of processing elements. For example, the processor device can include multiple processors or a processor and a controller. Additionally, different processing configurations are possible, such as parallel processors.

[0031] In addition, a non-transitory computer-readable medium can be any available medium accessible by a computer and can include both computer storage media and transmission media.

[0032] This specification includes details of multiple specific embodiments, but it should be understood that the details do not limit any invention or what can be claimed in the specification, but rather describe the features of specific example embodiments. Features described in the context of various exemplary embodiments in this specification can be implemented as a combination in a single exemplary embodiment. Conversely, different features described in the context of a single exemplary embodiment in the specification can be implemented individually or in a suitable sub-combination in multiple exemplary embodiments. Additionally, these features can operate in a specific combination and can be initially described as required in the combination, but in some cases, one or more features can be excluded from the required combination, and the required combination can be changed into a sub-combination or a modification of a sub-combination.

[0033] Similarly, even if operations are described on the drawings in a specific order, it should not be understood that the operations need to be performed in a specific order or sequence to obtain the desired result, nor should it be understood that all operations need to be performed. In certain cases, multitasking and parallel processing can be advantageous. Additionally, in all exemplary embodiments, it should not be understood as requiring the separation of various device components in the above exemplary embodiments, and it should be understood that the above program components and devices can be incorporated into a single software product or can be packaged in multiple software products.

[0034] It should be understood that the exemplary embodiments disclosed herein are merely illustrative and are not intended to limit the scope of the present invention. It is obvious to those of ordinary skill in the art that various modifications can be made to the exemplary embodiments without departing from the spirit and scope of the claims and their equivalents.

[0035] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings, so that those skilled in the art can easily implement the present disclosure. However, the present disclosure can be embodied in many different forms and is not limited to the embodiments described herein.

[0036] In the following description of the embodiments of the present disclosure, detailed descriptions of known functions and configurations included herein will be omitted when they may make the subject matter of the present disclosure rather unclear. Parts of the drawings that are not relevant to the description of the present disclosure are omitted, and similar parts are denoted by similar reference numerals.

[0037] In the present disclosure, components that are distinct from each other are intended to clearly show each feature. However, it does not necessarily mean that the components are separate. That is, multiple components can be integrated into one hardware or software unit, or a single component can be distributed into multiple hardware or software units. Therefore, unless otherwise indicated, such integrated or distributed embodiments are also included within the scope of the present disclosure.

[0038] In the present disclosure, the components described in various embodiments are not necessarily essential components, and some can be optional components. Therefore, embodiments that constitute a subset of the components described in one embodiment are also included within the scope of the present disclosure. In addition, embodiments that include other components in addition to the components described in various embodiments are also included within the scope of the present disclosure.

[0039] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily implement the present disclosure. However, the present disclosure can be embodied in many different forms and is not limited to the embodiments described herein.

[0040] In the following description of the embodiments of the present disclosure, detailed descriptions of known functions and configurations included herein will be omitted when they may make the subject matter of the present disclosure rather unclear. Parts of the drawings that are not relevant to the description of the present disclosure are omitted, and similar parts are denoted by similar reference numerals.

[0041] In the present disclosure, when a component is referred to as being "linked", "coupled" or "connected" to another component, it should be understood that not only a direct connection relationship is included, but also an indirect connection relationship through an intermediate component. In addition, when a part is referred to as "including" or "having" another part, it may mean further including another part rather than excluding it, unless explicitly described to the contrary.

[0042] In the present disclosure, unless otherwise specifically stated, the terms first, second, etc. are only for the purpose of distinguishing one component from another, and do not limit the order or importance of the components, etc. Therefore, within the scope of the present disclosure, the first component in one exemplary embodiment may be referred to as the second component in another embodiment, and similarly, the second component in one exemplary embodiment may be referred to as the first component.

[0043] In the present disclosure, components that are distinct from each other are intended to clearly show each feature. However, it does not necessarily mean that the components are separate. That is, multiple components can be integrated into one hardware or software unit, or a single component can be distributed into multiple hardware or software units. Therefore, unless otherwise indicated, such integrated or distributed embodiments are also included within the scope of the present disclosure.

[0044] In the present disclosure, the components described in various embodiments are not necessarily essential components, and some may be optional components. Accordingly, embodiments that constitute a subset of the components described in one embodiment are also included within the scope of the present disclosure. In addition, exemplary embodiments including other components in addition to the components described in various embodiments are also included within the scope of the present disclosure.

[0045] Figure 1 is a block diagram showing a light detection and ranging (LiDAR) device according to an embodiment of the present disclosure.

[0046] Referring Figure 1 , the LiDAR device 100 according to an embodiment of the present disclosure may be configured to include a transmitter 110, a receiver 120, a signal processor or readout integrated circuit (ROIC) 130, a controller, and a transmit driver 150.

[0047] The transmitter 110 may emit a laser signal toward a target. To this end, the controller 140 may send a control command to the transmitter 110 through the transmit driver 150. At this time, the transmitter 110 may send a start signal indicating a time point at which the transmitter 110 starts to send a laser signal toward the target to the signal processor 130.

[0048] The receiver 120 may receive a reflected laser signal that returns after being reflected by the target. In this case, the receiver 120 may send a stop signal indicating a time point at which the receiver 120 finishes receiving the reflected signal to the signal processor 130.

[0049] The signal processor 130 may perform signal processing according to a control command of the controller 140. That is, the signal processor 130 may use the start signal and the stop signal to calculate a time elapsed between emitting a laser through the transmitter 110 and receiving the reflected laser signal reflected by the target by the receiver 120, and then measure a distance to the target based on the calculated time.

[0050] To this end, conventionally, the entire arrays in the transmitter 110 and the receiver 120 operate according to a pre-specified operation algorithm when receiving a command from the controller 140, where a histogram method is used as a noise processing method.

[0051] The histogram method is a method of accumulating multiple measured values (inter-frame) to obtain a single time-of-flight (ToF) result (one frame). As a reference, frames per second (fps), which represents the speed of updating frames per second, is one of the main performance indicators of the LiDAR device.

[0052] According to the prior art, as the resolution (number of pixels) of the receiver 120 increases, the area of the receiver 120 that must be covered by one pixel of the transmitter 110 increases. Therefore, due to the limited number of readout integrated circuits (ROICs, one per column) including time-to-digital converters (counters), there is a problem of wasting the area of the receiver 120 during operation.

[0053] Therefore, as Figure 2 shown, the receiver 120 may include a plurality of column circuits, which are connected to the sensors 230 in the receiver 120 through N (where N is a natural number equal to or greater than 2) independent channels 240 and 250 in each column of the pixel array corresponding to the sensors 230 in the receiver 120, thereby allowing the receiver 120 to obtain an output without wasted pixels.

[0054] The receiver 120 may further include a sensor array module 210, which is configured such that the number of sensors 230 in the receiver 120 in each row of each column of the pixel array is N times the number of sensors in the transmitter 110.

[0055] The column circuit 220 may alternately operate the sensors 230 in the receiver 120 in each row of the pixel array constituting the sensor array module based on a control signal sent through a column scanner (see 301 in Figure 3 ), and may perform signal processing on the response of the sensors 230 in the receiver 120 to the reflected signal.

[0056] As a reference, Figure 2 the column scanner and the row scanner in

[0057] are circuits that can change channels according to the module operation algorithm. Hereinafter, embodiments of the present disclosure will be described based on the assumption that the number of channels connected to each column circuit is two. Obviously, this is only for ease of explanation and understanding and is not intended to limit the scope of the present disclosure.

[0058] The sensors 230 in the receiver 120 may be connected to the column circuit 220 through the first channel 240 and the second channel 250. That is, the sensors 230 in the receiver 120 may be connected to the corresponding column circuit 220 through two independent channels (the first channel 240 and the second channel 250).

[0059] The column circuit 220 may generate time-of-flight (ToF) histogram data through the first channel 240 and obtain intensity data through the second channel 250. In this case, the intensity data may represent the number of times the corresponding pixel has responded, which is the total number of data accumulated in the histogram.

[0060] To this end, the column circuit 220 may be configured to include a switch 310, a charge pump 320, a sample and hold amplifier (S / H: sample / hold) 330, a comparator 340, a multiplexer (MUX) 350, and a counter 360, as Figure 3 shown.

[0061] The column circuit 220 may operate such that the sensor 230 in the receiver 120 is alternately connected to the first channel 240 and the second channel 250 in frame units through the switch 310.

[0062] When the first channel 240 is selected through the switch 310, the column circuit 220 may generate ToF histogram data for the corresponding pixel through the first channel 240.

[0063] That is, as Figure 4 shown, the column circuit 220 may start counting at the start pulse and stop counting at the stop pulse on the first channel 240 through the counter 360 that inputs the start pulse and the stop pulse.

[0064] In this way, the column circuit 220 may output the ToF representing the time measurement value between the start pulse and the stop pulse, and then generate ToF histogram data by accumulating the output ToF into the histogram.

[0065] On the other hand, when the second channel 250 is selected through the switch 310, the column circuit 220 may generate intensity data for the corresponding pixel through the second channel 250. That is, as Figure 5 shown, the column circuit 220 may obtain intensity data on the second channel 250 through a circuit configuration based on a single ramp analog-to-digital converter (ADC) 510.

[0066] On the second channel, the charge pump 320 may receive a stop pulse input from the sensor 230 associated with the corresponding pixel and output a Vcp signal by accumulating the number of stop pulses. In this case, since the number of stop pulses needs to be accumulated, the measurement period of the stop pulse may be implemented in row units. The Vcp signal may represent the sum between several frames.

[0067] When the charge pump 320 outputs the Vcp signal, the single ramp ADC 510 may receive the Vcp signal and sample it through the S / H 330 to output a Vcp' signal that maintains a constant voltage. The comparator 340 may receive the Vcp' signal and the Vramp signal, and output a Vcomp signal when the two values (the Vcp' signal and the Vramp signal) become equal. Then, the counter 360 may receive the Vcomp signal output from the comparator 340 to obtain intensity data.

[0068] The column circuit 220 can enable the first channel 240 and the second channel 250 to share the counter 360 through the multiplexer 350. That is, when the first channel 240 is selected through the switch 310, the column circuit 220 can turn on the first channel 240 to send ToF to the counter 360, and when the second channel 250 is selected through the switch 310, the column circuit 220 can turn on the second channel 250 to send the Vcomp signal to the counter 360.

[0069] Meanwhile, even when the number of channels as described above is not RX:TX = 2:1 (i.e., two RXs per TX, but three or more RXs per TX), the present disclosure can also be applied. In this case, by using the interpolation method and pixel operation switching (according to an embodiment of the present disclosure), the FPS in another embodiment of the present disclosure can be improved.

[0070] Figure 6 is a flowchart showing a method of operating a LiDAR device according to an embodiment of the present disclosure.

[0071] The method of operating a LiDAR device described herein is merely one embodiment of the present disclosure, and various other steps can be added as needed, such as the following steps, and the following steps can be executed in a different order, so that the present disclosure is not limited to each step and order described below.

[0072] Refer to Figure 1 、 Figure 2 and Figure 6 , in step 610, the transmitter 110 in the LiDAR device 100 can emit a laser signal towards the target.

[0073] Next, in step 620, the receiver 120 in the LiDAR device 100 can receive the reflected laser signal returned from being reflected by the target.

[0074] Next, in step 630, the receiver 120 in the LiDAR device 100 can perform signal processing on the response of the sensor 230 in the receiver 120 to the reflected signal through a plurality of column circuits 220 connected to the sensor 230 in the receiver 120.

[0075] Here, each of the column circuits 220 can be connected to the sensor 230 in the receiver 120 through two independent channels 240 and 250 of each column of the pixel array corresponding to the sensor 230 in the receiver 120, so that signal processing can be performed on the response of the sensor 230 in the receiver 120 to the reflected signal.

Claims

1. A light detection and ranging device, comprising: a transmitter configured to transmit a laser signal toward a target; as well as a receiver configured to receive the reflected laser signal reflected back by the target, The receiver includes a plurality of column circuits connected to the sensor in the receiver through N independent channels of each column of a pixel array corresponding to the sensor in the receiver, wherein N is a natural number equal to or greater than 2.

2. The light detection and ranging device according to claim 1, in, The receiver further includes a sensor array module configured such that the number of sensors in the receiver in each row of each column of the pixel array is N times that of sensors in the transmitter.

3. The light detection and ranging device according to claim 1, in, The column circuit alternately operates the sensors in the receiver for each row of the pixel array based on a control signal sent through a column scanner to perform signal processing on responses of the sensors in the receiver to the reflected laser signal.

4. The light detection and ranging device according to claim 1, in, When the sensor in the receiver is connected to independent first and second channels, the column circuit generates time-of-flight histogram data of the corresponding pixel through the first channel and obtains intensity data representing the number of times the corresponding pixel has responded through the second channel.

5. The light detection and ranging device according to claim 4, in, The column circuit obtains the intensity data through a circuit configuration based on a single-slope analog-to-digital converter.

6. The light detection and ranging device according to claim 4, in, The column circuit includes a switch and operates so that the sensor in the receiver is alternately connected to the first channel and the second channel in units of frames through the switch.

7. The light detection and ranging device according to claim 6, in, The column circuit further includes a multiplexer and a counter, and the counter is shared by the first channel and the second channel through the multiplexer.

8. A method of operating a light detection and ranging device, comprising: The transmitter emits a laser signal to the target; receiving, by a receiver, a reflected laser signal reflected by the target and returned; as well as performing signal processing, by the receiver through a plurality of column circuits connected to the sensors in the receiver, on responses of the sensors in the receiver to the reflected laser signal; The column circuit is connected to the sensor in the receiver through N independent channels of each column of the pixel array corresponding to the sensor in the receiver, wherein N is a natural number equal to or greater than 2.

9. The method according to claim 8, in, The receiver includes a sensor array module configured such that the number of sensors in the receiver in each row of each column of the pixel array is N times that of sensors in the transmitter.

10. The method according to claim 8, in, When the sensor in the receiver is connected to independent first and second channels, the column circuit generates time-of-flight histogram data of the corresponding pixel through the first channel and obtains intensity data representing the number of times the corresponding pixel has responded through the second channel.

Images