Lidar system and control method therefor

Abstract

The present disclosure provides a system comprising: a LiDAR device including a light emitter for emitting light and a light detector for detecting light; a variable processor unit for converting and pre-processing a signal input from the LiDAR device; a fixed processor unit for receiving pre-processed data from the variable processor unit and generating a point cloud; and an external memory connected to the fixed processor unit, wherein the variable processor unit has a circuit, the configuration of which is changeable according to an application environment or a detection distance of the LiDAR device.

Description

Lidar system and its control method

[0001] The present invention relates to a lidar system and a control method thereof, and more specifically, to a lidar system comprising a fixed part and a variable part in relation to the chip architecture of the lidar.

[0002] With the recent expansion of autonomous vehicles and Advanced Driver Assistance Systems (ADAS), the importance of LiDAR (Light Detection and Ranging) technology for precisely recognizing three-dimensional information of the surrounding environment is growing. LiDAR systems calculate distance information by emitting laser light and measuring the time it takes for the light to reflect back from a target object, and based on this, they form a multidimensional point cloud to recognize the location and shape of surrounding objects. This LiDAR technology requires a high-speed signal processing chip structure to simultaneously ensure precision in distance measurement and computational efficiency.

[0003] Lidar systems have traditionally utilized System-on-Chip (SoC) based signal processing structures that include Time-to-Digital Converters (TDCs) or Analog-to-Digital Converters (ADCs). However, these conventional SoC structures have limitations in that they struggle to meet varying requirements depending on the application environment, such as for long-range (e.g., 240 meters) Lidar and short-range (e.g., 30 meters) Lidar. For instance, while lightweight signal processing logic is required for short-range detection applications, conventional SoCs have fixed logic designs, making application-specific optimization impossible. Consequently, changing the signal processing structure according to the application requires redesigning the entire SoC, including the central processing unit, which leads to excessively increased development costs and timelines. Therefore, there is a growing demand for technologies capable of solving these aforementioned problems.

[0004] The aforementioned background technology is technical information that the inventor possessed for the derivation of the present invention or acquired during the process of deriving the present invention, and it cannot be considered as prior art disclosed to the general public prior to the filing of the present invention.

[0005] The present invention has been devised in light of the aforementioned purpose, and the objective of the present invention is to provide a lidar system comprising a fixed part and a variable part in relation to the architecture of a lidar chip.

[0006] In addition, the objective of the present invention is to provide a LiDAR signal processing chip structure capable of flexibly optimizing the signal processing structure according to the application environment (e.g., long-range and short-range measurement environments).

[0007] Furthermore, the present invention provides a LiDAR signal processing chip structure that can provide advanced functions more efficiently by adding an external memory control structure to enable large-capacity computation on a frame-by-frame basis.

[0008] The problems to be solved in the embodiments of the present invention are not limited thereto, and may also include objectives or effects that can be identified from the means for solving the problems or embodiments described below.

[0009] A lidar system according to one embodiment of the present invention for solving the presented problem comprises: a lidar device including a light emitter that emits light and a light detector that detects light; a variable processor unit that converts and preprocesses a signal input from the lidar device; a fixed processor unit that receives preprocessed data from the variable processor unit and generates a point cloud; and an external memory connected to the fixed processor unit, wherein the circuit configuration of the variable processor unit may be changed according to the detection distance or application environment of the lidar device.

[0010] In addition, the above-mentioned fluctuation processor unit includes a TDC (Time to Digital Converter) or an ADC (Analog to Digital Converter) and can convert time information of a signal received from the photodetector of the lidar device into digital data.

[0011] In addition, the above-mentioned fluctuation processor may include a signal processing unit that receives digital data converted from the TDC or the ADC and performs a preprocessing operation.

[0012] In addition, the above-mentioned fluctuation processor unit further includes a scanner / laser driver unit that controls the driving of the scanner and the laser diode, and the scanner / laser driver unit can control the rotation speed and scanning angle of the scanner to adjust the scanning pattern.

[0013] In addition, the fixed processor unit may include a point cloud generation unit that generates a point cloud including distance and depth using the preprocessed data transmitted from the variable processor unit.

[0014] In addition, the fixed processor unit may further include an auxiliary function unit that performs a Clean Point Cloud or Blockage Detection using the data of the point cloud.

[0015] In addition, the above auxiliary function unit can communicate with external memory to perform large-scale computations.

[0016] In addition, the external memory is composed of dynamic random access memory (DRAM) and can store frame data of the fixed processor unit and buffer intermediate results of the generation of the clean point cloud.

[0017] In addition, the above-mentioned variable processor unit further includes an internal memory for temporarily storing the preprocessing results of the signal processing unit, and the internal memory may be configured as static random access memory (SRAM).

[0018] In addition, the fixed processor unit and the variable processor unit can transmit and receive data through an LVDS (Low Voltage Differential Signalling) interface.

[0019] In addition, the LVDS interface is configured as a single interface and can support the transmission of the preprocessed data of the variable processor unit.

[0020] In addition, the variable processor unit further includes a register interface, and the register interface can perform communication with a vehicle control unit included in the fixed processor unit.

[0021] In addition, the vehicle control unit can be implemented with a structure based on vehicle control software architecture standards.

[0022] A method for processing a signal of a LiDAR system according to an embodiment of the present invention for solving a presented problem comprises: receiving reflected light with a photodetector included in a LiDAR device; converting and preprocessing a signal input from the LiDAR device in a variable processor unit; generating a point cloud by receiving preprocessed data from the variable processor unit in a fixed processor unit; and storing the point cloud data or performing auxiliary operations using an external memory connected to the fixed processor unit, wherein the circuit configuration of the variable processor unit may be changed according to the detection distance or application environment of the LiDAR device.

[0023] The means for solving the above-described problem are described as a method and apparatus according to one embodiment of the present invention, and it should be understood that additions, deletions, and changes are possible within the spirit and scope of the invention other than those described above.

[0024] The present invention enables functional expansion or weight reduction through partial logic modifications based on the application purpose. For example, functional expansion and weight reduction are possible for short-range and long-range measurement applications by modifying the chip structure and logic.

[0025] In addition, the present invention can sufficiently support data transmission between the variable part and the fixed part through an interface implemented with a single channel (e.g., LVDS), thereby having the effect of minimizing costs during design.

[0026] In addition, the present invention can provide advanced functions more efficiently by adding an external memory control structure to enable large-scale computation on a frame-by-frame basis.

[0027] FIG. 1 is a conceptual diagram showing an example of the configuration and operation of a lidar device according to the present invention.

[0028] FIG. 2 is a block diagram illustrating the configuration of a measuring device for measuring the field of view of a lidar device according to the present invention.

[0029] FIG. 3 is a block diagram of a vehicle system having a lidar device according to the present invention.

[0030] Figure 4 is a diagram illustrating an example of a lidar system.

[0031] FIG. 5 is a drawing for explaining a lidar system including a fixed part and a variable part according to an embodiment of the present invention.

[0032] Figure 6 is a diagram illustrating examples of input and output data of the lidar system of Figure 5.

[0033] FIG. 7 is a drawing for explaining an example of a lidar system including a fixed part and a variable part according to an embodiment of the present invention.

[0034] The terms used in this specification will be briefly explained, and the present disclosure will be described in detail. In describing the present disclosure, detailed descriptions of related prior art may be omitted, and redundant descriptions of identical components will be omitted whenever possible.

[0035] The following detailed description of the invention refers to the accompanying drawings, which illustrate specific embodiments in which the invention may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the invention. It should be understood that various embodiments of the invention are different but need not be mutually exclusive. For example, specific shapes, structures, and characteristics described herein may be implemented in other embodiments without departing from the spirit and scope of the invention in relation to one embodiment.

[0036] Terms containing ordinal numbers, such as first, second, etc., may be used to describe various components, but the components are not limited by these terms. Terms may be used for the purpose of distinguishing one component from another. For example, it should be understood that a first component may be named a second component, and conversely, a second component may be named a first component.

[0037] The terms used in this application are used merely to describe specific embodiments and are not intended to limit the invention. The singular expression includes the plural expression unless the context clearly indicates otherwise. In this application, terms such as "comprising" or "having" are intended to specify the presence of the features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, or combinations thereof.

[0038] Furthermore, it should be understood that the location or arrangement of individual components within each disclosed embodiment may be changed without departing from the spirit and scope of the invention. Accordingly, the following detailed description is not intended to be taken in a limiting sense, and the scope of the invention is limited only by the appended claims, including all equivalents thereof, provided appropriately described. Similar reference numerals in the drawings refer to the same or similar functions across various aspects.

[0039] FIG. 1 is a conceptual diagram showing an example of the configuration and operation of a light detection and range measuring device (Light Detection And Ranging) or lidar device according to the present invention.

[0040] Referring to FIG. 1, the light detection and distance measuring device (100, hereinafter referred to as 'LiDAR') of the present invention may include a light emitter (110) for emitting light, a light detector (120) for detecting reflected light that is reflected back from an object (140) after the emitted light, and an optical device (130) provided in the light path emitted and received from the light emitter (110) and the light detector (120). Here, the light emitter (110) may be a diode and a laser light source. The LiDAR device can calculate information regarding the range, property, and depth of an object (140) using the reflected light that is reflected back from an object (140).

[0041] A point cloud refers to a set of data points in 3D space. A set of data points calculated by a LiDAR device (100) to which the present invention is applied can also be called a point cloud.

[0042] According to the present invention, the device subject to inspection is referred to as the Device Under Test (DUT). Meanwhile, in the specification of the present invention, the Device Under Test may be interchangeably referred to as a LiDAR device.

[0043] Among results where the measurement value is positive, the case where the measurement value is accurate—that is, where both the measurement value and the result are positive—is called a True Positive (TP). Among results where the measurement value is positive, the case where the measurement value is inaccurate—that is, where the measurement value is positive but the result is negative—is called a False Positive (FP).

[0044] The probability of valid points in a single measurement and / or multiple accumulated measurements for a single target is called the Probability of Detection (PoD) or True Positive Rate. The probability of detection may depend on background noise, the reflectivity of the target, the tolerance of the range, and other attributes. The PoD can be calculated, where True (TP) represents the total number of scan points hitting the detected target within a distance (actual) ±△. The probability of detection is calculated as the ratio of the number of valid points to the number of theoretical points.

[0045] In the point cloud of the LiDAR device (100), the angle formed by the connection between two adjacent detection points and the three-dimensional coordinate origin of the point cloud in terms of azimuth angle and elevation angle is called angular resolution. The angular resolution of the LiDAR device can be divided into azimuth resolution and elevation resolution.

[0046] In the point cloud of the LiDAR device (100), the angle between two outermost effective points is called the field of view (FOV). The field of view includes a horizontal field of view and a vertical field of view.

[0047] Capturing the entire field of view is called a frame.

[0048] Additionally, the object (140) can be a three-dimensional object of various shapes. For example, the object (140) can be a rectangular shape, a bow tie shape, a rhombus shape, a triangle shape, a human shape, a car shape, a geographical feature shape, etc.

[0049] FIG. 2 is a block diagram illustrating the configuration of a measuring device for measuring the field of view of a lidar device according to the present invention.

[0050] As illustrated in FIG. 2, the measuring device (10) may include a rotation adjustment unit (12). The rotation adjustment unit (12) is configured to rotate the device to be inspected and may include a rotation unit and a rotation control unit. The rotation adjustment unit (12) is configured to be connected to the device to be inspected and can rotate the device to be inspected by a predetermined rotation angle. More specifically, the rotation control unit of the rotation adjustment unit (12) receives a signal to rotate by a predetermined rotation angle and transmits a signal to the rotation unit to control the rotation, and the rotation unit receives the control signal and can rotate the device to be inspected by a predetermined angle.

[0051] Additionally, as shown in FIG. 2, the measuring device (10) may optionally further include a support part (11) for the device to be inspected, a processor part (13), and a memory part (14).

[0052] Specifically, the inspection target device support member (11) is intended to support the inspection target device. The inspection target device support member (11) may be configured to support the inspection target device and the rotation adjustment member (12) that rotate together.

[0053] Specifically, the processor unit (13) can control the operation of the rotation adjustment unit (12) and the operation of the inspection target device connected to the measuring device (10). The processor unit (13) can process information output by the inspection target device connected to the measuring device (10) and the rotation adjustment unit (12), and can determine the viewing angle of the inspection target device by processing the output information.

[0054] Specifically, the memory unit (140) may be configured to store information output by the device to be inspected connected to the measuring device (10) and information generated based on the information output by the device to be inspected. Additionally, the memory unit (14) may have a program recorded therein to enable the processor unit (13) to perform an operation of controlling the operation of the rotation adjustment unit (12) and processing the information output by the device to be inspected to determine the viewing angle of the device to be inspected.

[0055] FIG. 3 is a block diagram of a vehicle system having a lidar device according to the present invention.

[0056] As shown in FIG. 3, a vehicle system (200) having a lidar device (100) and a vehicle control module (212) receives input from a user or driver or provides information to a user or driver through a user interface (211). The user interface (211) may include a display device, a touch panel, a button, voice recognition, a wired or wireless input device, and may be connected via wired or wireless means to enable communication between the driver and various devices.

[0057] The vehicle system (200) communicates with a remote device (213), and the remote device (213) can communicate remotely with a user or the outside or receive an external control signal. The communication unit (215) can support wired or wireless communication and, for example, may be a wired or wireless module.

[0058] The storage unit (220) may include one or more sub-memories (221) internally. Additionally, the storage unit (220) may include a portable or removable storage device (222). The LiDAR device (100) may communicate with a user interface (211) and a camera unit (101).

[0059] Additionally, the storage unit (220) may include SRAM and DRAM, etc., capable of storing information for various applications of the lidar device.

[0060] The lidar device (100) includes a drive unit (115), such as a motor, and the drive unit (115) can rotate a part or the whole of the lidar device (100) 360 degrees by a control signal. The drive unit (115) communicates with an internal component of the lidar device (100), such as a measurement system (110), and enables the lidar device (100) to rotate axially.

[0061] The lidar device (100) may include a measurement system (150) and an optical device (130). The driving unit (115) may be configured to transmit driving force by being coupled to allow the measurement system (150) and the optical device (130) to rotate.

[0062] The measurement system (150) may include a main processor (151) and a main memory (152), and the main processor (151) may be implemented as a general-purpose processor, an Application Specific Integrated Circuit (ASIC), one or more Field Programmable Gate Arrays (FRPGA), a group of processing components, or other suitable electronic processing components. The main memory (e.g., memory, memory unit, storage device, etc.) (152) may include one or more devices (e.g., RAM, ROM, S-RAM, D-RAM, hard disk storage, SSD, etc.) for storing data and / or programming code to complete or facilitate the various processes described in the present invention. The main memory (152) may include database components, object code components, script components, or any other type of information structure to support the various activities and information structures described in the present invention. According to one embodiment of the present invention, the main memory (152) may be communicably connected to the main processor (151).

[0063] The measurement system (150) may include one or more processors (also referred to as a central processing unit or CPU). One or more processors may be connected to a communication infrastructure or a bus. Additionally, one or more processors may each be a graphics processing unit (GPU). In some other embodiments, the graphics processing unit (GPU) may include a processor, which is a specialized electronic circuit designed to process mathematically intensive applications. The GPU may have a parallel structure efficient for the parallel processing of large blocks of data, such as mathematically intensive data commonly used in computer graphics applications, images, videos, etc.

[0064] The measurement system (150) is a computer system and can be connected to one or more user input / output devices, such as a monitor, keyboard, and pointing device, which communicate with a communication infrastructure through a user input / output interface.

[0065] Additionally, one or more optical devices (130) may be arranged, for example, in a plurality, within the lidar device (100). The plurality of optical devices (130) may irradiate and sense laser beams in different directions relative to a rotation axis. Here, the different directions may be in the range of 10 to 180 degrees relative to each other, and may be arranged, for example, as any one of 30 degrees, 60 degrees, 90 degrees, 120 degrees, 150 degrees, or 180 degrees.

[0066] Below, a lidar system including a main processor (151) and a vehicle management module (212) of a lidar device (100) is described in more detail.

[0067] Figure 4 is a diagram illustrating an example of a lidar system.

[0068] Referring to FIG. 4, the lidar system (300) may include a system-on-chip (SoC, 310), an auxiliary chip (320), an external memory (330), a vehicle control module (340), an optical receiver (350), and a scanner (360).

[0069] The LiDAR system (300) receives light reflected from a target object to calculate distance information and generates point cloud data based on this, thereby recognizing the surrounding environment of the vehicle in three dimensions.

[0070] The system-on-chip (310, SoC) is a core computational module of the lidar system and may include a series of circuits that convert an analog signal input from an optical receiver into a digital signal and perform signal processing and point cloud generation.

[0071] For example, the SoC (310) may include a time-to-digital converter (TDC) / analog-to-digital converter (ADC) (311), a signal processing unit (312), a scanner / laser driver (313), an internal memory (314), and a point cloud generation unit (315).

[0072] The TDC / ADC (311) can convert an optical pulse signal input from the optical receiver (350) into electrical time information and convert the time information into digital data. That is, it can detect the time difference (Time-of-Flight) between the time when the optical pulse is received and the time when it is transmitted, and generate basic data for calculating the distance to the target object.

[0073] In one embodiment of the present invention, the TDC / ADC (311) may be configured to enable precise distance measurement by including a high-speed conversion circuit to secure a time resolution in the nanosecond (ns) range.

[0074] The signal processing unit (312) receives the digital signal converted by the TDC / ADC (311) and can perform preprocessing operations such as noise removal, filtering, threshold comparison, and peak detection. For example, the signal processing unit (312) can be implemented as hardware logic (e.g., ASIC or FPGA), thereby reducing data transmission bandwidth and minimizing the computational burden on the CPU.

[0075] The scanner / laser drive unit (313) can perform the role of controlling the laser emission of the lidar system and the rotational drive of the scanner.

[0076] For example, in one embodiment of the present invention, the scanner / laser driving unit (313) can form a LiDAR scanning pattern for a target object by controlling the driving current of the laser diode (LD) to emit a light pulse and controlling the rotation speed and scan angle of the scanner (360).

[0077] Additionally, the scanner / laser drive unit (313) receives rotational speed and angular position information fed back from the scanner (360) and can correct the accurate irradiation position of the lidar beam in real time. Through this, a constant scanning pattern and distance accuracy can be maintained despite changes in the vehicle's driving speed or vibration.

[0078] The internal memory (314) is a storage unit for temporarily storing computation data performed within the SoC (310) and can serve as a data buffer between the signal processing unit (312) and the point cloud generation unit (315).

[0079] The internal memory (314) can be configured as static random access memory (SRAM) and can minimize latency in real-time signal processing by storing computational data that requires high-speed access.

[0080] Additionally, the internal memory (314) can temporarily store distance and intensity information of each reflection point when generating frame-unit point cloud data, and support transmission to the auxiliary chip (320) or external memory (330) after the calculation is completed.

[0081] For example, the internal memory (314) can maintain data continuity and improve computational efficiency by storing intermediate data between the peak data of the reflected signal detected by the signal processing unit (312) and the distance calculation result of the point cloud generation unit (315) in a cache form.

[0082] The point cloud generation unit (315) receives preprocessed data from the signal processing unit (312) and can generate a point cloud including distance, intensity, and angle information of each reflection point. For example, the point cloud generation unit (315) can integrate multi-frame data to form a point distribution in three-dimensional space and transmit it to a vehicle control module (340) or an external processor.

[0083] The auxiliary chip (320) can perform additional post-processing operations based on point cloud data generated from the SoC (310). For example, the auxiliary chip (320) can improve the reliability of LiDAR recognition by performing functions such as generating a clean point cloud, blockage detection, or object classification.

[0084] In addition, the auxiliary chip (320) can be linked with external memory (330) to perform large-scale operations on a frame-by-frame basis.

[0085] The external memory (330) can be connected to the auxiliary chip (320) to store data to support additional functions. For example, the external memory (330) can be composed of high-speed memory such as dynamic random access memory (DRAM) or GDDR (GDDR6, etc.) and can be used to buffer point cloud frame data or temporarily store intermediate post-processing results.

[0086] The vehicle control module (340) performs communication between the LiDAR system and the vehicle control unit and can be linked with the vehicle's driving control or safety system. For example, the vehicle control module (340) supports an AUTOSAR (Automotive Open System Architecture)-based software platform so that it can be efficiently linked with the vehicle's control network.

[0087] The optical receiver (350) can receive light reflected from a target object, generate an electrical signal (light pulse) proportional to its intensity, and transmit it to the TDC / ADC (311) of the SoC (310). For example, the optical receiver (350) may be composed of a high-sensitivity light receiving element such as a photodiode (PD), an Avalanche Photodiode (APD), or a Single Photon Avalanche Diode (SPAD).

[0088] The scanner (360) can control the scanning path of the LiDAR light beam and provide rotational speed and angular position information to the scanner / laser driver (313) of the SoC (310). That is, the scanner (360) can precisely control the scanning pattern by scanning the laser beam at a specific angle according to the driving signal provided from the SOC (310) and transmitting actual scanning angle feedback information back to the SOC (310). For example, the scanner (360) may include a mechanical rotating device based on a MEMS mirror, a polygon mirror, or an optical prism.

[0089] However, in a lidar system (300) configured in this way, the main chip, the system-on-chip (SoC, 310), performs calculations by including a time-to-digital converter (TDC) or an analog-to-digital converter (ADC), signal processing logic, and a central processing unit (CPU) for generating point clouds, so there may be a problem that optimization according to the application is difficult.

[0090] For example, if a chip designed for long-distance (approx. 240 meters) measurement is used for short-distance (approx. 30 meters) measurement, the signal processing area needs to be lightweighted, but cost reduction is difficult because the signal processing logic is configured in the form of a fixed logic circuit. In addition, if a separate chip is manufactured to suit the application purpose, the entire main chip must be redesigned, which can lead to an excessive increase in development costs and time.

[0091] In addition, existing SoCs are not equipped with dynamic random access memory (DRAM) to perform large-scale frame-based computations, which limits their ability to respond to additional customer requirements for features such as Clean Point Cloud generation or Blockage Detection.

[0092] Accordingly, a lidar system including a fixed part and a variable part according to the present invention will be described in more detail below.

[0093] FIG. 5 is a drawing for explaining a lidar system including a fixed part and a variable part according to an embodiment of the present invention.

[0094] A lidar system (400) according to one embodiment of the present invention may include a fixed part (420), a variable part (410), an external memory (430), an optical receiver (450), and a scanner (460). In the lidar system (400) according to one embodiment of the present invention, some circuit configurations may be changed depending on the application environment (e.g., long distance / short distance), and the variable part (410) and the fixed part (420) may be configured as independent modules to improve the scalability and application flexibility of the system.

[0095] A lidar system (400) according to one embodiment of the present invention may include a variable unit (410), a fixed unit (420), an external memory (430), an optical receiver (450), and a scanner (460). The lidar system (400) can implement an efficient and flexible signal processing structure by converting an optical signal into an electrical signal at the variable unit (410), pre-processing it, and then transmitting it to the fixed unit (420) to perform high-level data analysis and post-processing.

[0096] A variation unit (410) according to one embodiment of the present invention is a base LiDAR chip configuration for processing a received signal of an optical pulse, and is a LiDAR chip configuration that performs core signal conversion and driving of the LiDAR. For example, the variation unit (410) can be implemented as a Field Programmable Gate Array (FPGA) or an Application Specific Integrated Circuit (ASIC), and operates based on the same hardware regardless of the application environment, thereby maintaining the basic performance of the LiDAR system at a constant level.

[0097] In addition, the variable unit (410) according to one embodiment of the present invention may be configured to exclude a computational processing structure, for example, a CPU. Specifically, separate processing through a CPU may be required to generate a point cloud, and the module for generating the point cloud is not included in the fixed unit.

[0098] A variation unit (410) according to one embodiment of the present invention may include a time-to-digital converter (TDC) / analog-to-digital converter (ADC) (411), a signal processing unit (412), a scanner / laser driving unit (413), and an internal memory (414). The aforementioned components are responsible for converting, pre-processing, laser driving, and data buffering of the light reception signal, thereby ensuring the efficiency and quality of the data transmitted to the variation unit (420).

[0099] In addition, the variable unit (410) must understand that it may adopt the configuration of the aforementioned lidar system (300) as needed.

[0100] A fixed unit (420) according to one embodiment of the present invention is a chip configuration that performs fixed control and computation functions of a LiDAR system and may include a vehicle control unit (421), a point cloud generation unit (422), and an auxiliary function unit (423). The fixed unit (420) operates on the same hardware platform regardless of the application environment and performs high-level computations, including point cloud generation, using preprocessed data received from the variable unit (410).

[0101] The TDC / ADC (411) included in the variation unit (410) according to one embodiment of the present invention can convert a reflected light signal input from an optical receiver (450) into electrical time data and convert it into a digital signal. That is, by calculating the time of flight of the transmitted laser pulse and the reflected light, it can provide basic data for calculating distance information to a target object.

[0102] In one embodiment of the present invention, the TDC / ADC (411) has nanosecond (ns) precision based on a high-speed clock and can perform low-power, high-speed conversion within the variation unit (410).

[0103] A signal processing unit (412) included in a variation unit (410) according to one embodiment of the present invention receives digital data converted from a TDC / ADC (411) and can perform preprocessing operations such as noise removal, filtering, peak detection, and threshold comparison. In one embodiment of the present invention, the signal processing unit (412) is implemented as logic hardware (e.g., FPGA circuit) to reduce the CPU-based computational burden and significantly reduce the amount of data transmitted to the variation unit (420). For example, raw data at a level of 16.95 Gbps can be compressed to a level of about 0.2 Gbps, thereby minimizing interface costs (number of LVDS channels).

[0104] A scanner / laser driving unit (413) included in a variation unit (410) according to one embodiment of the present invention can control the emission of a laser beam and the driving of a scanner (460). The scanner / laser driving unit (413) can control the driving current of a laser diode to generate a light pulse and control the rotation speed and scanning angle of the scanner (460) to adjust the object recognition range. In addition, by receiving position feedback from the scanner (460) and correcting the scanning pattern in real time, stable distance measurement can be maintained even with vibrations or environmental changes while driving.

[0105] An internal memory (414) according to one embodiment of the present invention can temporarily store intermediate data generated by a signal processing unit (412) and buffer data required for generating a point cloud.

[0106] In one embodiment of the present invention, the internal memory (414) is configured as static random access memory (SRAM) to provide high-speed access and prevent real-time computation delay. For example, the internal memory (414) stores frame-by-frame reflection point data, intermediate data for distance calculation, and timestamp information, thereby enabling stable transmission to the fixed unit (420).

[0107] A vehicle control unit (421) according to one embodiment of the present invention can perform communication and control functions with a vehicle. For example, the vehicle control unit (421) can be implemented by a structure based on a vehicle control software architecture standard, and more specifically, by supporting a standardized vehicle interface based on AUTOSAR (Automotive Open System Architecture), it can be linked with driving control, brake control, collision avoidance systems, etc. In addition, the vehicle control unit (421) can increase the stability of the system by transmitting the status of the LiDAR sensor to the vehicle ECU in real time.

[0108] A point cloud generation unit (422) according to one embodiment of the present invention can generate a point cloud based on preprocessed data transmitted from a variation unit (410). That is, it can calculate three-dimensional point data by integrating distance, intensity, and angle information of a reflected signal, and can be utilized for object recognition or distance mapping.

[0109] In one embodiment of the present invention, the point cloud generation unit (422) is composed of a CPU or GPU-based application processor (TI TDA4, etc.) and can perform large-scale frame-unit computations using DRAM.

[0110] An auxiliary function unit (423) according to one embodiment of the present invention can communicate with an external memory (430) to perform advanced functions such as generating a clean point cloud or blockage detection. Additionally, the auxiliary function unit (423) can perform advanced functions requiring high data communication capacity through communication with the external memory (430).

[0111] For example, the auxiliary function unit (423) can increase the reliability of the lidar measurement by removing outliers in the point cloud or detecting lens contamination and shielding conditions.

[0112] An external memory (430) according to one embodiment of the present invention is connected to a fixed part (420) and can store frame-unit point cloud data and post-processing intermediate results.

[0113] In one embodiment of the present invention, the external memory (430) may be composed of dynamic random access memory (DRAM) or graphics memory (GDDR) and may be utilized as a data buffer for large-scale parallel computation. Through this, the point cloud generation unit (422) and the auxiliary function unit (423) of the fixed unit (420) can stably perform real-time large-scale computation.

[0114] A light receiver (450) according to one embodiment of the present invention receives light reflected from a target object, generates an electrical signal (light pulse) proportional to the intensity of the received light, and transmits it to the TDC / ADC (411) of the variation unit (410).

[0115] In one embodiment of the present invention, the optical receiver (450) may include a high-sensitivity element such as an avalanche photodiode (APD) or a single-photon avalanche diode (SPAD).

[0116] A scanner (460) according to one embodiment of the present invention is a driving device for scanning a laser beam in a predetermined pattern and can rotate or vibrate according to a control signal from a scanner / laser driving unit (413) of a variable unit (410). In addition, the scanner (460) can maintain the scanning pattern precisely by feeding back its angular position and rotation speed to a fixed unit (410). For example, the scanner (460) may include a polygon mirror, a MEMS mirror, or a prism-based optical element.

[0117] As described above, compared to conventional technology, the LiDAR system (400) according to the present invention can expand functionality by changing only the signal processing unit according to the application environment (long distance / short distance, etc.) and can significantly reduce development costs and time since there is no need to redesign the entire main chip. In addition, since the fixed unit (420) is linked with an external memory (430) including DRAM, it can perform advanced functions such as generating clean point clouds and detecting shielding, thereby improving the recognition accuracy and reliability of the system. Since the data transmission bandwidth between the variable unit (410) and the fixed unit (420) is low (e.g., about 0.2 Gbps level), communication is possible through a single LVDS interface, so the increase in costs due to interface configuration can be minimized.

[0118] Figure 6 is a diagram illustrating examples of input and output data of the lidar system of Figure 5.

[0119] In one embodiment of the present invention, the variation unit (410) includes a time-to-digital converter (TDC) / analog-to-digital converter (ADC) (411) and a signal processing unit (412) to convert raw data (TDC Data) input from an optical receiver into a digital signal and preprocess the data. At this time, the bandwidth of the input TDC data is very large, approximately 16.95 Gbps, and the amount of data can be compressed to approximately 0.2 Gbps by removing unnecessary signal components through preprocessing operations such as filtering, peak detection, and threshold comparison performed in the signal processing unit (412).

[0120] The preprocessed data is transmitted to the fixed unit (420) and undergoes post-processing operations in the point cloud generation unit (422). The fixed unit (420) calculates distance, intensity, and angle information of reflection points based on the preprocessed data to generate a 3D point cloud and can convert it into output data (Processed Data) of approximately 0.093 Gbps. That is, the LiDAR system of the present invention achieves a data reduction rate of approximately 99.45% or more compared to the input, thereby maximizing computational efficiency and reducing overall system bandwidth and power consumption.

[0121] Accordingly, a LiDAR system according to one embodiment of the present invention performs high-speed preprocessing in a variable unit (410) including a TDC / ADC and a signal processing unit, and performs final calculations in a fixed unit (420) including a point cloud generation unit (422), thereby minimizing the amount of data transmitted between the fixed unit and the variable unit. As a result, sufficient data transmission is possible with only a single LVDS interface, which can reduce interface costs and design complexity, and enable the realization of an efficient structure tailored to various application environments (long distance / short distance).

[0122] FIG. 7 is a drawing for explaining an example of a lidar system including a fixed part and a variable part according to an embodiment of the present invention.

[0123] A LiDAR system (500) including a fixed unit and a variable unit according to one embodiment of the present invention may include a variable unit (510), a fixed unit (520), an external DDR memory (530), and an ADAS / AD system (Advanced Driver Assistance System / Autonomous Driving System) (540). Specifically, the variable unit (510) converts an analog signal from a photodetector through a TDC to generate raw data with high bandwidth, performs preprocessing through a statistical signal processing unit, and then transmits low-bandwidth processed data in a format that can be transmitted via one channel of MIPI (Mobile Industry Processor Interface) or LVDS (Low Voltage Differential Signaling) in real time to the fixed unit (520). The fixed unit (520) performs determined logic regarding spatial signal processing and functional safety, and can transmit the point cloud generated through this to the ADAS / AD system (540).

[0124] A variation unit (510) according to one embodiment of the present invention can perform acquisition and preprocessing of low-level data or raw data of a LiDAR sensor. The variation unit (510) can be designed based on an FPGA or ASIC. Additionally, the variation unit (510) can communicate with signals received from a photodetector, control / feedback signals of a laser, control / feedback signals of a motor, and a sensor SPI interface, and can generate processed data including distance, depth, and intensity and transmit it to a fixed unit. The output data can be transmitted via an LVDS or MIPI serial interface. Additionally, I2C and / or GPIO lines may be separately provided between the variation unit and the fixed unit to perform command setting and status feedback as a low-speed control channel.

[0125] A fixed unit (520) according to one embodiment of the present invention can perform spatial recognition and system control based on processing data input from a variable unit. Specifically, it can receive depth images and reflection intensity images received through an LVDS or MIPI interface, information related thereto, and control commands through an I2C / GPIO line, and output a 3D point cloud, object information, vehicle control signals, and ADAS / AD system (540) linkage data.

[0126] An external DDR memory according to one embodiment of the present invention is connected to the DDR control unit (523) of the fixed part as an external memory and can be used as a buffer space for storing large-capacity point cloud data and object information generated during the spatial signal processing process. In addition, it can also be utilized as a storage space for performing various auxiliary functions that the system (500) according to the present invention can provide.

[0127] Communication between the variable unit (510) and the fixed unit (520) according to one embodiment of the present invention can be performed in two ways. First, to transmit data, high-speed processed data (Processed Data) can be streamed in real time through an LVDS or MIPI serial interface. Here, LVDS 1 channel can be implemented with 12 differential pairs and has the characteristics of being resistant to EMI noise and having low transmission delay. Next, relatively low-speed control commands, status information, and error signals can be exchanged through an I2C or GPIO line as a channel for performing communication regarding control / status. For example, the fixed unit (520) can calculate the laser output level of the variable unit (510), the speed of the scanner, TDC gain, etc., using an I2C communication channel, and the variable unit (510) can report errors or temperature status, etc., through GPIO.

[0128] A variation unit (510) according to one embodiment of the present invention may include a TDC (511), a laser control unit (512), a scanner control unit (513), an SPI logic unit (514), a data path (515), a statistical signal processing unit (516), a packet generation unit (517), and a register interface (518).

[0129] A TDC (511) according to one embodiment of the present invention can convert a received signal input from a photodetector into digital time data. More specifically, it may include a circuit that measures the time difference between a received signal input from a photodetector and a transmission trigger signal and converts it into digital time data. A laser control unit (512) according to one embodiment of the present invention can control a driving signal of a transmitting laser diode, stabilize the output, and perform control. In addition, a scanner control unit (513) according to one embodiment of the present invention can control the scanning angle and transmit and receive signals to control a rotating motor.

[0130] According to one embodiment of the present invention, the SPI logic unit (514) may be composed of an internal logic circuit that communicates with an external sensor based on SPI (Serial Peripheral Interface) (e.g., temperature sensor, current monitor, power IC, etc.) to collect data regarding various states and transmit control commands. It may receive SPI signals from the external sensor (e.g., MOSI, MISO, SCLK, CS, etc.), interpret the SPI protocol, and execute commands. Information regarding the sensor state may be transmitted to a statistical signal processing unit for processing, and information regarding such state may be summarized and transmitted to a register interface (518).

[0131] A data path (515) according to one embodiment of the present invention may be a path for performing processing, such as aligning high-bandwidth raw data output from the TDC (511) according to time order and spatial coordinates. The data or data stream aligned in this way may be transmitted to a statistical signal processing unit (516) to perform processing.

[0132] A statistical signal processing unit (516) according to one embodiment of the present invention can remove noise, calculate distance, and calculate reflection intensity and reliability using information input from a data path (515). Specifically, it can remove noise by applying a noise suppression algorithm, calculate a value of distance or depth, calculate reflection intensity through the integral value of pulse intensity, and statistically process the reliability of each point.

[0133] A packet generation unit (517) according to one embodiment of the present invention can bundle the output data of a statistical signal processing unit (516) into frames and packetize it into a data format that can be transmitted to a fixed unit (520). For example, it can receive point data, frame synchronization information from a laser control unit (512), and format setting information from a register interface (518), and configure them together with a header of each frame data to convert them according to LVDS or MIPI interface specifications. In addition, the data transmitted from the packet generation unit (517) to the fixed unit (520) may include a depth image and an intensity image.

[0134] A register interface (518) according to one embodiment of the present invention can perform the role of relaying control and status signals between a fixed unit (520) and a variable unit (510). For example, it can receive a control command from the fixed unit (520) via an I2C or GPIO channel, receive register information regarding the status of modules included in the variable unit (510), interpret the control command, transmit it to each module, and return an internal status register according to a request from the fixed unit (520). As a more specific example, it can reflect the laser driving frequency, scanner scanning speed, TDC gain, etc. in real time according to a request from the fixed unit (520).

[0135] A fixed part (520) according to one embodiment of the present invention may include a vehicle control and functional safety control part (521), a spatial signal processing part (522), and a DDR control part (523).

[0136] According to one embodiment of the present invention, the vehicle control and functional safety control unit (521) can receive information regarding settings and status via I2C from the register interface (518) of the variation unit (510). Additionally, the vehicle control and functional safety control unit (521) can transmit a control signal regarding safety to the spatial signal processing unit (522) and can generate a signal to control various functions within the vehicle. Furthermore, the vehicle control and functional safety control unit (521) may be configured with a dual real-time processor to perform the above-described functions and can perform monitoring and inspection at 150 Hz. Additionally, the vehicle control and functional safety control unit (521) can perform control and functional safety control through a vehicle control program such as AUTOSAR.

[0137] A spatial signal processing unit (522) according to one embodiment of the present invention can receive depth, intensity, and confidence frames received via LVDS or MIPI and generate a point cloud. In addition, to perform point cloud generation and auxiliary functions, it can perform functions such as preprocessing / filtering, alignment between frames, object detection and tracking, and output reduction. Furthermore, to perform the aforementioned functions, it can utilize previously recorded frames recorded in a DDR control unit (523) and an external DDR memory (530).

[0138] A spatial signal processing unit (522) according to one embodiment of the present invention may include a quad application processor, and the quad application processor may be a processor operating at a clock speed of 1000 with a 4-core CPU based on the ARM Cortex-A53 architecture.

[0139] The above-described embodiments according to the present invention may be implemented in a manner that is implemented with the above-described components, and may be implemented in the form of a computer program that can be executed through various components on a computing device and a processor, and such a computer program and method may be recorded on a computer-readable medium. In this case, the medium may include a magnetic medium such as a hard disk, a floppy disk and a magnetic tape, an optical recording medium such as a CD-ROM and a DVD, a magneto-optical medium such as a floptical disk, and a hardware device specifically configured to store and execute program instructions, such as a ROM, RAM, a flash memory, etc.

[0140] Although the detailed description of the invention above has been given with reference to preferred embodiments of the invention, those skilled in the art should understand that various modifications and changes can be made to the invention without departing from the spirit and scope of the invention as described in the claimed patent claims.

Claims

1. In LiDAR systems, A lidar device comprising a light emitter that emits light and a light detector that detects light; A variation processor unit that converts and preprocesses signals input from the above-mentioned lidar device; A fixed processor unit that receives preprocessed data from the above-mentioned variable processor unit and generates a point cloud; and It includes an external memory connected to the above fixed processor unit, and The above-mentioned variable processor unit is a system in which the circuit configuration can be changed according to the detection distance or application environment of the lidar device.

2. In Paragraph 1, The above fluctuation processor unit is, A system comprising a TDC (Time to Digital Converter) or an ADC (Analog to Digital Converter) that converts time information of a signal received from the photodetector of the lidar device into digital data.

3. In Paragraph 2, The above fluctuation processor unit is, A system comprising a signal processing unit that receives digital data converted from the above TDC or the above ADC and performs a preprocessing operation.

4. In Paragraph 1, The above fluctuation processor unit is, It further includes a scanner / laser driver that controls the driving of the scanner and the laser diode, and A system in which the above scanner / laser drive unit controls the rotation speed and scanning angle of the scanner to adjust the scanning pattern.

5. In Paragraph 1, The above fixed processor unit is, A system comprising a point cloud generation unit that generates a point cloud including distance and depth using the preprocessed data transmitted from the above-mentioned fluctuation processor unit.

6. In Paragraph 1, The above fixed processor unit is, A system further comprising an auxiliary function unit that performs a Clean Point Cloud or Blockage Detection using the data of the above point cloud.

7. In Paragraph 6, The above auxiliary function unit communicates with external memory to perform large-scale computations, a system.

8. In Paragraph 7, The above external memory is composed of dynamic random access memory (DRAM) and stores frame data of the fixed processor unit and buffers intermediate results of the generation of the clean point cloud, a system.

9. In Paragraph 1, The above fluctuation processor unit is, It further includes internal memory for temporarily storing the preprocessing results of the signal processing unit, and A system in which the internal memory is composed of static random access memory (SRAM).

10. In Paragraph 1, The above fixed processor unit and the above variable processor unit are, A system that transmits and receives data via an LVDS (Low Voltage Differential Signaling) interface.

11. In Paragraph 10, A system in which the above LVDS interface is configured as a single interface and supports the transmission of the above preprocessed data of the above fluctuation processor unit.

12. In Paragraph 1, The above fluctuation processor unit is, Includes additional register interfaces, The above register interface is a system that performs communication with a vehicle control unit included in the above fixed processor unit.

13. In Paragraph 1, The above vehicle control unit is a system implemented in a structure based on vehicle control software architecture standards.

14. In a method for processing signals of a LiDAR system, A step of receiving reflected light with a photodetector included in a lidar device; In the fluctuation processor unit, a step of converting and preprocessing the signal input from the lidar device; A step of generating a point cloud by receiving preprocessed data from the variable processor unit in a fixed processor unit; and The method includes the step of storing the point cloud data or performing auxiliary operations using an external memory connected to the fixed processor unit. The above-mentioned variation processor unit is a method in which the circuit configuration can be changed according to the detection distance or application environment of the lidar device.

Images