Lidar sensor based on indirect time-of-flight sensor and control method thereof

Abstract

A Lidar sensor includes: a light emitting unit; comprising a plurality of pixels; and one or more processors for outputting light of a first frequency and light of a second frequency greater than the first frequency through the light emitting unit when the Lidar sensor is rotated, and if the output light of the first frequency and the output light of the second frequency are reflected from an object and received by a plurality of pixels, outputting the light of the first frequency and the light of the second frequency. Acquiring a first pixel value based on the reflected light of the first frequency and acquiring a second pixel value based on the reflected light of the second frequency, identifying a third pixel value from the first pixel value using a reliability level of the first pixel value, a plurality of fourth pixel values are identified from the second pixel values using a reliability level of the second pixel values determined based on a size of the second pixel values, and a distance between the Lidar sensor and the object is identified.

Description

Technical Field

[0001] Apparatuses and methods consistent with the present disclosure relate to a Lidar sensor and a control method thereof, and more particularly, to a Lidar sensor for performing a function of recognizing a distance to an object using an iToF sensor and a control method thereof. Background Art

[0002] Recently, Lidar sensors are being used in various fields. In particular, Lidar sensors can detect objects around a user and recognize the distance between the object and the user, thereby preventing accidents that the user is not aware of in advance, and also realize autonomous driving of various electronic devices.

[0003] In addition, the ToF sensor is used in the Lidar sensor to identify the distance of the object. The ToF sensor can be divided into a direct time-of-flight (dToF) sensor and an indirect time-of-flight (iToF) sensor according to its operation method. The Lidar sensor based on the dToF sensor identifies the distance between the object and the Lidar sensor based on the flight time required for the light output from the Lidar sensor to be reflected by the object and received. In this case, the dToF sensor requires a single photon avalanche diode (SPAD) device for high-speed measurement of the flight time of light, which creates a problem that the manufacturing cost of the Lidar sensor becomes expensive. In addition, for long-distance measurement, the size of the plano-convex lens of the dToF sensor that collects the light reflected by the object needs to be larger, which leads to the problem that the manufacturing cost of the Lidar sensor becomes expensive and another problem that the Lidar sensor based on the dToF sensor may not be mounted on a small electronic device. Summary of the invention

[0004] Technical Solution According to one aspect of the present disclosure, a Lidar sensor includes a light emitting unit; an indirect time-of-flight (iToF) sensor including a plurality of pixels; and one or more processors configured to: output light of a first frequency and light of a second frequency greater than the first frequency through the light emitting unit when the Lidar sensor rotates, based on the outputted light of the first frequency and the outputted light of the second frequency being reflected by an object and received by the plurality of pixels, acquire a plurality of first pixel values ​​based on the reflected light of the first frequency, and acquire a plurality of second pixel values ​​based on the reflected light of the second frequency, identify a plurality of third pixel values ​​from the plurality of first pixel values ​​using reliability levels of the plurality of first pixel values ​​determined based on sizes of the plurality of first pixel values, identify a plurality of fourth pixel values ​​from the plurality of second pixel values ​​using reliability levels of the plurality of second pixel values ​​determined based on sizes of the plurality of second pixel values, and identify a distance between the Lidar sensor and the object based on at least one of a first distance value and a second distance value, the first distance value being acquired based on the plurality of third pixel values, and the second distance value being acquired based on the plurality of fourth pixel values.

[0005] The one or more processors may also be configured to: based on the reliability levels of the multiple first pixel values, identify the multiple third pixel values, each having a reliability level greater than or equal to a first preset value, from the multiple first pixel values; and based on the reliability levels of the multiple second pixel values, identify the multiple fourth pixel values, each having a reliability level greater than or equal to a second preset value, from the multiple second pixel values, and wherein, as the size of the pixel values ​​in the multiple first pixel values ​​and the multiple second pixel values ​​increases, each of the reliability levels of the multiple first pixel values ​​and the reliability levels of the multiple second pixel values ​​is higher.

[0006] The one or more processors may be further configured to obtain the plurality of filtered pixels by applying a filter having a preset size to the plurality of pixels, and identify the plurality of third pixel values ​​and the plurality of fourth pixel values ​​based on pixel values ​​of the plurality of filtered pixels.

[0007] The one or more processors may also be configured to: identify multiple distance values ​​based on multiple third pixel values, and identify a first distance value based on an average of the identified multiple distance values, and identify multiple distance values ​​based on multiple fourth pixel values, and identify a second distance value based on the average of the identified multiple distance values.

[0008] The one or more processors may also be configured to: identify a distance value corresponding to a maximum reliability level among a plurality of distance values ​​identified based on the plurality of third pixel values ​​as a first distance value, and identify a distance value corresponding to a maximum reliability level among a plurality of distance values ​​identified based on the plurality of fourth pixel values ​​as a second distance value.

[0009] The one or more processors may also be configured to: determine a compensation value based on the first distance value and the second frequency based on a difference between the first distance value and the second distance value being greater than or equal to a preset value, compensate the second distance value based on the determined compensation value, and identify the compensated second distance value as the distance between the Lidar sensor and the object.

[0010] The one or more processors may be further configured to identify the second distance value as a distance between the Lidar sensor and the object based on a difference between the first distance value and the second distance value being less than a preset value.

[0011] The one or more processors may be further configured to: output light of a first frequency and a second frequency at each preset time interval to identify the distance between the Lidar sensor and the object, and based on the difference between the distance identified at the first time and the distance identified at the second time in the distance identified at each preset time interval being greater than or equal to a preset value, identify the distance between the Lidar sensor and the object at the second time based on the first distance value, and wherein the second time is a time after the preset time interval from the first time.

[0012] The multiple first pixel values ​​corresponding to one of the multiple pixels may include a first value and a second value, and the multiple second pixel values ​​corresponding to one of the multiple pixels may include a third value and a fourth value, and the one or more processors may also be configured to identify each of the reliability levels of the multiple first pixel values ​​and the reliability levels of the multiple second pixel values ​​based on the following equations, respectively.

[0013] Reliability level of the first pixel value = Reliability level of the second pixel value = Each of the plurality of pixels may include a plurality of receivers configured to be activated at preset time intervals.

[0014] According to one aspect of the present disclosure, a method for controlling a Lidar sensor including an indirect time-of-flight (iToF) sensor includes: outputting light of a first frequency and light of a second frequency greater than the first frequency through a light emitting unit of the Lidar sensor when the Lidar sensor rotates; receiving light of the first frequency and light of the second frequency reflected by an object through a plurality of pixels included in the iToF sensor; acquiring a plurality of first pixel values ​​based on the reflected light of the first frequency, and acquiring a plurality of second pixel values ​​based on the reflected light of the second frequency; identifying a plurality of third pixel values ​​from the plurality of first pixel values ​​using reliability levels of the plurality of first pixel values ​​determined based on sizes of the plurality of first pixel values; identifying a plurality of fourth pixel values ​​from the plurality of second pixel values ​​using reliability levels of the plurality of second pixel values ​​determined based on sizes of the plurality of second pixel values; and identifying a distance between the Lidar sensor and the object based on at least one of a first distance value and a second distance value, the first distance value being acquired based on the plurality of third pixel values, and the second distance value being acquired based on the plurality of fourth pixel values.

[0015] Identifying the plurality of third pixel values ​​may include: based on the reliability levels of the plurality of first pixel values, identifying the plurality of third pixel values ​​each having a reliability level greater than or equal to a first preset value from the plurality of first pixel values. Identifying the plurality of fourth pixel values ​​may include: based on the reliability levels of the plurality of second pixel values, identifying the plurality of fourth pixel values ​​each having a reliability level greater than or equal to a second preset value from the plurality of second pixel values, and each of the reliability levels of the plurality of first pixel values ​​and the reliability levels of the plurality of second pixel values ​​may be higher as the size of the pixel values ​​in the plurality of first pixel values ​​and the plurality of second pixel values ​​increases.

[0016] The method may also include obtaining a plurality of filtered pixels by applying a filter having a preset size to the plurality of pixels, identifying the plurality of third pixel values ​​may include identifying the plurality of third pixel values ​​based on first pixel values ​​of the plurality of filtered pixels, and identifying the plurality of fourth pixel values ​​may include identifying the plurality of fourth pixel values ​​based on second pixel values ​​of the plurality of filtered pixels.

[0017] Identifying the distance may include: identifying multiple distance values ​​based on the multiple third pixel values, and identifying a first distance value based on an average of the identified multiple distance values; and identifying multiple distance values ​​based on the multiple fourth pixel values, and identifying a second distance value based on the average of the identified multiple distance values.

[0018] Identifying the distance may include: identifying a distance value corresponding to a maximum reliability level among a plurality of distance values ​​identified based on the plurality of third pixel values ​​as a first distance value, and identifying a distance value corresponding to a maximum reliability level among a plurality of distance values ​​identified based on the plurality of fourth pixel values ​​as a second distance value.

[0019] Identifying the distance may include: determining a compensation value based on the first distance value and the second frequency based on a difference between the first distance value and the second distance value being greater than or equal to a preset value, compensating the second distance value based on the determined compensation value, and identifying the compensated second distance value as the distance between the Lidar sensor and the object.

[0020] Identifying the distance may include identifying the second distance value as the distance between the Lidar sensor and the object based on a difference between the first distance value and the second distance value being less than a preset value.

[0021] Outputting light of a first frequency and light of a second frequency may include: outputting light of the first frequency and light of the second frequency at each preset time interval to identify the distance between the Lidar sensor and the object, identifying the distance may include: based on the difference between the distance identified at the first time and the distance identified at the second time in the distance identified at each preset time interval being greater than or equal to a preset value, identifying the distance between the Lidar sensor and the object at the second time based on the first distance value, the second time may be a time after the preset time interval from the first time.

[0022] The multiple first pixel values ​​corresponding to one of the multiple pixels may include a first value and a second value, and the multiple second pixel values ​​corresponding to one of the multiple pixels include a third value and a fourth value, and the method may also include determining a reliability level of the multiple first pixel values ​​and each of the reliability levels of the multiple second pixel values ​​based on the following equations, respectively.

[0023] Reliability level of the first pixel value = Reliability level of the second pixel value = According to one aspect of the present disclosure, a non-transitory computer-readable recording medium stores instructions, the instructions causing a lidar sensor including an indirect time-of-flight (iToF) sensor to perform a method of controlling the lidar sensor, and the method includes: outputting light of a first frequency and light of a second frequency greater than the first frequency through a light emitting unit of the lidar sensor when the lidar sensor rotates; receiving light of the first frequency and light of the second frequency reflected by an object through a plurality of pixels included in the iToF sensor; acquiring a plurality of first pixel values ​​based on the reflected light of the first frequency, and acquiring a plurality of second pixel values ​​based on the reflected light of the second frequency; identifying a plurality of third pixel values ​​from the plurality of first pixel values ​​using reliability levels of the plurality of first pixel values ​​determined based on sizes of the plurality of first pixel values; identifying a plurality of fourth pixel values ​​from the plurality of second pixel values ​​using reliability levels of the plurality of second pixel values ​​determined based on sizes of the plurality of second pixel values; and identifying a distance between the lidar sensor and the object based on at least one of a first distance value and a second distance value, the first distance value being acquired based on the plurality of third pixel values, and the second distance value being acquired based on the plurality of fourth pixel values. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The above and / or other aspects of the present disclosure will become more apparent by describing certain embodiments of the present disclosure with reference to the accompanying drawings, in which: Figure 1 is a diagram for schematically describing a Lidar sensor according to an embodiment of the present disclosure; Figure 2 is a schematic block diagram of a Lidar sensor according to an embodiment of the present disclosure; Figure 3 is a diagram for describing a plurality of pixels included in an iToF sensor according to an embodiment of the present disclosure; Figure 4 is a diagram showing a plurality of pixel values ​​acquired from each of a plurality of pixels according to an embodiment of the present disclosure; Figure 5 is a diagram showing a first pixel value corresponding to 5 MHz light and a second pixel value corresponding to 100 MHz light according to an embodiment of the present disclosure; Figure 6 is a diagram illustrating selecting a plurality of third pixel values ​​based on a first reliability level and selecting a plurality of fourth pixel values ​​based on a second reliability level according to an embodiment of the present disclosure; Figure 7 is a diagram for describing filtering a plurality of pixels by applying a filter having a preset size to the plurality of pixels according to an embodiment of the present disclosure; Figure 8is a diagram for identifying a first distance value and a second distance value among a plurality of distance values ​​corresponding to a plurality of first pixels and a plurality of distance values ​​corresponding to a plurality of second pixels according to an embodiment of the present disclosure; Fig. 9 is a diagram illustrating a method of adjusting a distance between a Lidar sensor recognized at a relatively late time and an object when a distance difference between the Lidar sensor and the object recognized at adjacent times is greater than or equal to a preset value according to an embodiment of the present disclosure; Fig.10 is a diagram for describing a method of recognizing distances between a plurality of points of an object and a Lidar sensor based on a plurality of pixels classified into a plurality of pixel groups according to an embodiment of the present disclosure; Fig.11 is a flow chart schematically illustrating a method for controlling a Lidar sensor based on an iToF sensor according to an embodiment of the present disclosure; Fig.12 is a flowchart schematically illustrating a method for controlling a Lidar sensor based on an iToF sensor according to an embodiment of the present disclosure; and Fig.13 is a schematic block diagram of a robot including a Lidar sensor based on an iToF sensor according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0025] After briefly describing the terms used in the specification, the present disclosure will be described in detail.

[0026] Considering the functions in the present disclosure, currently widely used general terms are selected as the terms used in the embodiments of the present disclosure, but may be changed according to the intention of those skilled in the art or the emergence of judicial precedents, new technologies, etc. In addition, in certain cases, there may be terms arbitrarily selected by the applicant. In this case, the meanings of these terms will be mentioned in detail in the corresponding description parts of the present disclosure. Therefore, the terms used in the present disclosure should be defined based on the meanings of the terms and the content throughout the present disclosure rather than the simple names of the terms.

[0027] In the present disclosure, the expressions “having”, “may have”, “including”, “may include”, etc. indicate the existence of corresponding features (eg, values, functions, operations, components such as parts, etc.), and do not exclude the existence of additional features.

[0028] The expression “at least one of A and / or B” should be understood to mean “A” or “B” or “any one of A and B”.

[0029] The expressions "first", "second", "1st" or "2nd" etc. used in the present disclosure may indicate various components, and regardless of the order and / or importance of the components, will only be used to distinguish one component from other components and will not limit the corresponding components.

[0030] When it is mentioned that any component (e.g., a first component) is (operably or communicatively) coupled with / coupled to another component or connected to another component (e.g., a second component), it will be understood that any component is directly coupled to another component or may be coupled to another component through other components (e.g., a third component).

[0031] Unless the context clearly indicates otherwise, the singular form is intended to include the plural form. It will be further understood that the terms "comprising" or "formed by" used in this specification specify the presence of features, numbers, steps, operations, components, parts or combinations thereof mentioned in this specification, but do not exclude the presence or addition of one or more other features, numbers, steps, operations, components, parts or combinations thereof.

[0032] In the present disclosure, a "module" or "~" may perform at least one function or operation, and may be implemented by hardware or software, or by a combination of hardware and software. In addition, in addition to a "module" or "~" that needs to be implemented by specific hardware, multiple "modules" or multiple "~" may be integrated into at least one module and implemented by at least one processor (not shown).

[0033] Hereinafter, the present disclosure will be described in detail with reference to the accompanying drawings.

[0034] Figure 1 is a diagram for schematically describing a Lidar sensor according to an embodiment of the present disclosure.

[0035] Reference Figure 1 , the Lidar sensor 100 according to an embodiment of the present disclosure recognizes a distance to an object 200 based on an indirect time of flight (iToF) sensor. Specifically, the Lidar sensor 100 recognizes a distance between the Lidar sensor 100 and the object 200 based on a phase difference between light output from the Lidar sensor 100 and light received by the iToF sensor of the Lidar sensor 100.

[0036] In the case of a conventional Lidar sensor including a direct time-of-flight (dToF) sensor, the distance between the Lidar sensor 100 and the object 200 is identified based on the time required for light output from the Lidar sensor 100 to be received by the dToF sensor. That is, the Lidar sensor 100 including the dToF sensor may calculate the flight time of light and identify the distance between the Lidar sensor 100 and the object 200 based on the calculated flight time.

[0037] In the case of such a Lidar sensor based on a dToF sensor, in order to obtain an accurate time point at which light reflected by the object 200 is received, it is necessary to modulate light output from a light emitting unit of the Lidar sensor at high speed, and it is necessary to include a single photon avalanche diode (SPAD) device for high-speed measurement. As a result, in the case of a Lidar sensor based on a dToF sensor, the manufacturing cost is expensive.

[0038] In addition, in the case of a dToF sensor, a plano-convex lens needs to be included in order to receive light reflected by the object 200. In particular, for long-distance measurement, the size of the plano-convex lens of the dToF sensor needs to be larger, which also leads to an increase in the manufacturing cost of the Lidar sensor.

[0039] Furthermore, due to the increased size of the plano-convex lens, it may be difficult to mount the dToF sensor based Lidar sensor on a small electronic device.

[0040] On the other hand, the iToF sensor does not need to include a single photon avalanche diode (SPAD) device and a plano-convex lens for high-speed measurement, and therefore can be mounted on a small electronic device more easily than a dToF sensor, and is more advantageous in terms of manufacturing cost of the Lidar sensor. Hereinafter, a Lidar sensor 100 based on an iToF sensor according to an embodiment of the present disclosure for solving the problems of the Lidar sensor based on the dToF sensor described above will be described.

[0041] Figure 2 is a schematic block diagram of a Lidar sensor according to an embodiment of the present disclosure.

[0042] According to an embodiment of the present disclosure, the Lidar sensor 100 includes a light emitting unit 110, an iToF sensor 120, and a processor 130. On the other hand, the Lidar sensor 100 does not necessarily have to include all of the above components, and some components may be omitted or added.

[0043] The light emitting unit 110 outputs modulated light toward the object 200 around the lidar sensor 100. In this case, the modulated light output from the light emitting unit 110 (hereinafter referred to as output light) may have a waveform of a square wave or a sine wave. For example, the light emitting unit 110 may output a square wave of a preset frequency in each preset period. In addition, the present disclosure is not limited thereto, and the light emitting unit 110 may have various types of waveforms.

[0044] In addition, the light emitting unit 110 may include various types of laser devices. For example, the light emitting unit 110 may include a vertical cavity surface emitting laser (VCSEL) or a laser diode. In this case, the light emitting unit 110 may include a plurality of laser devices. The plurality of laser devices may be arranged in an array.

[0045] According to an embodiment of the present disclosure, the light emitting unit 110 may output light of various frequency bands. Specifically, the light emitting unit 110 may sequentially output light of different frequency bands. For example, the light emitting unit 110 may sequentially output light of 5 MHz and light of 100 MHz. Optionally, the light emitting unit 110 may output light of different frequency bands according to the driving mode of the Lidar sensor.

[0046] The iToF sensor 120 acquires light reflected by the object 200 (hereinafter referred to as reflected light). Specifically, the iToF sensor 120 receives the reflected light output from the light emitting unit 110, reflected by the object 200, and then returned toward the Lidar sensor 100. In addition, the iToF sensor 120 may identify a phase difference between the received reflected light and the light output from the light emitting unit 110, and then identify a depth value of the object 200 based on the phase difference. Here, the depth value may be a distance value between the Lidar sensor 100 and the object 200 identified by the iToF sensor 120.

[0047] To this end, the iToF sensor 120 may be connected to the light emitting unit 110 to acquire phase information of light output from the light emitting unit 110, or may acquire phase information from the processor 130 connected to the light emitting unit 110. Based on the acquired phase information, the iToF sensor 120 may identify a phase difference between the light output from the light emitting unit 110 and the received reflected light.

[0048] Figure 3 is a diagram for describing a plurality of pixels included in an iToF sensor according to an embodiment of the present disclosure.

[0049] In addition, according to an embodiment of the present disclosure, the iToF sensor 120 may be implemented as a CCD image sensor or a CMOS image sensor. In this case, the iToF sensor 120 may include a plurality of pixels 10. Here, each of the plurality of pixels 10 may be a sensing device for receiving light reflected by the object 200.

[0050] The plurality of pixels 10 may be arranged in various forms to construct the iToF sensor 120. For example, the plurality of pixels 10 may be arranged in an N×M matrix (where N and M are natural numbers greater than or equal to 1) to implement a pixel array. Figure 3 , the plurality of pixels 10 included in the iToF sensor 120 are shown to be arranged in a 5×5 matrix form. However, this is for the convenience of describing the present disclosure, and the plurality of pixels 10 may include various numbers of pixels, and the plurality of pixels 10 may be arranged in various forms.

[0051] One or more processors (hereinafter referred to as processor 130) may control the overall operation of the Lidar sensor 100. Specifically, the processor 130 is connected to the light emitting unit 110 and the iToF sensor 120 to control the overall operation of the Lidar sensor 100. The processor 130 may include one or more processors 130. Specifically, the processor 130 may perform the operation of the Lidar sensor 100 according to various embodiments of the present disclosure by executing at least one instruction stored in a memory (not shown).

[0052] According to an embodiment of the present disclosure, the processor 130 may be implemented by a digital signal processor (DSP) that processes a digital image signal, a microprocessor, a graphics processing unit (GPU), an artificial intelligence (AI) processor, a neural processing unit (NPU), or a time controller (TCON). However, the processor 130 is not limited thereto, but may include one or more of a central processing unit (CPU), a microcontroller unit (MCU), a microprocessing unit (MPU), a controller, an application processor (AP), a communication processor (CP), and an ARM processor 130, or may be defined by these terms. In addition, the processor 130 may be implemented by a system on chip (SoC) or a large-scale integration (LSI) embedded with a processing algorithm, or may be implemented in the form of an application-specific integrated circuit (ASIC) and a field-programmable gate array (FPGA).

[0053] In addition, the Lidar sensor 100 may further include a driving unit. Specifically, the driving unit is configured to rotate the Lidar sensor 100, output light to the object 200 around the Lidar sensor 100 by rotating the Lidar sensor 100 360° in a preset direction, and receive light reflected by the object 200. To this end, the driving unit may include a motor.

[0054] In addition, the Lidar sensor 100 may further include a collecting unit for collecting the reflected light received by the iToF sensor 120. In this case, the collecting unit may include a lens, a reflector, and the like.

[0055] According to an embodiment of the present disclosure, the processor 130 may acquire a distance value corresponding to the distance between the Lidar sensor 100 and the object 200 based on a pixel value acquired from each of the plurality of pixels 10 included in the iToF sensor 120. In addition, the processor 130 may recognize the distance between the Lidar sensor 100 and the object 200 based on the acquired distance value.

[0056] More specifically, refer to Figure 3 , the pixel 10 included in the iToF sensor 120 may include a plurality of receivers that receive light reflected by the object 200. The plurality of receivers may be activated at preset time intervals and receive the reflected light at preset time intervals. Figure 3 , each of the plurality of pixels is illustrated as including two receivers, but the number of receivers included in a pixel may be variously changed according to embodiments.

[0057] Hereinafter, a receiver that first receives reflected light among a plurality of receivers included in each of a plurality of pixels is referred to as a first receiver 12. In addition, a receiver that receives reflected light after a preset time from the time when the first receiver 12 starts receiving reflected light is referred to as a second receiver 14. Figure 3 , a receiver disposed on the left side among the plurality of receivers included in each of the plurality of pixels is illustrated as a first receiver 12 , and a receiver disposed on the right side among the plurality of receivers is illustrated as a second receiver 14 .

[0058] Figure 4 is a diagram illustrating a plurality of pixel values ​​acquired from each of a plurality of pixels according to an embodiment of the present disclosure.

[0059] In addition, each of the multiple receivers may obtain a pixel value corresponding to the intensity of the received reflected light. To this end, the receiver may include a photodiode (PD) and an avalanche photodiode (APD). More specifically, when the receiver receives the reflected light, the amount of charge accumulated in the diode corresponding to the intensity of the reflected light may be obtained as a pixel value. However, the present disclosure is not limited thereto, and when the receiver receives the reflected light, the receiver may obtain a current value flowing through the diode as a pixel value. However, in the following description, for the convenience of describing the present disclosure, it is assumed that the pixel value is the amount of charge.

[0060] In addition, as described above, since each of the plurality of pixels includes a plurality of receivers (e.g., the first receiver 12 and the second receiver 14), and the plurality of receivers are activated at a preset time interval, the processor 130 may acquire a plurality of pixel values ​​for each of the plurality of pixels. That is, the plurality of pixel values ​​acquired from each of the plurality of pixels may include a pixel value acquired by the first receiver 12 and a pixel value acquired by the second receiver 14.

[0061] Reference Figure 4 , a plurality of pixel values ​​may be acquired from each of the plurality of pixels 10 (e.g., the first receiver 12 and the second receiver 14). Specifically, the pixels arranged in the first row and the first column acquire 1 and 4 as the plurality of pixel values. That is, the pixels arranged in the first row and the first column have 1 as the pixel value acquired by the first receiver 12 among the plurality of receivers, and have 4 as the pixel value acquired by the second receiver 14. In this way, each of the plurality of pixels may acquire a plurality of pixel values ​​through the plurality of receivers (e.g., the first receiver 12 and the second receiver 14) activated at different times.

[0062] The processor 130 may identify a phase difference between the light output from the light emitting unit 110 and the light received by the iToF sensor 120 based on a difference between pixel values ​​acquired at different times (or acquired by different receivers). The processor 130 may acquire a distance value corresponding to the distance between the Lidar sensor 100 and the object 200 based on the identified phase difference between the output light and the received light.

[0063] In addition, the processor 130 may acquire a depth image for the object 200 based on pixel values ​​acquired through the plurality of pixels 10. More specifically, when the first receiver 12 included in each pixel is activated at time t1 and the second receiver 14 is activated at time t2, the processor 130 may acquire a first depth image based on the pixel value acquired through the first receiver 12 activated at time t1. In addition, the processor 130 may acquire a second depth image based on the pixel value acquired through the second receiver 14 activated at time t2.

[0064] The processor 130 may identify the difference between the pixel values ​​of the same pixel 10 based on the first depth image and the second depth image, and identify the phase difference between the output light and the reflected light based on the first depth image and the second depth image. To this end, the Lidar sensor 100 may further include an RGB diode and a driver IC corresponding to each of the plurality of pixels. That is, the RGB diode may receive a driving current corresponding to the pixel value of each of the plurality of pixels through the driver IC, and output light having brightness and color corresponding to each pixel value.

[0065] In addition, according to an embodiment of the present disclosure, the processor 130 may select only pixels having a high reliability level for pixel values ​​from the plurality of pixels 10, and use only the plurality of pixel values ​​of the selected pixels to identify the distance between the Lidar sensor 100 and the object 200. Here, the reliability level is a value indicating the reliability of a plurality of pixel values ​​acquired from each of the plurality of pixels. The reliability level may be acquired based on a plurality of pixel values ​​corresponding to each of the plurality of pixels. For example, the reliability level may be acquired for each of the plurality of pixels based on the size of the plurality of pixel values ​​corresponding to each of the plurality of pixels.

[0066] In addition, according to an embodiment of the present disclosure, the processor 130 may use light of different frequencies to recognize the distance between the Lidar sensor 100 and the object 200. Hereinafter, an embodiment of the present disclosure related to an operation of recognizing a distance will be described in detail.

[0067] First, according to an embodiment of the present disclosure, when the Lidar sensor 100 rotates 360°, the processor 130 outputs light of a first frequency and light of a second frequency greater than the first frequency through the light emitting unit 110 .

[0068] Specifically, the processor 130 may control the driving unit of the Lidar sensor 100 so that the Lidar sensor 100 rotates 360°. For example, the Lidar sensor 100 may include a frame in which the light emitting unit 110 and the iToF sensor 120 are disposed, and a motor connected to the frame to rotate the frame. In this case, the processor 130 may send an electrical signal for driving the motor to the driving unit, so that the driving unit can rotate the motor 360° at a preset speed. In addition, the frame connected to the motor can also rotate 360° at a preset speed.

[0069] The processor 130 may output light of a first frequency and light of a second frequency greater than the first frequency through the light emitting unit 110 while the lidar sensor 100 rotates 360°. Specifically, the processor 130 may control the light emitting unit 110 to generate light of different frequencies, and then, the light of different frequencies may be output toward the object 200 through the light emitting unit 110. Optionally, the lidar sensor 100 may include a plurality of light emitting units configured to output light of different frequencies. In this case, the processor 130 may output light of the first frequency and light of the second frequency respectively through a light emitting unit configured to output light of the first frequency and a light emitting unit configured to output light of the second frequency.

[0070] In addition, the processor 130 may sequentially output light of a first frequency and light of a second frequency. For example, the processor 130 may control the light emitting unit 110 to output light of a first frequency, and then output light of a second frequency at a preset time interval. Alternatively, the processor 130 may control the light emitting unit 110 to output light of a first frequency, and then rotate the Lidar sensor 100 by a preset angle to output light of a second frequency. However, the present disclosure is not limited thereto, and the processor 130 may output light of a second frequency greater than the first frequency through the light emitting unit 110, and then output light of the first frequency.

[0071] Light of different frequencies may be referred to as an output light (or reflected light) group, a pair of lights, etc., because it is used to identify the distance between a specific point on the object 200 and the lidar sensor 100. For example, assume that the light emitting unit 110 is set to output light of a first frequency and light of a second frequency every 0.005 seconds. In this case, when the driving of the lidar sensor 100 starts, the processor 130 may output light of a first frequency, and then output light of a second frequency after 0.005 seconds have passed. In addition, the processor 130 may output light of a first frequency, and then output light of a second frequency after 0.005 seconds have passed. That is, the processor 130 may output another light of a first frequency after 0.010 seconds after the driving of the lidar sensor starts. The processor 130 may output light of a first frequency, and then output another light of a second frequency after 0.005 seconds have passed. That is, the processor 130 may output another light of a second frequency after 0.015 seconds after the driving of the lidar sensor 100 starts. In this case, the processor 130 may identify the distance between a specific point on the object 200 and the Lidar sensor 100 using the light of the first frequency output at the start of driving of the Lidar sensor and the light of the second frequency output 0.005 seconds after the start of driving of the Lidar sensor 100. The processor 130 may identify the distance between another specific point on the object 200 and the Lidar sensor 100 using the light of the first frequency output after the start of driving of the Lidar sensor 100 and the light of the second frequency output 0.015 seconds after the start of driving of the Lidar sensor 100. In this way, the processor 130 may identify the distance between the Lidar sensor 100 and the object 200 based on the light of the first frequency and the light of the second frequency output at each preset time interval (or preset time period).

[0072] In addition, the light of the second frequency may be set to a larger value than the light of the first frequency. The processor 130 may use the light of the first frequency having a relatively small value to identify the distance from the lidar sensor 100 to the distant object 200. That is, when the processor 130 uses the light of the first frequency, the sensing distance of the lidar sensor may be greater than when the light of the second frequency is used. Here, the sensing distance may be a recognizable distance between the lidar sensor 100 and the object 200. And, the processor 130 may use the light of the second frequency having a relatively large value to more accurately identify the distance between the lidar sensor 100 and the object 200. That is, when the processor 130 uses the light of the second frequency, the sensing distance of the lidar sensor 100 may be more accurate than when the light of the first frequency is used.

[0073] Therefore, according to an embodiment of the present disclosure, since the processor 130 can identify the distance between the lidar sensor 100 and the object 200 using light of the first frequency and light of the second frequency greater than the first frequency, the processor 130 can identify the distance between the object 200 and the lidar sensor 100 and simultaneously accurately identify the distance between the object 200 and the lidar sensor 100. Hereinafter, for the convenience of describing the present disclosure, it is assumed that the first frequency is 5 MHz and the second frequency is 100 MHz greater than the first frequency.

[0074] According to an embodiment of the present disclosure, the processor 130 may output light of a first frequency and light of a second frequency, and then, when the output light of the first frequency and the output light of the second frequency are reflected by the object 200 and received by a plurality of pixels 10, the processor 130 may obtain a plurality of first pixel values ​​based on the reflected light of the first frequency, and obtain a plurality of second pixel values ​​based on the reflected light of the second frequency.

[0075] Specifically, when the 5MHz light output by the light emitting unit is reflected by the object 200 and returns toward the Lidar sensor 100, the processor 130 may receive the reflected 5MHz light through the iToF sensor 120. When the 100MHz light output by the light emitting unit is reflected by the object 200 and returns toward the Lidar sensor 100, the processor 130 may receive the reflected 100MHz light through the iToF sensor 120. In this case, the processor 130 may acquire a plurality of pixel values ​​corresponding to the light of each frequency of the first frequency and the second frequency. In detail, when the reflected light of 5MHz is received by the iToF sensor 120, the processor 130 may acquire the pixel values ​​of the plurality of pixels 10 included in the iToF sensor 120. Here, the pixel value may be a charge amount value accumulated in the plurality of receivers included in each of the plurality of pixels 10 based on the intensity of the reflected light of 5MHz. Similarly, when the reflected light of 100MHz is received by the iToF sensor 120, the processor 130 may acquire the pixel values ​​of the plurality of pixels 10 included in the iToF sensor 120. Hereinafter, a plurality of pixel values ​​acquired by receiving the reflected light of 5 MHz will be referred to as first pixel values, and a plurality of pixel values ​​acquired by receiving the reflected light of 100 MHz will be referred to as second pixel values.

[0076] Figure 5 is a diagram illustrating a first pixel value corresponding to 5 MHz light and a second pixel value corresponding to 100 MHz light according to an embodiment of the present disclosure.

[0077] Reference Figure 5 , when the 5 MHz light output by the light emitting unit is reflected by the object 200 and returns toward the Lidar sensor 100, the processor 130 may receive the reflected light of the 5 MHz light through the iToF sensor 120. In addition, when the processor 130 receives the reflected light of the 5 MHz light, the processor 130 may acquire a plurality of first pixel values ​​corresponding to the 5 MHz light based on the amount of charge accumulated in the first receiver 12 and the second receiver 14 included in each of the plurality of pixels 10. Similarly, when the 100 MHz light output by the light emitting unit is reflected by the object 200 and returns toward the Lidar sensor 100, the processor 130 may receive the reflected light of the 100 MHz light through the iToF sensor 120. In addition, when the processor 130 receives the reflected light of the 100 MHz light, the processor 130 may acquire a plurality of second pixel values ​​corresponding to the 100 MHz light based on the amount of charge accumulated in the first receiver 12 and the second receiver 14 included in each of the plurality of pixels 10.

[0078] In addition, the plurality of first pixel values ​​and the plurality of second pixel values ​​may be different from each other. Specifically, in a specific pixel among the plurality of pixels 10 included in the iToF sensor 120, the plurality of first pixel values ​​obtained when the specific pixel receives reflected light of 5 MHz light and the plurality of second pixel values ​​obtained when the specific pixel receives reflected light of 100 MHz light may be different from each other. Figure 5 , in the case of pixels arranged in the first row and the first column among the plurality of pixels 10, the plurality of first pixel values ​​corresponding to the light of 5MHz are 1 and 4, and the plurality of second pixel values ​​corresponding to the light of 100MHz are 0.5 and 0.8. Since the frequencies of the light output from the light emitting unit are different from each other, the period of the reflected light of each light received by the iToF may be different from each other. Therefore, since the value of the amount of charge accumulated in the receiver may be different, the plurality of pixel values ​​corresponding to each of the first frequency and the second frequency may be different from each other. In addition, since the time and time interval at which the plurality of receivers included in each of the plurality of pixels are activated (or turned on) can be set based on the frequency output from the light emitting unit, the plurality of pixels corresponding to each of the first frequency and the second frequency may be different from each other.

[0079] After acquiring multiple first pixel values ​​and multiple second pixel values, the processor 130 may identify multiple third pixel values ​​from the multiple first pixel values ​​using reliability levels of the multiple first pixel values ​​determined based on the sizes of the multiple first pixel values, and identify multiple fourth pixel values ​​from the multiple second pixel values ​​using reliability levels of the multiple second pixel values ​​determined based on the sizes of the multiple second pixel values.

[0080] Specifically, even if a single light is output from the light emitting unit 110, the iToF sensor 120 may receive a plurality of reflected lights by diffraction, scattering, reflection, etc. of the output light. This may be caused by the surrounding environment of the Lidar sensor or the shape or color of the object. Therefore, the plurality of pixel values ​​of each of the plurality of pixels 10 for the same light output from the light emitting unit 110 may be different from each other. That is, returning to the reference Figure 5 , a plurality of first pixel values ​​acquired from the plurality of pixels 10 corresponding to 5 MHz may be different for each of the plurality of pixels 10 .

[0081] Therefore, according to an embodiment of the present disclosure, the processor 130 may select only pixels having pixel values ​​with a high reliability level based on the reliability levels of the plurality of pixel values, and identify the distance between the Lidar sensor 100 and the object 200 based on the plurality of pixel values ​​of the selected pixels. The processor 130 may select at least one pixel from the plurality of pixels 10 based on the reliability level, and identify the distance between the Lidar sensor 100 and the object 200 based on the plurality of pixel values ​​of the selected at least one pixel.

[0082] To this end, the processor 130 may identify the reliability level of each of the plurality of pixels based on the plurality of pixel values ​​for each pixel. Specifically, the processor 130 may identify the reliability level of the plurality of pixels 10 for light of a first frequency based on the plurality of first pixel values ​​(hereinafter, the first reliability level). In addition, the processor 130 may identify the reliability level of the plurality of pixels 10 for light of a second frequency based on the plurality of second pixel values ​​(hereinafter, the second reliability level).

[0083] In this case, according to an embodiment of the present disclosure, the processor 130 may respectively identify each of the reliability levels of the plurality of first pixel values ​​and the reliability levels of the plurality of second pixel values ​​based on the following equations.

[0084] Reliability level of the first pixel value = Reliability level of the second pixel value = In this case, the first value may be a first pixel value acquired when the first receiver 12 of the plurality of receivers included in each of the plurality of pixels receives the reflected light of the first frequency, and the second value may be a first pixel value acquired when the second receiver 14 of the plurality of receivers included in each of the plurality of pixels receives the reflected light of the first frequency. In this case, the third value may be a second pixel value acquired when the first receiver 12 of the plurality of receivers included in each of the plurality of pixels receives the reflected light of the second frequency, and the fourth value may be a second pixel value acquired when the second receiver 14 of the plurality of receivers included in each of the plurality of pixels receives the reflected light of the second frequency.

[0085] For example, refer to Figure 5 , the processor 130 may identify a first reliability level of light of a first frequency (ie, light of 5 MHz) of pixels arranged in a first row and a first column among the plurality of pixels 10 as The processor 130 may identify the second reliability level of the light of the second frequency (ie, the light of 100 MHz) of the pixels arranged in the first row and the first column among the plurality of pixels 10 as In this way, the processor 130 may respectively identify a first reliability level of a plurality of first pixel values ​​based on the light of the first frequency and a second reliability level of a plurality of second pixel values ​​based on the light of the second frequency.

[0086] In addition, the method of identifying the first reliability level and the second reliability level is not limited to the above-described embodiment, and various methods may be applied.

[0087] For example, the processor 130 may identify a first reliability level of the plurality of pixels 10 based on a first pixel value (ie, a first value) acquired from each of the plurality of pixels and acquired by a first receiver 12 among the plurality of receivers included in the plurality of pixels 10. Return to Reference Figure 5 , the processor 130 may identify the first reliability level of the pixels arranged in the first row and the first column among the plurality of pixels 10 as 1, where 1 is the value of the first pixel 20 acquired by the first receiver 12 (i.e., the first value). Similarly, the processor 130 may identify the second reliability level of the plurality of pixels 10 for light of the second frequency based on the plurality of second pixel values. Based on the above example, described again, specifically, the processor 130 may identify the second reliability level of the plurality of pixels 10 based on the second pixel value (i.e., the third value) acquired from each of the plurality of pixels and acquired by the first receiver 12 among the plurality of receivers included in the plurality of pixels 10. Return to reference Figure 5 , the processor 130 may identify a first reliability level of pixels arranged in a first row and a first column among the plurality of pixels 10 as 0.5, which is a value of the second pixel 30 acquired by the first receiver 12 (ie, a third value).

[0088] As another example, the processor 130 may identify a first reliability level and a second reliability level of the multiple pixels 10 based on a first pixel value (i.e., the second value) and a second pixel value (i.e., the fourth value) obtained from each pixel by a second receiver 14 among the multiple receivers included in the multiple pixels 10.

[0089] Alternatively, the processor 130 may identify the reliability level of each of the plurality of pixels as the sum of pixel values ​​acquired by a plurality of receivers included in each of the plurality of pixels for light of each of the first frequency and the second frequency (light of the first frequency and light of the second frequency). Figure 5 , the processor 130 may identify the first reliability level of the pixel arranged in the first row and the first column among the plurality of pixels 10 as 5, 5 being the sum of 1 and 4, 1 being the value of the first pixel 20 acquired by the first receiver 12 (i.e., the first value), and 4 being the value of the first pixel 20 acquired by the second receiver 14 (i.e., the second value). In addition, the processor 130 may identify the second reliability level of the pixel arranged in the first row and the first column among the plurality of pixels 10 as 1.3, 1.3 being the sum of 0.5 and 0.8, 0.5 being the value of the second pixel 30 acquired by the first receiver 12 (i.e., the third value), and 0.8 being the value of the second pixel 30 acquired by the second receiver 14 (i.e., the fourth value).

[0090] In addition, according to an embodiment of the present disclosure, the processor 130 may identify a plurality of third pixel values ​​each having a reliability level greater than or equal to a preset value from the plurality of first pixel values ​​based on the reliability levels of the plurality of first pixel values, and identify a plurality of fourth pixel values ​​each having a reliability level greater than or equal to a preset value from the plurality of second pixel values ​​based on the reliability levels of the plurality of second pixel values. In this case, each of the reliability levels of the plurality of first pixel values ​​and the second pixel values ​​has a high reliability level as the size of the pixel value increases.

[0091] Specifically, the processor 130 may select a plurality of pixels 10 having a plurality of first reliability levels greater than or equal to a preset reliability level (hereinafter, preset first value). In this case, the plurality of pixels 10 selected from the plurality of pixels 10 based on the first reliability level are referred to as first pixels 20. In addition, the processor 130 may identify a plurality of third pixel values ​​based on a plurality of first pixel values ​​of the selected first pixels 20. The third pixel value refers to a first pixel value of each of the plurality of first pixels 20 selected according to the first reliability level. Similarly, the processor 130 may select pixels having a second reliability level greater than or equal to a preset reliability level (hereinafter, preset second value). In this case, the plurality of pixels 10 selected from the plurality of pixels 10 based on the second reliability level are referred to as second pixels 30. The processor 130 may identify a plurality of fourth pixel values ​​based on a plurality of second pixel values ​​of the selected second pixels 30. The fourth pixel value refers to a second pixel value of each of the plurality of second pixels 30 selected according to the second reliability level.

[0092] In addition, the preset first value and the preset second value as the criterion for selecting the third pixel value and the fourth pixel value may be set based on the amplitude of each of the first frequency and the second frequency, etc. Therefore, the number of the selected third pixel values ​​and the number of the selected fourth pixel values ​​may be different.

[0093] Figure 6 is a diagram illustrating selecting a plurality of third pixel values ​​based on a first reliability level and selecting a plurality of fourth pixel values ​​based on a second reliability level according to an embodiment of the present disclosure.

[0094] Reference Figure 6, when the preset first value as a criterion for identifying the third pixel value is set to 22, the processor 130 may select nine pixels from the plurality of pixels 10 as pixels having a first reliability level of 22 or greater. That is, the processor 130 may identify the nine first pixels 20 based on the first reliability level. In addition, the processor 130 may identify the plurality of first pixel values ​​acquired by the nine pixels as a plurality of third pixel values. In addition, when the preset second value as a criterion for identifying the fourth pixel value is set to 5.2, the processor 130 may select twelve pixels from the plurality of pixels 10 as pixels having a second reliability level of 5.2 or greater. That is, the processor 130 may identify the twelve second pixels 30 based on the second reliability level. In addition, the processor 130 may identify the plurality of second pixel values ​​acquired by the twelve pixels as a plurality of fourth pixel values.

[0095] In addition, according to an embodiment of the present disclosure, before selecting the first pixel 20 and the second pixel 30 from the plurality of pixels 10 based on the first reliability level and the second reliability level, the processor 130 may perform a filtering process of applying a filter having a preset size to the plurality of pixels 10. An embodiment of the present disclosure related to the filtering process will be described.

[0096] Figure 7 is a diagram for describing filtering a plurality of pixels by applying a filter having a preset size to the plurality of pixels according to an embodiment of the present disclosure.

[0097] According to an embodiment of the present disclosure, the processor 130 may acquire a plurality of filtered pixels by applying a filter 50 having a preset size to a plurality of pixels 10 , and identify a plurality of third pixel values ​​and a plurality of fourth pixel values ​​based on the filtered pixel values ​​of the plurality of pixels 10 .

[0098] Specifically, the processor 130 may perform primary filtering on the plurality of pixels 10 before selecting the first pixel 20. It may mean that, as the number of pixels included in the iToF increases, there are many objects that the processor 130 needs to identify the first reliability level and compare the first reliability level with the preset first value. That is, the problem of time and resources required for the processor 130 to select the first pixel 20 from the plurality of pixels 10 is caused. Therefore, the processor 130 may apply a filter 50 with a preset size before selecting the first pixel 20 from the plurality of pixels 10 based on the first reliability level, so as to first select the plurality of pixels included in the filter 50 with a preset size from the plurality of pixels 10. Here, among the plurality of pixels 10, the plurality of pixels selected by the filter 50 with a preset size are referred to as third pixels.

[0099] For example, refer to Figure 7 , the processor 130 may 3 Filter 50 is applied to 5 5, the processor 130 selects the plurality of pixels 10 arranged in the matrix to first select the nine third pixels arranged in the center. In addition, the processor 130 may select the first pixel 20 from the nine third pixels by comparing the first reliability levels of the selected nine third pixels. In this case, when it is assumed that the preset first reliability level for selecting the first pixel 20 is 22, the processor 130 may select the remaining seven third pixels except the two third pixels having the reliability level of 19 from the nine third pixels as the first pixel 20. That is, the processor 130 may preliminarily select the plurality of third pixels from the plurality of pixels 10 included in the iToF sensor 120 using the filter 50 having a preset size, and reselect the first pixel 20 from the plurality of third pixels based on the first reliability levels of the selected plurality of third pixels.

[0100] Similarly, the processor 130 may apply the filter 50 having a preset size before selecting the second pixel 30 from the plurality of pixels 10 based on the second reliability level to first select the plurality of pixels 10 included in the filter 50 having the preset size from the plurality of pixels 10. For example, referring to Figure 7 , the processor 130 may 3 Filter 50 is applied to 5 5 matrix arrangement of the plurality of pixels 10 to first select nine third pixels arranged in the center. In addition, the processor 130 may select the second pixel 30 from the nine third pixels by comparing the second reliability levels of the selected nine third pixels. In this case, when it is assumed that the preset second reliability level for selecting the second pixel 30 is 5.2, the processor 130 may select the remaining eight third pixels except one third pixel with a reliability level of 5.1 from the nine third pixels as the second pixel 30. That is, the processor 130 may preliminarily select the plurality of third pixels from the plurality of pixels 10 included in the iToF sensor 120 using the filter 50 having a preset size, and reselect the second pixel 30 from the plurality of third pixels based on the second reliability levels of the selected plurality of third pixels.

[0101] In addition, according to the reference Figure 7In the described embodiment of the present disclosure, filters having the same size are applied to a plurality of pixels 10 having a plurality of first pixel values ​​corresponding to light of a first frequency and a plurality of pixels 10 having a plurality of second pixel values ​​corresponding to light of a second frequency, but are not limited thereto. That is, the processor 130 may apply a filter having a preset first size to a plurality of pixels 10 having a plurality of first pixel values ​​corresponding to light of a first frequency, and apply a filter having a preset second size to a plurality of pixels 10 having a plurality of second pixel values ​​corresponding to light of a second frequency. In this case, the size of each filter may be determined based on the size of the frequency of the light.

[0102] In addition, the size and position of the filter 50 applied to the plurality of pixels 10 arranged in a matrix form may be set differently according to the number, arrangement, etc. of the plurality of pixels 10. That is, although Figure 7 The 3×3 filter 50 is shown to be applied to the center of the pixel matrix, but the filter may be applied to various positions, such as the upper left and upper right of the pixel matrix according to an embodiment.

[0103] In addition, according to an embodiment of the present disclosure, the processor 130 may identify the distance between the Lidar sensor 100 and the object 200 based on at least one of a first distance value and a second distance value, wherein the first distance value is obtained based on a plurality of third pixel values ​​and the second distance value is obtained based on a plurality of fourth pixel values.

[0104] Specifically, the processor 130 may identify a phase difference between the output light of the first frequency and the reflected light of the first frequency in each first pixel 20 based on the first pixel value (ie, the third pixel value) of the first pixel 20, wherein the first pixel 20 is selected based on the first reliability level. Return to reference Figure 3 , the processor 130 activates the first receiver 12 among the plurality of receivers at time t1 when the light of the first frequency is output from the light emitting unit 110. The processor 130 activates the first receiver 12 among the plurality of receivers at a preset time interval T after t1. P At time t2, the activated first receiver 12 is turned off and the second receiver 14 is activated. In this case, different amounts of charges Q1 and Q2 are accumulated in the first receiver 12 and the second receiver 14, respectively, according to the intensity of the received reflected light of the first frequency.

[0105] Specifically, in the case of pixels arranged in the first row and the first column among the plurality of pixels 10 arranged in a matrix form, when the light of the first frequency output at t1 is reflected by the object 200, charges are accumulated in the first receivers 12 of the pixels arranged in the first row and the first column from the time t3 when the reflected light of the first frequency is received. When the first receiver 12 is turned off and the second receiver 14 is turned on at t2, charges are also accumulated in the second receivers 14 of the pixels arranged in the first row and the first column as the reflected light of the first frequency is received. In this case, Q1 and Q2 may be acquired as a plurality of first pixel values ​​in the pixels arranged in the first row and the first column. In this case, the time period during which the first receiver 12 and the second receiver 14 are activated or turned on may be set based on the frequency or period of the output light.

[0106] In addition, the processor 130 may identify a phase difference between the reflected first frequency light received from the pixels arranged in the first row and the first column and the output first frequency light output from the light emitting unit based on the plurality of first pixel values ​​Q1 and Q2. For example, the processor 130 may identify a phase difference between the reflected first frequency light received from the pixels arranged in the first row and the first column and the output first frequency light output from the light emitting unit 110 based on the phase, frequency, cycle, or a plurality of first pixel values ​​Q1 and Q2 of the first frequency light output from the light emitting unit 110. Optionally, the processor 130 may further identify a phase difference between the reflected light of the first frequency received from the pixels arranged in the first row and the first column and the output light of the first frequency output from the light emitting unit based on the difference between the plurality of pixel values ​​Q1 and Q2. t.

[0107] In addition, the processor 130 may also apply the above method to identify the phase difference between the output light and the reflected light of the second frequency light.

[0108] In addition, the processor 130 may acquire a distance value corresponding to each first pixel 20 based on a phase difference between the output light of the first frequency and the reflected light of the first frequency identified for each first pixel 20. In this case, the distance value corresponding to the first pixel 20 may be referred to as a distance value corresponding to the light of the first frequency. In addition, as an example, the distance value corresponding to the first pixel 20 may be identified by the following equation.

[0109] The distance value corresponding to the first pixel 20 = Here, c is the speed of light, T p1represents a period of light of the first frequency, Q1 represents an amount of charge accumulated in the first receiver 12 when receiving light of the first frequency, and Q2 represents an amount of charge accumulated in the second receiver 14 when receiving light of the first frequency.

[0110] In addition, the processor 130 may obtain a distance value corresponding to each second pixel 30 based on a phase difference between the output light of the second frequency and the reflected light of the second frequency identified for each second pixel 30. In this case, the distance value corresponding to the second pixel 30 may be referred to as a distance value corresponding to the light of the second frequency. In addition, as an example, the distance value corresponding to the second pixel 30 may be identified by the following equation.

[0111] The distance value corresponding to the second pixel 30 = Here, c is the speed of light, T p2 represents the period of the light of the second frequency, Q3 represents the amount of charge accumulated in the first receiver 12 when the light of the second frequency is received, and Q4 represents the amount of charge accumulated in the second receiver 14 when the light of the second frequency is received.

[0112] In addition, the processor 130 may select a plurality of first pixels 20 based on the first reliability level, and obtain a plurality of distance values ​​based on a plurality of first pixel values ​​(i.e., a plurality of third pixel values) of each of the selected plurality of first pixels 20. That is, the processor 130 may obtain a plurality of distance values ​​corresponding to the plurality of first pixels 20. Similarly, the processor 130 may select a plurality of second pixels 30 based on the second reliability level, and obtain a plurality of distance values ​​based on a plurality of second pixel values ​​(i.e., a plurality of fourth pixel values) of each of the selected plurality of second pixels 30. That is, the processor 130 may obtain a plurality of distance values ​​corresponding to the plurality of second pixels 30.

[0113] In this case, the processor 130 may identify one distance value (i.e., first distance value) corresponding to the first frequency based on the multiple distance values ​​corresponding to the multiple first pixels 20. In addition, the processor 130 may identify one distance value (i.e., second distance value) corresponding to the second frequency based on the multiple distance values ​​corresponding to the multiple second pixels 30. Hereinafter, embodiments of the present disclosure related to the operation of identifying the distance will be described.

[0114] Figure 8 is a diagram for identifying a first distance value and a second distance value among a plurality of distance values ​​corresponding to a plurality of first pixels and a plurality of distance values ​​corresponding to a plurality of second pixels according to an embodiment of the present disclosure.

[0115] According to an embodiment of the present disclosure, the processor 130 may identify multiple distance values ​​based on multiple third pixel values, and identify the first distance value based on the average value of the identified multiple distance values. The processor 130 may identify multiple distance values ​​based on multiple fourth pixel values, and identify the second distance value based on the average value of the identified multiple distance values.

[0116] Specifically, the processor 130 may acquire a plurality of distance values ​​corresponding to the plurality of first pixels 20 based on a plurality of first pixel values ​​(i.e., third pixel values) corresponding to each of the plurality of first pixels 20 selected based on the first reliability level. In this case, the processor 130 may calculate an average value of the plurality of distance values ​​corresponding to the first pixels 20, and identify the calculated average value as the first distance value corresponding to the first frequency. In addition, the processor 130 may acquire a plurality of distance values ​​corresponding to the plurality of second pixels 30 based on a plurality of second pixel values ​​(i.e., fourth pixel values) corresponding to each of the plurality of second pixels 30 selected based on the second reliability level. In this case, the processor 130 may calculate an average value of the plurality of distance values ​​corresponding to the second pixels 30, and identify the calculated average value as the second distance value corresponding to the second frequency.

[0117] Reference Figure 8 , the processor 130 may obtain an average value of multiple distance values ​​corresponding to multiple first pixels 20 based on multiple third pixel values ​​of the first pixel 20 corresponding to the first frequency of 5 MHz. The processor 130 may identify the obtained average value as the first value. That is, the processor 130 may identify the first distance value corresponding to the first frequency as 1.716m ((1.71 + 1.72 + 1.69 + 1.72 + 1.72 + 1.74 + 1.73 + 1.71 + 1.71) / 8). In addition, the processor 130 may obtain an average value of multiple distance values ​​corresponding to multiple second pixels 30 based on multiple fourth pixel values ​​of the second pixel 30 corresponding to the second frequency of 100 MHz. The processor 130 may identify the obtained average value as the second value. That is, the processor 130 may identify the second distance value corresponding to the second frequency as 0.288 m ((0.25 + 0.255 + 0.280 + 0.275 + 0.255 + 0.275 + 0.295 + 0.280 + 0.255 + 0.270 + 0.265 + 0.255) / 12).

[0118] According to an embodiment of the present disclosure, the processor 130 may identify the maximum distance value among the multiple distance values ​​identified based on the multiple third pixel values ​​as the first distance value. The processor 130 may identify the maximum distance value among the multiple distance values ​​identified based on the multiple fourth pixel values ​​as the second distance value.

[0119] Specifically, the processor 130 may acquire a plurality of distance values ​​corresponding to the plurality of first pixels 20 based on a plurality of first pixel values ​​(i.e., third pixel values) corresponding to each of the plurality of first pixels 20 selected based on the first reliability level. In this case, the processor 130 may identify the maximum distance value among the plurality of distance values ​​as the first distance value corresponding to the first frequency. In addition, the processor 130 may acquire a plurality of distance values ​​corresponding to the plurality of second pixels 30 based on a plurality of second pixel values ​​(i.e., fourth pixel values) corresponding to each of the plurality of second pixels 30 selected based on the second reliability level. In this case, the processor 130 may identify the maximum distance value among the plurality of distance values ​​as the second distance value corresponding to the second frequency.

[0120] Reference Figure 8 , the processor 130 may obtain a plurality of distance values ​​corresponding to the plurality of first pixels 20 based on a plurality of third pixel values ​​of the first pixels 20 corresponding to the first frequency of 5 MHz. In this case, the processor 130 may identify a distance value corresponding to the first reliability level, which is the largest of the plurality of distance values. That is, the processor 130 may select the first pixel 20 having the largest first reliability level from among the plurality of first pixels 20. Figure 8 , the processor 130 may select the first pixel 20 having the first reliability level of 27 among the nine first pixels 20. In addition, the processor 130 may identify 1.74 m as the first distance value, which is the distance value corresponding to the selected first pixel 20.

[0121] In addition, refer to Figure 8 , the processor 130 may obtain a plurality of distance values ​​corresponding to the plurality of second pixels 30 based on a plurality of fourth pixel values ​​of the second pixels 30 corresponding to the second frequency of 100 MHz. In this case, the processor 130 may identify a distance value corresponding to the second reliability level, which is the largest of the plurality of distance values. That is, the processor 130 may select the second pixel 30 having the largest second reliability level from among the plurality of second pixels 30. Figure 8 , the processor 130 may select the second pixel 30 having the second reliability level of 6.1 among the twelve second pixels 30. In addition, the processor 130 may identify 0.295 m, which is a distance value corresponding to the selected second pixel 30, as the second distance value.

[0122] On the other hand, although not shown in the drawings, when a plurality of first pixels 20 having the maximum first reliability level are selected, the processor 130 may identify the first distance value based on an average value of the distance values ​​of the selected plurality of first pixels 20. In addition, when a plurality of second pixels 30 having the maximum first reliability level are selected, the processor 130 may identify the second distance value based on an average value of the distance values ​​of the selected plurality of second pixels 30.

[0123] In addition, according to the embodiments of the present disclosure, Figure 8 Unlike the illustrated case, the processor 130 may only obtain the distance value of the first pixel 20 selected based on the first reliability level. Similarly, the processor 130 may only obtain the distance value of the second pixel 30 selected based on the second reliability level.

[0124] In addition, according to the above-mentioned embodiment of the present disclosure, the same method has been described as being applied to the method of obtaining the first distance value and the second distance value, but the method of obtaining the first distance value and the second distance value may be different. For example, the first distance value may be obtained based on the average value of the plurality of distance values ​​corresponding to the first pixel 20, and the second distance value may be obtained based on the distance value of the pixel having the maximum second reliability level among the plurality of distance values ​​corresponding to the second pixel 30.

[0125] In addition, the processor 130 may acquire the first distance value and the second distance value based on various methods. For example, the processor 130 may acquire the first distance value and the second distance value based on a weighted average value. Specifically, the processor 130 may select a pixel having a first reliability level greater than or equal to a preset first value and less than a preset third value from among the plurality of pixels 10 as the first pixel 20, and may select a pixel having a first reliability level greater than or equal to a preset third value from among the plurality of pixels 10 as the third pixel. In this case, the processor 130 may calculate an average value (hereinafter, a first average value) by applying a first weight to a distance value corresponding to the first pixel 20 for the first pixel 20, and calculate an average value (hereinafter, a second average value) by applying a second weight greater than the first weight to a distance value corresponding to the third pixel for the third pixel. In addition, the processor 130 may acquire a first distance value as an average value of the first average value and the second average value. In addition, the processor 130 may select a pixel having a second reliability level greater than or equal to a preset first value and less than a preset fourth value from among the plurality of pixels 10 as the second pixel 30, and may select a pixel having a second reliability level greater than or equal to a preset fourth value from among the plurality of pixels 10 as the fourth pixel. In this case, the processor 130 may calculate an average value (hereinafter, a third average value) by applying the third weight to the distance value corresponding to the second pixel 30 for the second pixel 30, and calculate an average value (hereinafter, a fourth average value) by applying a fourth weight greater than the third weight to the distance value corresponding to the fourth pixel for the fourth pixel. And, the processor 130 may acquire a second distance value which is an average value of the third average value and the fourth average value.

[0126] In addition, the processor 130 may identify the distance between the Lidar sensor 100 and the object 200 based on the acquired first distance value and the second distance value.

[0127] In this regard, according to an embodiment of the present disclosure, when the difference between the first distance value and the second distance value is greater than or equal to a preset value, the processor 130 may determine a compensation value based on the first distance value and the second frequency, compensate the second distance value based on the determined compensation value, and identify the compensated second distance value as the distance between the lidar sensor 100 and the object 200. In this case, the preset value may be set based on the second frequency.

[0128] As described above, the second distance value may be acquired based on the light of the second frequency which is relatively greater than the first frequency. In this case, when the light of the second frequency is relatively greater than the light of the first frequency, due to the periodicity of the light of the second frequency, there may be a maximum distance value (or a limit distance value) of the recognizable distance between the lidar sensor 100 and the object 200. For example, when the second frequency is 100 MHz, the processor 130 may recognize the distance between the lidar sensor 100 and the object 200 as a maximum of 1.5 m based on the light of the second frequency. As described above, this is due to the period of the second frequency of 100 MHz (1.5 m = c / 2f = (3×10 8 ) / (2×100×10 6 ), where c represents the speed of light and f represents the amplitude of the second frequency). As a result, when using the light of the second frequency, the processor 130 may recognize that the distance of the object located 1.7m away from the Lidar sensor is 0.2m. In addition, when using the light of the second frequency, the processor 130 may recognize that the distance of the object located 3.7m away from the Lidar sensor is 0.7m. Therefore, the processor 130 may determine the compensation value for the second distance value based on the first distance value acquired by using the light of the first frequency.

[0129] Specifically, the processor 130 may compare the first distance value with the second distance value and identify whether the difference between the first distance value and the second distance value is greater than or equal to a preset value. In addition, when it is determined that the difference between the first distance value and the second distance value is greater than or equal to the preset value, the processor 130 may determine a compensation value to be applied to the second distance value based on the first distance value and the second frequency.

[0130] In this case, the processor 130 may determine a proportional constant for calculating a compensation value based on a first distance value compared to a maximum distance value, which may be identified based on light of the second frequency. The processor 130 may determine a compensation value based on a maximum distance value, which may be identified based on the determined proportional constant and light of the second frequency. For example, when the maximum distance value that can be identified based on light of the second frequency is 1.5 m and the first distance value is 4.8 m, the processor 130 may obtain 3.2 as a first distance value compared to the maximum distance value, which may be identified based on light of the second frequency. The processor 130 may identify an integer 3 that does not include a decimal point in the obtained 3.2 as a proportional constant. The processor 130 may determine a compensation value obtained by multiplying 1.5 m by 3, 1.5 m being the maximum distance value that can be identified based on light of the second frequency, and 3 being the identified proportional constant. That is, the processor 130 may determine 4.5 as the compensation value.

[0131] The processor 130 may compensate the second distance value based on the determined compensation value, and identify the compensated second distance value as the distance between the lidar sensor 100 and the object 200. For example, assuming that the first frequency is 5 MHz, the first distance value corresponding to the first frequency is 1.716 m, the second frequency is 100 MHz, the second distance value corresponding to the second frequency is 0.288 m, and the preset value is 1.4 m. In this case, the processor 130 may identify that the difference between the first distance value and the second distance value is 1.428 m. Therefore, the processor 130 may identify that the difference between the first distance value and the second distance value is greater than or equal to 1.4 m corresponding to the preset value. Therefore, the processor 130 may determine the compensation value for the second distance value based on the first distance value. According to the above method of determining the compensation value, the processor 130 may determine the compensation value for the second distance value as 1.5 m (rounded down (1.716 m / 1.5 m)×1.5 m). The processor may identify the distance between the lidar sensor 100 and the object 200 by applying the determined compensation value to the second distance value. That is, the processor 130 may recognize 1.788 m, which is the second distance value (0.288 m+1.5 m) to which the compensation value is applied, as the distance between the Lidar sensor 100 and the object 200 .

[0132] In addition, according to an embodiment of the present disclosure, when the difference between the first distance value and the second distance value is less than a preset value, the processor 130 may identify the second distance value as the distance between the Lidar sensor 100 and the object 200 .

[0133] The processor 130 may compare the first distance value with the second distance value, and identify whether the difference between the first distance value and the second distance value is less than a preset value. When it is identified that the difference between the first distance value and the second distance value is less than the preset value, the processor 130 may identify the distance between the lidar sensor 100 and the object 200 as the second distance value. For example, assuming that the first frequency is 5 MHz, the first distance value corresponding to the first frequency is 0.460 m, the second frequency is 100 MHz, the second distance value corresponding to the second frequency is 0.453 m, and the preset value is 1.4 m. In this case, the processor 130 may identify that the difference between the first distance value and the second distance value is 0.007 m. Therefore, the processor 130 may identify that the difference between the first distance value and the second distance value is less than the preset value. In this case, the processor 130 may identify that the distance between the lidar sensor 100 and the object 200 is 0.453 m as the second distance value.

[0134] In addition, according to an embodiment of the present disclosure, the processor 130 may identify the distance between the Lidar sensor 100 and the object 200 by outputting light of the first frequency and the second frequency at each preset time interval. In this case, when the difference between the distance identified at the first time adjacent to each other and the distance identified at the second time after the preset time interval from the first time among the distances identified at each preset time interval is greater than or equal to a preset value, the processor 130 may identify the distance between the Lidar sensor and the object at the second time based on the first distance value.

[0135] As described above, the processor 130 outputs light of a first frequency and light of a second frequency so as to identify the distance between a specific point on the object 200 and the Lidar sensor 100. In this case, light of a first frequency and light of a second frequency may be output from the light emitting unit at each preset time interval (or preset cycle). In this case, it has been described above that light of a first frequency and light of a second frequency corresponding to a specific point may be referred to as an output light (or reflected light) group or a pair of lights. Hereinafter, for the convenience of describing the present disclosure, light of a first frequency and light of a second frequency corresponding to the same point will be referred to as an output light group.

[0136] In this case, the processor 130 may acquire a distance value corresponding to each output light group based on the light of the first frequency and the light of the second frequency included in each output light group. Here, the acquired distance value may be a value representing the distance between the Lidar sensor 100 and the object 200 identified based on the above-mentioned first distance value and the second distance value. That is, the processor 130 may acquire a first distance value and a second distance value corresponding to each output light group based on the light of the first frequency and the light of the second frequency included in each output light group. The processor 130 may compare the acquired first distance value with the second distance value to acquire the distance value corresponding to each output light group. That is, the distance value corresponding to each output light group may be a value obtained by applying a compensation value determined based on the second distance value corresponding to each output light group or the first distance value corresponding to each output light group to the second distance value corresponding to each output light group.

[0137] Hereinafter, for the convenience of describing the present invention, the distance value corresponding to each output light group is referred to as a third distance value.

[0138] In addition, after obtaining the third distance value corresponding to each output light group, the processor 130 may compare the third distance values ​​between adjacent output light groups. In this case, when the difference between the third distance values ​​between adjacent output light groups is greater than or equal to a preset value, the processor 130 may adjust the third distance value corresponding to the later output light group among the adjacent output light groups.

[0139] Fig. 9is a diagram illustrating a method of adjusting a distance between a Lidar sensor and an object recognized at a relatively late time when a distance difference between the Lidar sensor and the object recognized at adjacent times is greater than or equal to a preset value according to an embodiment of the present disclosure.

[0140] Reference Fig. 9 , the processor 130 may output light of the first frequency at time t1, and output light of the second frequency at time t2. In this case, the processor 130 may recognize the distance between the specific point on the object 200 and the Lidar sensor 100 based on the light of the first frequency output at time t1 and the light of the second frequency output at time t2. That is, the processor 130 may recognize the light of the first frequency output at time t1 and the light of the second frequency output at time t2 as the first output light group. The processor 130 may output light of the first frequency output at time t3, and output light of the second frequency at time t4. In this case, the processor 130 may recognize the distance between the specific point on the object 200 and the Lidar sensor 100 based on the light of the first frequency output at time t3 and the light of the second frequency output at time t4. That is, the processor 130 may recognize the light of the first frequency output at time t3 and the light of the second frequency output at time t4 as the second output light group. In this case, it is assumed that the processor 130 acquires 0.460m as a first distance value acquired based on the light of the first frequency included in the first output light group and 0.453m as a second distance value acquired based on the light of the second frequency included in the first output light group, and acquires 0.463m as a first distance value acquired based on the light of the first frequency included in the second output light group and 1.232m as a second distance value acquired based on the light of the second frequency included in the second output light group. In this case, when it is assumed that the reference value for determining the compensation value based on the first distance value is 1.4m, the processor 130 may identify the third distance value corresponding to the first output light group as 0.453m (i.e., the second distance value corresponding to the first output light group to which the compensation value is not applied). The processor 130 may identify the third distance value corresponding to the second output light group as 1.232m (i.e., the second distance value corresponding to the second output light group to which the compensation value is not applied). In this case, when it is assumed that the preset value as a criterion for determining the adjustment of the third distance value is 0.5 m, the processor 130 may recognize that the difference (0.779 m) between the third distance value (0.453 m) corresponding to the first output light group and the third distance value (1.232 m) corresponding to the second output light group is greater than or equal to the preset value (0.5 m). In this case, the processor 130 may adjust the third distance value corresponding to the second output light group output later in time. That is, the processor 130 may recognize that the third distance value corresponding to the second output light group is 0.463 m as the first distance value.

[0141] In addition, according to other embodiments of the present disclosure, when the difference between the distance identified at the first time adjacent to each other and the distance identified at the second time after the first time in the distance identified at each preset time interval is greater than or equal to a preset value, the processor 130 may not identify the distance between the Lidar sensor 100 and the object 200 at the second time.

[0142] Specifically, when the difference between the third distance values ​​between adjacent output light groups is greater than or equal to a preset value, the processor 130 may exclude the third distance value corresponding to the output light group output later in time from the plurality of third distance values ​​corresponding to the plurality of output light groups. That is, the processor 130 may determine that there is an error in the third distance value corresponding to the output light group output later in time when the difference between the third distance values ​​between adjacent output light groups is greater than or equal to a preset value, and ignore the third distance value determined to be an error.

[0143] Fig.10 is a diagram for describing a method of recognizing distances between a plurality of points of an object and a Lidar sensor based on a plurality of pixels classified into a plurality of pixel groups according to an embodiment of the present disclosure.

[0144] In addition, according to an embodiment of the present disclosure, the processor 130 may recognize distances between the Lidar sensor 100 and a plurality of points of the object 200 based on pixel values. To this end, the Lidar sensor 100 may further include a diffusion unit 150 .

[0145] The diffusion unit may diffuse the light output by the light emitting unit 110. More specifically, the light output from the light emitting unit 110 may be diffracted and scattered in a wider range by passing through the diffusion unit 150. Through this, the range of the light reaching the object 200 may be extended. For example, when the light emitting unit 110 outputs a single laser pulse, the single laser pulse output from the light emitting unit may be separated into a plurality of laser pulses by the diffusion unit 150.

[0146] In this case, the diffusion unit 150 may include a diffractive optical element (DOE), a diffusion plate, a lens, or the like.

[0147] In addition, the plurality of pixels 10 arranged in a matrix form included in the iToF sensor 120 may be divided into a plurality of pixel 10 groups. Here, the plurality of pixel 10 groups may include a plurality of different pixels 10 (or different objects 200) to recognize distances to different points on the object 200.

[0148] For example, refer to Fig.10 , when preset to identify the distance between the first point A and the second point B on the object 200, the processor 130 may The plurality of pixels 10 arranged in the matrix are divided into a first pixel group 10-A corresponding to the first point A and a second pixel group 10-B corresponding to the second point B. Specifically, the processor 130 may classify the plurality of pixels 10 arranged in the first to third rows as the first pixel group 10-A corresponding to the first point on the object, and classify the plurality of pixels 10 arranged in the fourth to sixth rows as the second pixel group 10-B corresponding to the second point on the object.

[0149] According to an embodiment of the present disclosure, when the plurality of pixels 10 receive diffuse light from a plurality of different points on the object, the processor 130 may identify a plurality of first pixels 20 and a plurality of second pixels 30 in each of a plurality of groups based on a plurality of acquired pixel values. The processor 130 may identify a distance between each of a plurality of points on the object and the Lidar sensor 100 based on the plurality of first pixels and the plurality of second pixels identified in each of the plurality of groups.

[0150] Specifically, refer to Fig.10 , the processor 130 may receive the reflected light of the first frequency through the iToF sensor 120 to obtain a first pixel value corresponding to a plurality of pixels 10 included in the first pixel group 10-A. The processor 130 may obtain a first reliability level based on a plurality of first pixel values ​​of the plurality of pixels 10 included in the first pixel group 10-A. The processor 130 may select a first pixel 20-A in the first pixel group 10-A based on the obtained first reliability level. The processor 130 may obtain a first distance value corresponding to the first pixel 20-A of the first pixel group 10-A based on a plurality of third pixel values ​​of the selected first pixel 20-A. Similarly, the processor 130 may obtain a second distance value corresponding to the second pixel 30-A of the first pixel group 10-A. The processor 130 may identify the distance between the Lidar sensor 100 and the first point on the object 200 based on the obtained first distance value corresponding to the first pixel 20-A of the first pixel group 10-A and the obtained second distance value corresponding to the second pixel 30-A of the first pixel group 10-A.

[0151] In addition, the processor 130 may receive the reflected light of the first frequency through the iToF sensor 120 to obtain a first pixel value corresponding to a plurality of pixels 10 included in the second pixel group 10-B. The processor 130 may obtain a first reliability level based on a plurality of first pixel values ​​of the plurality of pixels 10 included in the second pixel group 10-B. The processor 130 may select a first pixel 20-B in the second pixel group 10-B based on the obtained first reliability level. The processor 130 may obtain a first distance value corresponding to the first pixel 20-B of the second pixel group 10-B based on a plurality of third pixel values ​​of the selected first pixel 20-B. Similarly, the processor 130 may obtain a second distance value corresponding to the second pixel 30-B of the second pixel group 10-B. The processor 130 may identify the distance between the Lidar sensor 100 and the second point on the object 200 based on the obtained first distance value corresponding to the first pixel 20-B of the second pixel group 10-B and the obtained second distance value corresponding to the second pixel 30-B of the second pixel group 10-B.

[0152] As such, the Lidar sensor 100 according to an embodiment of the present disclosure may output light to the object 200 in a wider range than through the diffusion unit, and thus may recognize distances to a plurality of points on the object based on acquired pixel values ​​of a plurality of pixels 10 .

[0153] In addition, according to an embodiment of the present disclosure, at least one or more of the plurality of pixels 10 arranged in a matrix form may be overlapped and included in different pixel groups. In this case, the presence or absence, quantity, position, etc. of at least one pixel included in each pixel group may be set based on the distance, position, etc. of different points to be identified.

[0154] In addition, according to an embodiment of the present disclosure, the processor 130 may apply a filter 50 having a preset size to each classified pixel group. That is, the processor 130 may select a plurality of pixels 10 by applying a filter 50 having a preset size to each pixel group, and then identify a first pixel 20 and a second pixel 30 from the plurality of selected pixels 10. In this regard, the above-described embodiment of the present disclosure in which a filter 50 having a preset size is applied to a plurality of pixels 10 may be similarly applied.

[0155] In addition, according to an embodiment of the present disclosure, the Lidar sensor 100 may further include a memory and a communication interface.

[0156] The memory may store data necessary for various embodiments of the present disclosure. For example, a preset first value for selecting the first pixel 20, a preset second value for selecting the second pixel 30, information about the filter 50 having a preset size, etc. may be stored in the memory.

[0157] The communication interface receives various types of information for driving the Lidar sensor 100. For example, the communication interface may receive a signal from an external device (e.g., a source device), an external storage medium (e.g., a USB memory), an external server (e.g., a network hard disk), etc. in a streaming or downloading method through a communication method such as AP-based Wi-Fi (wireless LAN network), Bluetooth, Zigbee, wired / wireless local area network (LAN), wide area network (WAN), Ethernet, IEEE 1394, High Definition Multimedia Interface (HDMI), Universal Serial Bus (UBS), Mobile High Definition Link (MHL), Audio Engineering Society / European Broadcasting Union (AES / EBU), optical and coaxial, etc.

[0158] Fig.11 is a flowchart schematically illustrating a method for controlling a Lidar sensor based on an iToF sensor according to an embodiment of the present disclosure.

[0159] Reference Fig.11 In order to control the Lidar sensor 100 based on the iToF sensor according to an embodiment of the present disclosure, when the Lidar sensor 100 rotates 360°, the processor 130 outputs light of a first frequency and light of a second frequency greater than the first frequency through the light emitting unit 110 of the Lidar sensor 100 (S1120).

[0160] The processor 130 receives the light of the first frequency and the light of the second frequency reflected by the object 200 through the plurality of pixels 10 included in the iToF sensor 120 ( S1120 ).

[0161] The processor 130 acquires a plurality of first pixel values ​​based on the reflected light of the first frequency, and acquires a plurality of second pixel values ​​based on the reflected light of the second frequency ( S1130 ).

[0162] After acquiring the first pixel value and the second pixel value, the processor 130 identifies a plurality of third pixel values ​​among the plurality of first pixel values ​​using reliability levels of the plurality of first pixel values ​​determined based on the sizes of the plurality of first pixel values ​​( S1140 ).

[0163] In this case, according to an embodiment of the present disclosure, each of the plurality of pixels 10 may include a first receiver 12 and a second receiver 14. When the reflected light of the first frequency is received by the first receiver 12 and the second receiver 14 included in each of the plurality of pixels 10, the processor 130 may acquire a first value corresponding to the first receiver 12 and a second value corresponding to the second receiver 14 as a plurality of first pixel values.

[0164] In this case, the processor 130 may identify each of the reliability levels (ie, first reliability levels) of the plurality of first pixel values ​​based on the following equation.

[0165] Reliability level of the first pixel value = In addition, according to an embodiment of the present disclosure, the processor 130 may identify multiple third pixel values, each of which has a reliability level greater than or equal to a preset value, based on the reliability levels of the multiple first pixel values ​​(ie, the first reliability levels).

[0166] After identifying the third pixel value, the processor 130 identifies a plurality of fourth pixel values ​​among the plurality of second pixel values ​​using reliability levels of the plurality of second pixel values ​​determined based on the sizes of the plurality of second pixel values ​​( S1150 ).

[0167] In this case, according to an embodiment of the present disclosure, when the reflected light of the second frequency is received by the first receiver 12 and the second receiver 14 included in each of the multiple pixels 10, the processor 130 may obtain a third value corresponding to the first receiver 12 and a fourth value corresponding to the second receiver 14 as multiple second pixel values.

[0168] In this case, the processor 130 may identify each of the reliability levels of the plurality of second pixel values ​​(ie, second reliability levels) based on the following equation.

[0169] Reliability level of the second pixel value = In addition, according to an embodiment of the present disclosure, the processor 130 may identify multiple fourth pixel values, each of which has a reliability level greater than or equal to a preset value, based on the reliability levels of the multiple second pixel values ​​(ie, the second reliability levels).

[0170] After identifying the plurality of third pixel values ​​and the plurality of fourth pixel values, the processor 130 identifies the distance between the Lidar sensor 100 and the object 200 based on at least one of a first distance value acquired based on the plurality of third pixel values ​​and a second distance value acquired based on the plurality of fourth pixel values ​​(S1160).

[0171] In addition, according to an embodiment of the present disclosure, the processor 130 may identify multiple distance values ​​based on multiple third pixel values, and identify the first distance value based on the average value of the identified multiple distance values. The processor 130 may identify multiple distance values ​​based on multiple fourth pixel values, and identify the second distance value based on the average value of the identified multiple distance values.

[0172] In addition, according to an embodiment of the present disclosure, the processor 130 may identify a distance value corresponding to a maximum reliability level among a plurality of distance values ​​identified based on a plurality of third pixel values ​​as a first distance value. The processor 130 may identify a distance value corresponding to a maximum reliability level among a plurality of distance values ​​identified based on a plurality of fourth pixel values ​​as a second distance value.

[0173] In addition, according to an embodiment of the present disclosure, the processor 130 may identify the distance between the Lidar sensor 100 and the object 200 by outputting light of the first frequency and the second frequency at each preset time interval. When the difference between the distance identified at the first time adjacent to each other and the distance identified at the second time after the preset time interval from the first time among the distances identified at each preset time interval is greater than or equal to a preset value, the processor 130 may identify the distance between the Lidar sensor and the object at the second time based on the first distance value.

[0174] In addition, before identifying the plurality of third pixel values ​​and the plurality of fourth pixel values, the processor 130 may apply the filter 50 having a preset size to the plurality of pixels 10 to obtain a plurality of filtered pixels 10 .

[0175] In this case, the processor 130 may identify a plurality of third pixel values ​​based on the first pixel values ​​of the plurality of pixels 10 filtered in step S1140. That is, the processor 130 may identify a first reliability level for each of the plurality of filtered pixels 10 based on a plurality of first pixel values ​​corresponding to each of the plurality of filtered pixels 10. The processor 130 may identify a plurality of third pixel values ​​among the plurality of first pixels 20 based on the first reliability level of each of the plurality of filtered pixels 10.

[0176] The processor 130 may identify a plurality of fourth pixel values ​​based on the second pixel values ​​of the plurality of pixels 10 filtered in step S1150. That is, the processor 130 may identify a second reliability level for each of the plurality of filtered pixels 10 based on a plurality of second pixel values ​​corresponding to each of the plurality of filtered pixels 10. The processor 130 may identify a plurality of fourth pixel values ​​in the plurality of second pixels 30 based on the second reliability level for each of the plurality of filtered pixels 10.

[0177] Fig.12 is a flowchart schematically illustrating a method for controlling a Lidar sensor based on an iToF sensor according to an embodiment of the present disclosure.

[0178] Fig.12 Steps S1210 to S1250 shown in Fig.11 Therefore, a detailed description thereof will be omitted.

[0179] Reference Fig.12 , the processor 130 may identify whether the difference between the first distance value and the second distance value is greater than or equal to a preset value (S1260). In this case, the preset value may be set based on the second frequency.

[0180] When it is identified that the difference between the first distance value and the second distance value is greater than or equal to the preset value, the processor 130 may determine a compensation value based on the first distance value and the second frequency (S1270).

[0181] The processor 130 may compensate the second distance value based on the determined compensation value, and identify the compensated second distance value as the distance between the Lidar sensor 100 and the object 200 ( S1280 ).

[0182] In addition, when the difference between the first distance value and the second distance value is less than a preset value, the processor 130 may recognize the second distance value as the distance between the Lidar sensor 100 and the object 200 .

[0183] Fig.13 is a schematic block diagram of a robot including a Lidar sensor based on an iToF sensor according to an embodiment of the present disclosure.

[0184] In addition, the iToF sensor based Lidar sensor 100 according to an embodiment of the present disclosure may be included in the robot 1000. Through the iToF sensor based Lidar sensor 100, the robot 1000 may detect an object 200 around the robot 1000 and recognize a distance between the detected object 200 and the robot 1000.

[0185] According to an embodiment of the present disclosure, the robot 1000 may include a Lidar sensor 100, a driving unit 200, a memory 300, a display 400, an input / output interface 500, a communication interface 600, and a processor 700. Since the Lidar sensor 100 has been described above, a detailed description thereof will be omitted.

[0186] The driving unit 200 is a component for moving the robot 1000. The driving unit 200 may be implemented as a wheel, a leg, etc. of the robot 1000. To this end, the driving unit 200 may include a motor.

[0187] The memory 300 may store data required for various embodiments of the iToF-based Lidar sensor 100 of the present disclosure or various data for driving the robot 1000. The memory 300 may be implemented in the form of a memory 300 embedded in the robot 1000, or in the form of a memory 300 detachable from the robot 1000 according to the data storage purpose. For example, data for driving the robot 1000 may be stored in the memory 300 embedded in the robot 1000, and data for extended functions of the robot 1000 may be stored in the memory 300 detachable from the robot 1000.

[0188] In addition, the memory 300 embedded in the robot 1000 may include at least one of the following: for example, a volatile memory (e.g., dynamic random access memory (DRAM), static RAM (SRAM), synchronous dynamic RAM (SDRAM), etc.), a non-volatile memory (e.g., one-time programmable read-only memory (OTPROM), programmable ROM (PROM), erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), mask ROM, flash ROM, etc.), a flash memory (e.g., NAND flash memory, NOR flash memory, etc.) a hard disk drive and a solid state drive (SSD)).

[0189] In addition, the memory 300 removable from the robot 1000 can be implemented in the form of a memory 300 card (e.g., compact flash (CF), secure digital (SD), micro secure digital (Micro-SD), mini secure digital (Mini-SD), extreme digital (xD), multimedia card (MMC)), etc.), an external memory connectable to a USB port (e.g., USB memory), etc.

[0190] The display 400 may display various types of visual information. For example, the display 400 may display the driving path information of the robot 1000, etc. In addition, the display 400 may be implemented by various types of displays, such as a liquid crystal display (LCD), an organic light emitting diode (OLED) display, a quantum dot light emitting diode (QLED) display, a plasma display panel (PDP), etc. A driving circuit, a backlight unit, etc., which may be implemented in a form such as a TFT, a low temperature polycrystalline silicon (LTPS) TFT, an organic TFT (OTFT), etc., may be included in the display 150. In addition, the display 400 may be implemented as a flexible display, a 3D display, etc.

[0191] The input / output interface 500 is a component for the robot 1000 to interact with a user, and the processor 700 may receive various types of information about the driving space through the input / output interface 500. In addition, the input / output interface 500 may include at least one of a touch sensor, a motion sensor, a button, a jog dial, a switch, or a speaker, but is not limited thereto.

[0192] The communication interface 600 can input and output various types of data. For example, the communication interface 600 can send and receive various types of data to and from an external device (e.g., a source device), an external storage medium (e.g., a USB memory), an external server (e.g., a network hard disk), etc. through communication methods such as AP-based Wi-Fi (wireless LAN network), Bluetooth, Zigbee, wired / wireless local area network (LAN), wide area network (WAN), Ethernet, IEEE 1394, High Definition Multimedia Interface (HDMI), Universal Serial Bus (UBS), Mobile High Definition Link (MHL), Audio Engineering Society / European Broadcasting Union (AES / EBU), optical and coaxial.

[0193] The processor 700 may control general operations of the robot 1000. To this end, the processor 700 may be connected to the Lidar sensor 100 to obtain distance information between the robot 1000 and the object 200 from the Lidar sensor 100. Alternatively, the processor 700 may control the driving unit 200 to control various driving operations of the robot 1000 such as movement, stop, speed control, and direction change.

[0194] In addition, the various embodiments described above may be implemented by software including instructions stored in a machine-readable storage medium (e.g., a computer-readable storage medium). A machine is a device that can call stored instructions from a storage medium and operate according to the called instructions, and may include the Lidar sensor 100 of the disclosed embodiment. In the case where the processor 130 executes a command, the processor 130 may directly execute the function corresponding to the command, or other components may execute the function corresponding to the command under the control of the processor. The command may include a code created or executed by a compiler or an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, the term "non-transitory" means that the storage medium is tangible without including a signal, and does not distinguish whether the data is stored semi-permanently or temporarily in the storage medium.

[0195] In addition, the above methods according to various embodiments may be included and provided in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a storage medium (e.g., a compact disc read-only memory (CD-ROM)), which may be read by a machine or downloaded through an application store (e.g., PlayStore). TM In the case of online distribution, at least a portion of the computer program product may be at least temporarily stored in a storage medium (such as a memory of a manufacturer's server, an application store's server, or a relay server), or may be temporarily generated.

[0196] In addition, each of the components (e.g., modules or programs) according to the various embodiments described above may include a single entity or multiple entities, and some of the corresponding subcomponents described above may be omitted, or other subcomponents may be further included in various embodiments. Alternatively or additionally, some components (e.g., modules or programs) may be integrated into one entity and perform the same or similar functions performed by each corresponding component before integration. The operations performed by the modules, programs or other components according to the various embodiments may be performed in a sequential manner, in a parallel manner, in an iterative manner or in a heuristic manner, at least some of the operations may be performed in a different order or omitted, or other operations may be added.

[0197] Although exemplary embodiments of the present disclosure have been shown and described above, the present disclosure is not limited to the above-mentioned specific exemplary embodiments, but various modifications may be made by those skilled in the art to which the present disclosure belongs without departing from the gist of the present disclosure disclosed in the appended claims. These modifications should also be understood to fall within the scope and spirit of the present disclosure.

Claims

1. A Lidar sensor, comprising: Light emitting unit; An indirect time-of-flight iToF sensor, comprising multiple pixels; as well as One or more processors configured to: When the Lidar sensor rotates, light of a first frequency and light of a second frequency greater than the first frequency are output through the light emitting unit, Based on the output light of the first frequency and the output light of the second frequency being reflected by the object and received by the plurality of pixels, a plurality of first pixel values ​​are acquired based on the reflected light of the first frequency, and a plurality of second pixel values ​​are acquired based on the reflected light of the second frequency, identifying a plurality of third pixel values ​​from the plurality of first pixel values ​​using reliability levels of the plurality of first pixel values ​​determined based on the magnitudes of the plurality of first pixel values, and identifying a plurality of fourth pixel values ​​from the plurality of second pixel values ​​using reliability levels of the plurality of second pixel values ​​determined based on the magnitudes of the plurality of second pixel values, and The distance between the Lidar sensor and the object is identified based on at least one of a first distance value and a second distance value, wherein the first distance value is obtained based on the plurality of third pixel values ​​and the second distance value is obtained based on the plurality of fourth pixel values.

2. The Lidar sensor according to claim 1, wherein: The one or more processors are further configured to: identifying, from the plurality of first pixel values, the plurality of third pixel values ​​each having a reliability level greater than or equal to a first preset value based on the reliability levels of the plurality of first pixel values, and identifying, based on the reliability levels of the plurality of second pixel values, the plurality of fourth pixel values ​​each having a reliability level greater than or equal to a second preset value from the plurality of second pixel values, and Among them, as the size of the pixel value in the plurality of first pixel values ​​and the plurality of second pixel values ​​increases, each of the reliability levels of the plurality of first pixel values ​​and the reliability levels of the plurality of second pixel values ​​is higher.

3. The Lidar sensor according to claim 2, wherein: The one or more processors are further configured to obtain a plurality of filtered pixels by applying a filter having a preset size to the plurality of pixels, and identify the plurality of third pixel values ​​and the plurality of fourth pixel values ​​based on pixel values ​​of the plurality of filtered pixels.

4. The Lidar sensor according to claim 1, wherein: The one or more processors are further configured to: identifying a plurality of distance values ​​based on the plurality of third pixel values, and identifying a first distance value based on an average of the identified plurality of distance values, and A plurality of distance values ​​are identified based on the plurality of fourth pixel values, and a second distance value is identified based on an average of the identified plurality of distance values.

5. The Lidar sensor according to claim 1, wherein: The one or more processors are further configured to: identifying, as a first distance value, a distance value corresponding to a maximum reliability level among a plurality of distance values ​​identified based on the plurality of third pixel values, and A distance value corresponding to a maximum reliability level among the plurality of distance values ​​identified based on the plurality of fourth pixel values ​​is identified as a second distance value.

6. The Lidar sensor according to claim 1, wherein: The one or more processors are further configured to: determining a compensation value based on the first distance value and the second frequency based on a difference between the first distance value and the second distance value being greater than or equal to a preset value, compensating the second distance value based on the determined compensation value, and The compensated second distance value is identified as the distance between the Lidar sensor and the object.

7. The Lidar sensor according to claim 1, wherein: The one or more processors are further configured to identify the second distance value as a distance between the Lidar sensor and the object based on a difference between the first distance value and the second distance value being less than a preset value.

8. The Lidar sensor according to claim 1, wherein: The one or more processors are further configured to: outputting light of a first frequency and light of a second frequency at each preset time interval to identify the distance between the Lidar sensor and the object, and Based on the difference between the distance identified at the first time and the distance identified at the second time among the distances identified at each preset time interval being greater than or equal to a preset value, identifying the distance between the Lidar sensor and the object at the second time based on the first distance value, and The second time is a time after a preset time interval from the first time.

9. The Lidar sensor according to claim 1, wherein: The plurality of first pixel values ​​corresponding to one of the plurality of pixels include a first value and a second value, and the plurality of second pixel values ​​corresponding to one of the plurality of pixels include a third value and a fourth value, and The one or more processors are further configured to identify each of the reliability levels of the plurality of first pixel values ​​and each of the reliability levels of the plurality of second pixel values ​​based on the following equations, respectively. Reliability level of the first pixel value = Reliability level of the second pixel value = 10. The Lidar sensor according to claim 1, wherein: Each of the plurality of pixels includes a plurality of receivers configured to be activated at preset time intervals.

11. A method of controlling a Lidar sensor including an indirect time-of-flight (iToF) sensor, the method comprising: When the Lidar sensor rotates, light of a first frequency and light of a second frequency greater than the first frequency are output through a light emitting unit of the Lidar sensor; receiving, by a plurality of pixels included in the iToF sensor, light of a first frequency and light of a second frequency reflected by an object; Acquire a plurality of first pixel values ​​based on the reflected light of the first frequency, and acquire a plurality of second pixel values ​​based on the reflected light of the second frequency; identifying a plurality of third pixel values ​​from the plurality of first pixel values ​​using reliability levels of the plurality of first pixel values ​​determined based on magnitudes of the plurality of first pixel values; identifying a plurality of fourth pixel values ​​from the plurality of second pixel values ​​using reliability levels of the plurality of second pixel values ​​determined based on magnitudes of the plurality of second pixel values; as well as The distance between the Lidar sensor and the object is identified based on at least one of a first distance value and a second distance value, wherein the first distance value is obtained based on the plurality of third pixel values ​​and the second distance value is obtained based on the plurality of fourth pixel values.

12. The method of claim 11, wherein: Identifying the plurality of third pixel values ​​comprises: based on the reliability levels of the plurality of first pixel values, identifying the plurality of third pixel values ​​each having a reliability level greater than or equal to a first preset value from the plurality of first pixel values, Wherein, identifying the plurality of fourth pixel values ​​comprises: based on the reliability levels of the plurality of second pixel values, identifying the plurality of fourth pixel values ​​each having a reliability level greater than or equal to a second preset value from the plurality of second pixel values, and Among them, as the size of the pixel value in the plurality of first pixel values ​​and the plurality of second pixel values ​​increases, each of the reliability levels of the plurality of first pixel values ​​and the reliability levels of the plurality of second pixel values ​​is higher.

13. The method according to claim 12, further comprising: obtaining a plurality of filtered pixels by applying a filter having a preset size to the plurality of pixels, wherein identifying the plurality of third pixel values ​​comprises identifying the plurality of third pixel values ​​based on the first pixel values ​​of the plurality of filtered pixels, and Wherein, identifying the plurality of fourth pixel values ​​comprises identifying the plurality of fourth pixel values ​​based on the second pixel values ​​of the plurality of filtered pixels.

14. The method according to claim 11, wherein: Identifying the distance includes: identifying a plurality of distance values ​​based on the plurality of third pixel values, and identifying a first distance value based on an average of the identified plurality of distance values; and A plurality of distance values ​​are identified based on the plurality of fourth pixel values, and a second distance value is identified based on an average of the identified plurality of distance values.

15. The method according to claim 10, wherein: Identifying the distance includes: identifying, as a first distance value, a distance value corresponding to a maximum reliability level among a plurality of distance values ​​identified based on the plurality of third pixel values; and A distance value corresponding to a maximum reliability level among the plurality of distance values ​​identified based on the plurality of fourth pixel values ​​is identified as a second distance value.

Images