Electronic device and control method thereof

Abstract

An electronic device is provided. The electronic device according to the disclosure includes a light emitter, a light receiver, a memory, and a processor, wherein the processor is configured to perform the following operations: acquire a first depth image based on first reflected light acquired during a first time period, and store the first depth image in the memory, acquire a second depth image based on second reflected light acquired during a second time period after the first time period, and acquire distance information between the electronic device and an object included in the second depth image by subtracting a first depth value of each pixel of the first depth image from a second depth value of each pixel of the second depth image, and correct the distance information by using a compensation value acquired corresponding to time information about the second time period.

Description

TECHNICAL FIELD

[0001] The disclosure relates to an electronic device and a control method thereof, and more particularly, to an electronic device for acquiring distance information of an object based on a depth image and a control method thereof. BACKGROUND

[0002] A time-of-flight (TOF) sensor acquires distance information of an object based on a time of flight or phase information of light emitted toward the object. According to a method of acquiring distance information, the TOF sensor is divided into a direct TOF method of measuring a round-trip time of flight of light and an indirect TOF method of measuring a phase difference between emitted light and reflected light. SUMMARY

[0003] TECHNICAL PROBLEM

[0004] A disadvantage of the TOF sensor using the direct TOF method is that an expensive time-to-digital converter (TDC) element is required to perform high-speed measurement, and thus has a high unit cost. In addition, because the TOF sensor using the indirect TOF method measures a distance based on a phase difference between emitted light and reflected light, there is a problem that an object located at a distance beyond one period of an emitted light signal cannot be accurately measured.

[0005] Accordingly, there is a need for a technology capable of measuring an object existing at a long distance at a low cost.

[0006] TECHNICAL SOLUTION

[0007] An electronic device capable of measuring a distance of an object existing at a long distance by using a phase shift of light is provided.

[0008] Additional aspects will be set forth in part in the description which follows, and in part will become apparent to those skilled in the art by reference to the description, or can be learned by practice of the presented embodiments.

[0009] According to an aspect of the disclosure, an electronic device is provided, including: a light emitter configured to emit light as emission light toward an area around the electronic device; a light receiver configured to acquire reflected light from an object, wherein the object reflects the emission light emitted from the light emitter; a memory storing one or more instructions; and a processor configured to execute the one or more instructions to perform operations of: acquiring a first depth image based on first reflected light acquired through the light receiver during a first time period, and storing the first depth image in the memory, acquiring a second depth image based on second reflected light acquired through the light receiver during a second time period after the first time period, and acquiring distance information between the electronic device and the object included in the second depth image by subtracting a first depth value of each pixel of the first depth image stored in the memory from a second depth value of each pixel of the second depth image, and correcting the distance information by using a compensation value acquired based on time information corresponding to the second time period.

[0010] The processor can be further configured to execute the one or more instructions to acquire the compensation value based on the time information corresponding to the second time period and a wavelength of the emission light.

[0011] The processor can be further configured to execute the one or more instructions to perform operations of: acquiring image data based on the emission light and the first reflected light, acquiring phase information of the first reflected light based on the image data, and acquiring the first depth image based on the phase information of the first reflected light.

[0012] The light receiver can include a plurality of sensors, wherein the plurality of sensors include a first sensor and a second sensor, and the processor can be further configured to execute the one or more instructions to perform operations of: controlling the light receiver such that the first sensor and the second sensor are activated with a preset time difference, acquiring the image data by acquiring first image data and second image data based on the first reflected light acquired by each of the first sensor and the second sensor, and acquiring the phase information of the first reflected light based on the first image data and the second image data.

[0013] The light receiver can include a first sensor unit and a second sensor unit, wherein each of the first sensor unit and the second sensor unit includes a plurality of sensors, and the processor can be further configured to execute the one or more instructions to control the light receiver such that the first sensor unit is activated synchronously with the first time period, and the second sensor unit is activated synchronously with the second time period.

[0014] The electronic device can further include a lens unit disposed on a transmission path of the emitted light, wherein the lens unit is configured to adjust a transmission range of the emitted light, and wherein the processor is further configured to execute the one or more instructions to control the lens unit to adjust the transmission range of the emitted light in synchronization with the first time period and the second time period.

[0015] The lens unit can include a non-reflective coating member configured to prevent at least a portion of the reflected light acquired by the light receiver from being reflected by the light receiver and then reflected by the lens unit to reach the light receiver again.

[0016] The processor can be further configured to execute the one or more instructions to acquire a confidence image indicating reliability of a second depth value of each pixel of a second depth image, and update the second depth image using the confidence image.

[0017] According to another aspect of the disclosure, a method for controlling an electronic device is provided, the method including: emitting light toward an area around the electronic device as emitted light; acquiring reflected light from an object, wherein the object reflects the emitted light emitted from the light emitter; acquiring and storing a first depth image based on first reflected light acquired during a first time period; acquiring a second depth image based on second reflected light acquired during a second time period after the first time period; and acquiring distance information between the electronic device and the object included in the second depth image by subtracting a first depth value of each pixel of the stored first depth image from a second depth value of each pixel of the second depth image, and correcting the difference information by using a compensation value acquired based on time information corresponding to the second time period to acquire a distance.

[0018] The step of correcting the difference information can include acquiring the compensation value based on the time information corresponding to the second time period and a wavelength of the emitted light.

[0019] The step of acquiring the first depth image can include acquiring image data based on the emitted light and the first reflected light, acquiring phase information of the first reflected light based on the image data, and acquiring the first depth image based on the phase information about the first reflected light.

[0020] In acquiring the phase information, the phase information about the first reflected light can be acquired based on a plurality of image data acquired through a plurality of sensors activated by a preset time difference.

[0021] The step of acquiring the first depth image can include controlling a first sensor unit and a second sensor unit so that the first sensor unit and the second sensor unit each including a plurality of sensors are activated in synchronization with the first time period and the second time period, respectively.

[0022] The method can further include controlling an emission range of the emission light in synchronization with the first time period and the second time period.

[0023] The acquiring of the second depth image can include acquiring a confidence image indicating reliability of a second depth value of each pixel of the second depth image, and updating the second depth image using the confidence image.

[0024] According to another aspect of the disclosure, an electronic device is provided, including a memory storing one or more instructions, and a processor configured to execute the one or more instructions to perform operations of: acquiring first reflected light during a first time period; acquiring second reflected light during a second time period; acquiring a first depth image based on the first reflected light; acquiring a second depth image based on the second reflected light; acquiring a virtual depth image by subtracting a first depth value of each pixel of the first depth image from a second depth value of each pixel of the second depth image, and acquiring distance information about a first object in the second depth image after performing correction on the second depth image.

[0025] The first reflected light and the second reflected light can be acquired by repeatedly emitting light in an area around the electronic device.

[0026] The first depth image can include first depth information about a second object, and the second depth image includes second depth information about the first object and the second object.

[0027] The processor can be further configured to execute the one or more instructions to acquire a compensation value for performing correction on the second depth image based on a frequency, a luminous flux of the emission light, and a number corresponding to the second time period.

[0028] Advantageous effects

[0029] The electronic device can obtain distance information about an object existing at a long distance at a low cost. BRIEF DESCRIPTION OF DRAWINGS

[0030] The above and other aspects, features, and advantages of certain embodiments of the disclosure will be more apparent from the following description taken in conjunction with the following drawings, in which:

[0031] Figure 1 is a diagram illustrating a configuration of an electronic device according to an embodiment of the disclosure;

[0032] Figure 2 is a diagram for describing an indirect TOF method;

[0033] Figure 3 is a diagram for describing an indirect TOF method;

[0034] Figure 4is a diagram illustrating a configuration of an electronic device according to an embodiment;

[0035] Figure 5 is a diagram for describing an operation of an electronic device according to an embodiment;

[0036] Figure 6A is a flowchart for describing an operation of an electronic device according to an embodiment;

[0037] Figure 6B is a diagram for describing a method for acquiring image data according to an embodiment;

[0038] Figure 7 is a diagram illustrating a light receiver according to an embodiment;

[0039] Figure 8 is a diagram illustrating a configuration of an electronic device according to another embodiment;

[0040] Figures 9A to 9C is a diagram illustrating a state in which a field of view of an electronic device and a transmission distance of emitted light are adjusted according to an embodiment;

[0041] Figure 10 is a diagram for describing a lens flare;

[0042] Figure 11 is a flowchart illustrating a method for controlling an electronic device according to an embodiment; and

[0043] Figure 12 is a block diagram illustrating a configuration of an electronic device according to an embodiment. DETAILED DESCRIPTION

[0044] BEST MODE

[0045] In consideration of the functions in the present disclosure, general terms widely used at the time of preparation of the present disclosure are selected as terms used in the embodiments of the present disclosure, but can be changed according to the intentions of the skilled person or judicial precedents, appearance of new technologies, etc. In addition, there can be terms arbitrarily selected by the applicant in specific cases. In this case, the meanings of the terms will be mentioned in detail in the corresponding description part of the present disclosure. Therefore, the terms used in the present disclosure should be defined based on the meanings of the terms and the contents throughout the present disclosure rather than the simple names of the terms.

[0046] Embodiments of the present disclosure can be applied various modifications and have various embodiments, and a specific embodiment will be illustrated in the accompanying drawings and described in detail in the detailed description. However, this is not intended to limit the scope to a specific embodiment, and it should be understood to include all modifications, equivalents and alternatives included in the spirit and technical scope of the disclosure. In describing the embodiments, detailed descriptions of the related known technology will be omitted when it is determined that the detailed descriptions can obscure the gist.

[0047] The terms "first", "second", and the like can be used to describe various components, but the components should not be construed as being limited by the terms. The terms are used only to distinguish one component from other components.

[0048] Unless the context clearly indicates otherwise, the singular expression includes the plural expression. It should also be understood that the term "comprise" or "consist of" used in the present application specifies the existence of the features, numbers, steps, operations, components, parts or combinations thereof mentioned in the specification, but does not exclude the existence or addition of one or more other features, numbers, steps, operations, components, parts or combinations thereof.

[0049] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings so that those skilled in the art to which the present disclosure belongs can easily practice the present disclosure. However, the present disclosure can be implemented in various different forms and is not limited to the embodiments described herein. In addition, in the drawings, parts irrelevant to the description will be omitted to clearly describe the present disclosure, and similar parts will be denoted by similar reference numerals throughout the specification.

[0050] Figure 1 is a diagram for describing the operation of the electronic device 100 according to an embodiment. According to an embodiment, the electronic device 100 can be a TOF sensor, but is not limited thereto. According to another embodiment, the electronic device 100 can be one of various electronic devices equipped with a TOF sensor. For example, the electronic device can be one of various robot devices having a TOF sensor to measure the distance of a surrounding object. The robot device can be a robot cleaner, but is not limited thereto.

[0051] Referring to Figure 1 , the electronic device 100 can include a light emitter 110, a light receiver 120, and a processor 170. In addition, detailed descriptions of each component of the electronic device 100 will be described later, and the operation of the electronic device 100 will be described below. Figure 3

[0052] ​The electronic device 100 can emit the emission light 11 toward the object 13 using the light emitter 110. Also, the electronic device 100 can receive the reflection light 12 reflected from the object 13 through the light receiver 120. The reflection light 12 is the emission light 11 reflected by the object 13. At this time, the electronic device 100 can acquire the distance from the electronic device 100 to the object 13 based on the time of flight of the emission light 11 (i.e., the time from when the electronic device 100 emits the emission light 11 and the unit time when the electronic device 100 receives the reflection light again).

[0053] As an example of a method of acquiring a distance using the time of flight of light, there is a direct TOF method. A TOF sensor using the direct TOF method calculates the time of flight of light by comparing the emission time point of the emission light 11 and the reception time point of the reflection light 12, and acquires distance information about the object 13 based on the calculated time of flight. On the other hand, the TOF sensor using the direct TOF method needs to modulate the emission light 11 at high speed in order to acquire the accurate time point of receiving the reflection light 12, and needs to include an expensive time-to-digital converter (TDC) element for high-speed measurement. Therefore, the related-art TOF sensor using the direct TOF method has the disadvantage of high manufacturing cost.

[0054] As another example of a method of acquiring a distance using the time of flight of light, there is an indirect TOF method. The indirect TOF method refers to a method of calculating the time of flight of light based on the phase difference (i.e., the phase shift of light) between the emission light 11 and the reflection light 12. Hereinafter, the indirect TOF method will be described in more detail with reference to Figure 2 and Figure 3 The indirect TOF method will be described in more detail.

[0055] Figure 2 and Figure 3 are diagrams for describing the indirect TOF method; as described above, a sensor using the indirect TOF method (hereinafter, referred to as an indirect TOF sensor) can acquire first phase information about the emission light 21 and second phase information about the reflection light 22. The indirect TOF sensor can acquire a first phase difference θ1 by comparing the first phase information and the second phase information, and acquire distance information about the object 13 based on the first phase difference θ1.

[0056] On the other hand, because it can be impossible to recognize a phase difference of 360 degrees or more, the indirect TOF sensor has a problem when long distance measurement is required. Specifically, the indirect TOF sensor calculates the distance of an object based on the phase of the emission light and the phase of the reflection light belonging to the time period when the reflection light is acquired. Therefore, as Figure 3As illustrated, when the reflected light 32 having a phase difference of 360 degrees or more from the emitted light 31 is acquired, the indirect TOF sensor calculates the distance of the object 13 based on the first phase difference θ1 rather than the second phase difference θ2 (i.e., θ1+360°).

[0057] Accordingly, the indirect TOF sensor acquires a distance value smaller than the actual distance of the object 13 as the distance of the object 13. Thus, the problem of the indirect TOF sensor is that it is impossible to acquire an accurate distance of the object 13 located at a long distance (i.e., when the phases of the emitted light and the reflected light differ by 360 degrees or more).

[0058] Hereinafter, an electronic device 100 capable of acquiring an accurate distance of an object located at a long distance while using the above-described indirect TOF method, which is different from the prior art indirect TOF sensor, will be described.

[0059] Figure 4 is a diagram illustrating a configuration of an electronic device according to an embodiment.

[0060] Referring to Figure 4 The electronic device 100 can include a light emitter 110, a light receiver 120, a memory 130, and a processor 170. The electronic device 100 can be a TOF sensor or various electronic devices equipped with a TOF sensor. For example, the electronic device can be various robot devices including a robot cleaner, but is not limited thereto. On the other hand, the electronic device 100 does not necessarily have to be implemented to include all of the above-described components, and can be implemented by omitting or adding some components.

[0061] Hereinafter, the configuration of the above-described components will be described in detail.

[0062] The light emitter 110 can emit light toward an object. At this time, the light emitted from the light emitter 110 (hereinafter, referred to as emitted light) can have a waveform in the form of a sine wave. However, this is only an example, and the emitted light can have a waveform in the form of a square wave. In addition, the light emitter 110 can include various types of laser elements. For example, the light emitter 110 can include a vertical cavity surface emitting laser (VCSEL) or a laser diode (LD). In addition, the light emitter 110 can include a plurality of laser elements. At this time, the plurality of laser elements can be arranged in the form of an array. In addition, the light emitter 110 can emit light of various frequency bands. For example, the light emitter 110 can emit a laser beam having a frequency of 100 MHz.

[0063] The light receiver 120 can be a component for acquiring reflected light reflected from an object. The light receiver 120 can include various types of sensor elements. For example, the light receiver 120 can include a charge-coupled device (CCD) sensor, a complementary metal-oxide semiconductor (CMOS) sensor, a photodiode (PD), and an avalanche photodiode (APD). In addition, the light receiver 120 can include a plurality of sensor units configured with a plurality of sensor elements (or sensors). At this time, each of the plurality of sensor elements and the sensor units can be arranged in an array form. The plurality of sensors can be activated at a preset time interval to acquire reflected light at a preset time interval.

[0064] The memory 130 can store an operating system (OS) for controlling overall operations of the components of the electronic device 100, and commands or data related to the components of the electronic device 100. According to an embodiment, the memory 130 can be implemented as a non-volatile memory (e.g., a hard disk, a solid state drive (SSD), or a flash memory), a volatile memory, or the like.

[0065] The processor 170 can control overall operations of the electronic device 100. The processor 170 can include an image data acquisition module 171, a phase information acquisition module 172, a depth image acquisition module 173, and an object distance acquisition module 174.

[0066] The image data acquisition module 171 can acquire image data based on the reflected light acquired by the light receiver 120. According to an embodiment, the image data acquisition module 171 can acquire image data based on an electrical signal of the reflected light and an electrical signal of the emitted light. The image data can be a pixel value of each pixel of an image acquired through the light receiver 120. In addition, the image data acquisition module 171 can acquire image data corresponding to each reflected light acquired by the plurality of sensors included in the light receiver 120. When the plurality of sensors acquire the reflected light at a certain interval, the image data acquisition module 171 can acquire image data of each reflected light acquired at the certain interval. According to an embodiment, the interval can have a preset time difference. For example, the image data acquisition module 171 can acquire first image data to fourth image data acquired by first to fourth sensors at a preset time interval.

[0067] The phase information acquisition module 172 can acquire phase information about the reflected light based on the image data acquired through the image data acquisition module 171. The phase information can include a phase difference value between the emitted light and the reflected light corresponding to each pixel of the light receiver 120. Specifically, the phase information acquisition module 172 can acquire a phase difference between the emitted light and the reflected light by applying the first image data to the fourth image data acquired through the image data acquisition module 171 to a predefined mathematical expression. In addition, a detailed description thereof will be made later with reference to Figure 6A andFigure 6B The method of acquiring the phase difference between the emitted light and the reflected light is described in more detail.

[0068] The depth image acquisition module 173 can acquire a depth image based on the phase information acquired through the phase information acquisition module 172. Specifically, the depth image acquisition module 173 can acquire a time of flight of light acquired through the light receiver 120 corresponding to light emitted from the light emitter 110 based on the phase difference value between the emitted light and the reflected light included in the phase information. That is, the depth image acquisition module 173 can acquire the time of flight of light using an indirect TOF method. In addition, the depth image acquisition module 173 can acquire a depth value of each pixel of the depth image based on the time of flight of light.

[0069] The object distance acquisition module 174 can acquire distance information about an object included in the depth image based on the depth image acquired through the depth image acquisition module 173. Here, the distance information about the object can indicate a distance from the electronic device 100 to the object. Specifically, the object distance acquisition module 174 can detect the object based on a distribution of the depth value of each pixel of the depth image. In addition, the object distance acquisition module 174 can acquire the depth value of the pixel corresponding to the detected object as the distance to the object.

[0070] In addition, the depth image acquired through the depth image acquisition module 173 can include an error due to a characteristic of the indirect TOF method. Specifically, the depth value of the pixel corresponding to the object existing at a distance of a preset distance range or more from the electronic device 100 can have an error. Accordingly, the object distance acquisition module 174 can correct the distance to the object based on the depth image.

[0071] The object distance acquisition module 174 can acquire a plurality of depth images at each interval, that is, at each preset time period. For example, the object distance acquisition module 174 can acquire a first depth image corresponding to a first time period and store the first depth image in the memory 130. In addition, the object distance acquisition module 174 can acquire a second depth image corresponding to a second time period, which is a time period immediately after the first time period.

[0072] The object distance acquisition module 174 can perform an operation of subtracting the depth value of each pixel of the first depth image from the depth value of each pixel of the second depth image. At this time, the object distance acquisition module 174 can acquire distance information about at least one object included in the second depth image. Specifically, the object distance acquisition module 174 can acquire distance information about a second object present in the second depth image but not present in the first depth image. Specifically, the object distance acquisition module 174 can acquire distance information about the second object by identifying the depth value of the pixel corresponding to the second object.

[0073] In addition, the object distance acquisition module 174 can correct the distance information about the second object by using a compensation value acquired based on time information about the second time period. At this time, the object distance acquisition module 174 can acquire the compensation value based on the time information about the second time period and the wavelength of the emission light emitted from the light emitter 110. The object distance acquisition module 174 can acquire corrected distance information of the second object by adding the acquired compensation value to the depth value of the pixel corresponding to the second object. In this way, the object distance acquisition module 174 can acquire distance information about an object included only in a depth image corresponding to a current time period by subtracting a depth image corresponding to an immediately previous time period from the depth image corresponding to the current time period. In addition, a more detailed operation of the object distance acquisition module 174 will be described later with reference to FIGS. 6 to 8. Figure 5 A more detailed operation of the object distance acquisition module 174 will be described.

[0074] In addition, the processor 170 can control the light receiver 120 so that the reflected light reflected from the object is acquired at a preset time interval and the plurality of sensors are activated at a preset time interval.

[0075] Each module included in the processor 170 described above can be implemented as software, but this is only an example, and can be implemented as hardware or a combination of software and hardware. Hereinafter, an object acquisition operation of the electronic device will be described in more detail.

[0076] Figure 5 is a diagram for describing an operation of an electronic device according to an embodiment.

[0077] The electronic device 100 can acquire the first reflected light 52 and the second reflected light 54 by repeatedly emitting the emission light 51 for the first time period T1. Specifically, the electronic device 100 can acquire the first reflected light 52 during a first time period T1-1 that is a first time period within the time period T1, and can acquire the second reflected light 54 during a second time period T1-2 that is a second time period.

[0078] The electronic device 100 can acquire the first depth image 53 based on the first reflected light 52, and can acquire the second depth image 55 based on the second reflected light 54. At this time, the first depth image 53 can include distance information about the first object ob1. The second depth image 55 can include distance information about the first object ob1 and the second object ob2.

[0079] The electronic device 100 can acquire distance information about the first object ob1 (i.e., a distance from the electronic device 100 to the first object ob1) based on the first depth image 53. At this time, because the first reflected light 52 has a phase difference of 360 degrees or less from the first periodic signal of the emission light, the electronic device 100 can acquire the distance information about the first object ob1 without correcting the depth data of the first depth image 53.

[0080] On the other hand, because the second reflected light 54 has a phase difference of 360 degrees or more from the first periodic signal of the emission light, the depth data of the second depth image 55 can include an error. Specifically, a distance value (or depth data) of the second object ob2 that is not present in the first depth image 53 but present in the second depth image 55 can be less than an actual distance value of the second object ob2. Accordingly, the electronic device 100 can acquire distance information about the second object ob2 after performing correction on the depth data of the second depth image 55.

[0081] Specifically, the electronic device 100 can acquire the virtual depth image 56 by subtracting the depth value of each pixel of the first depth image 53 from the depth value of each pixel of the second depth image 55. At this time, the depth value of a pixel corresponding to the first object ob1 that is commonly included in the first depth image 53 and the second depth image 55 can be reduced to less than a threshold value.

[0082] In addition, the electronic device 100 can acquire a compensation value (d a ) for correcting the depth value of a pixel corresponding to the second object ob2. Specifically, the electronic device 100 can acquire the compensation value (d a ) using [Mathematical Formula 1].

[0083] [Mathematical Formula 1]

[0084]

[0085] Here, f denotes a frequency of an emission light signal, C L denotes luminous flux (i.e., 300 M m / s), and n denotes time information of a time period.

[0086] For example, a first compensation value for correcting a first depth image 53 acquired based on first reflected light 52 acquired in a first time period T1-1 becomes 0 (i.e., n is 1). As another example, a second compensation value for correcting a second depth image 55 acquired based on second reflected light 54 acquired in a second time period T1-2 is 1.5 m

[0087] The electronic device 100 can correct distance information of an object by adding the acquired compensation value to the distance information of the object included in the depth image. For example, when f is 100 MHz, the electronic device 100 can acquire corrected object distance information 57 by adding the second compensation value (i.e., 1.5 m) to the depth value of the pixel corresponding to the second object ob2. Also, in Figure 5 In the above, for convenience of explanation, the virtual depth image 56 is illustrated as an image, but the electronic device 100 can acquire the corrected object distance information 57 based on the first depth image 53 and the second depth image 55 without generating the virtual depth image 56. Also, in the above, Figure 5 In the above, for convenience of explanation, only the first time period T1 is described, but the electronic device 100 can acquire the corrected object distance information 57 in a plurality of time periods. Also, the electronic device 100 can correct distance information of an object by calculating a compensation value based on a time period corresponding to a depth image acquired in each of a plurality of time periods. At this time, the size of each time period including the first time period T1 can be determined based on at least one of a time point at which the transmission light 51 is emitted and a measurement range of the electronic device 100.

[0088] Also, as described above, the electronic device 100 can acquire phase information (i.e., a phase difference between the transmission light and the reflected light) about the reflected light based on image data acquired every preset time period. Also, the electronic device 100 can acquire a depth image based on the phase information. Hereinafter, the operation of the above-described electronic device 100 will be described with reference to Figure 6A and Figure 6B

[0089] Figure 6A is a flowchart for describing an operation of an electronic device according to an embodiment. The electronic device 100 can include an image data acquisition module 171, a phase information acquisition module 172, a depth image acquisition module 173, and an object distance acquisition module 174.

[0090] The image data acquisition module 171 can acquire image data 61 based on the reflected light 60. Specifically, the image data acquisition module 171 can acquire image data based on electrical signals of the reflected light acquired through a plurality of sensors activated at a time difference of each preset time period. For example, as described above with reference to FIG. 2, the image data acquisition module 171 can acquire image data 61 based on the electrical signals of the reflected light 60 acquired through the plurality of sensors 110-1, 110-2, and 110-3. Figure 6B ​As illustrated, the first sensor 620-1, the second sensor 620-2, the third sensor 620-3, and the fourth sensor 620-4 can be sequentially activated to acquire the reflected light 60. At this time, the image data acquisition module 171 can acquire the first image data to the fourth image data I1, I2, I3, and I4 based on the electrical signals 60-1, 60-2, 60-3, and 60-4 of the reflected light acquired by the first sensor to the fourth sensor, respectively. Here, the electronic device 100 can control each of the first sensor 620-1, the second sensor 620-2, the third sensor 620-3, and the fourth sensor 620-4 to be activated at a time difference corresponding to a 90-degree phase difference with respect to the electrical signal of the emitted light. In addition, each of the acquired image data I1, I2, I3, and I4 can include the intensity of the plurality of sensors 620-1, 620-2, 620-3, and 620-4 for each pixel.

[0091] The phase information acquisition module 172 can acquire the phase information 62 by applying the image data 61 to [Mathematical Formula 2]. 62.

[0092] [Mathematical Formula 2]

[0093]

[0094] Here, the phase information 62 is a phase difference between the emitted light and the reflected light, and I1 to I4 are the first image data to the fourth image data, respectively.

[0095] In addition, in Figure 6B which the first image data to the fourth image data I1, I2, I3, and I4 have been described to be acquired by four sensors, the disclosure is not limited to four sensors, and the electronic device 100 can acquire the first image data to the fourth image data I1, I2, I3, and I4 using two sensors. For example, the first image data I1 and the third image data I3 can be acquired by the first sensor 620-1, and the second image data I2 and the fourth image data I4 can be acquired by the second sensor 620-2.

[0096] The depth image acquisition module 173 can acquire the depth image 63 by applying the phase information 62 acquired via the phase information acquisition module 172 to [Mathematical Formula 3].

[0097] [Mathematical Formula 3]

[0098]

[0099] Here, C L is the speed of light, Δt is the time of flight of light, T is the modulation period of the emitted light, and is phase information 62.

[0100] The object distance acquisition module 174 can acquire corrected object distance information 64 from the depth image 63 by performing a correction operation as described in Figure 5

[0101] Figure 7 is a schematic diagram illustrating a light receiver according to an embodiment.

[0102] The light receiver 120 can include a sensor module configured with a plurality of sensor units arranged in an array form. The sensor module can include a first sensor unit 620 and a second sensor unit 720. As Figure 6B described, the electronic device 100 can acquire a depth image through a plurality of sensors belonging to the first sensor unit 620. In addition, as Figure 5 illustrated, the electronic device 100 can acquire a plurality of depth images corresponding to a preset time period.

[0103] Hereinafter, an implementation example of the light receiver 120 for such an operation will be described.

[0104] The electronic device 100 can acquire a first depth image 53 by activating the first sensor unit 620 in a first time period T1-1. At this time, the electronic device 100 can acquire the first depth image 53 by sequentially activating a plurality of sensors belonging to the first sensor unit 620 at predetermined time intervals. In addition, the electronic device 100 can acquire a second depth image 55 by activating the second sensor unit 720 in a second time period T1-2. At this time, the electronic device 100 can acquire the first depth image 53 by sequentially activating a plurality of sensors belonging to the second sensor unit 720 at predetermined time intervals. In addition, the electronic device 100 can acquire the first depth image 53 and the second depth image 55 by activating other sensor units belonging to each row in which the first sensor unit 620 and the second sensor unit 720 are located. That is, the electronic device 100 can acquire a depth image by controlling the sensor units in units of rows or columns. In this way, the sensor module can acquire one depth image using all the sensor units, but can acquire a plurality of depth images using each sensor unit.

[0105] ​However, this is merely an example, and the electronic device 100 can acquire a depth image using the entirety of a plurality of sensor units including the first sensor unit 620 and the second sensor unit 720 in each time period. For example, the electronic device 100 can acquire a first depth image 53 by activating the entire sensor unit in a first time period T1-1. In addition, the electronic device 100 can acquire a second depth image 55 by activating the entire sensor unit in a second time period T1-2. In this way, by acquiring a depth image using the entire sensor unit included in the light receiver 120, the resolution of the depth image can be increased.

[0106] In the above, a method of acquiring distance information about a distant object by processing a depth image corresponding to each time period to correct an object distance has been described. On the other hand, in order to measure the distance of an object present at a long distance, the emitted light needs to be able to reach the long distance. There is a method of increasing the intensity of the emitted light in order to emit the emitted light to the long distance, but increasing the intensity of the emitted light increases power consumption. Hereinafter, an electronic device capable of transmitting emitted light to a long distance by controlling a lens unit without increasing the intensity of the emitted light will be described.

[0107] Figure 8 is a diagram illustrating a configuration of an electronic device according to another embodiment.

[0108] Referring to Figure 8 , the electronic device 800 can include a light emitter 810, a light receiver 820, a memory 830, a processor 870, and a lens unit 840. In addition, the processor 870 can include an image data acquisition module 871, a phase information acquisition module 872, a depth image acquisition module 873, an object distance acquisition module 874, and a lens driving module 875. In addition, the remaining components of the electronic device 800 other than the lens driving module 875 and the lens unit 840 are the same as those of the electronic device 100 of Figure 4 the configuration of the electronic device 100 of

[0109] The lens unit 840 can be a component for controlling the emission direction of the emitted light from the light emitter 810. Specifically, the lens unit 840 can form a field of view (FOV) of the electronic device 800 by refracting the emitted light. For example, the electronic device 800 can have a field of view of 60 degrees in the horizontal direction. In addition, the lens unit 840 can include a zoom lens for adjusting the field of view, a collimator for forming parallel light from the emitted light, and a light receiver 820 for guiding the reflected light reflected from an object to the light receiver 820.

[0110] In addition, the lens driving module 875 can control the lens unit 840. Specifically, the lens driving module 875 can adjust the field of view of the electronic device 800 by controlling the zoom lens included in the lens unit 840. Specifically, the lens driving module 875 can adjust the field of view by adjusting the distance between the light emitter 810 and the zoom lens. In addition, the lens driving module 875 can control the emission distance of the emitted light by using the zoom lens included in the lens unit 840. For example, the lens driving module 875 can reduce the field of view of the electronic device 800 and increase the emission distance of the emitted light emitted from the light emitter 810 by adjusting the distance between the light emitter 810 and the zoom lens. Accordingly, the measurable distance of the electronic device 800 can be increased without increasing the power for outputting the light output from the light emitter 810.

[0111] Figure 9A Figure 9B Figure 9C are diagrams illustrating states of adjusting the field of view of the electronic device and the emission distance of the emitted light according to an embodiment.

[0112] As shown in Figure 9A Figure 9B Figure 9C , the electronic device 800 can adjust the field of view and the emission distance of the emitted light by controlling the lens unit 840.

[0113] The electronic device 800 according to Figure 9A may have a larger field of view and a smaller output distance of the emitted light than the electronic device 800 according to Figure 9B . In addition, the electronic device 800 according to Figure 9B may have a larger field of view and a smaller output distance of the emitted light than the electronic device 800 according to FIG. 9c.

[0114] ​​​​The lens driving module 875 can control the lens unit 840 so that the field of view of the electronic device 800 is adjusted in synchronization with the time period in which the depth image is acquired by the depth image acquisition module 873. Specifically, the lens driving module 875 can control the lens unit 840 so that the field of view of the electronic device 800 is changed every time period in which the depth image is acquired by the depth image acquisition module 873. For example, the lens driving module 875 can control the lens unit 840 so that the electronic device 800 has a first field of view FOV1 during a first time period T1-1. In addition, the lens driving module 875 can control the lens unit 840 so that the electronic device 800 has a second field of view (FOV2) that is smaller than the first field of view FOV1 during a second time period T1-2. In addition, the electronic device 800 can control the lens unit 840 so that a third field of view (FOV3) that is smaller than the second field of view FOV2 is formed during a third time period T1-3. In this way, the electronic device 800 can increase the emission distance of the emitted light while maintaining the light output power by controlling the lens unit 840. However, this is merely an example, and the field of view and the measurement range of the electronic device 800 can be adjusted in various ways.

[0115] In addition, the electronic device 800 can include an anti-reflection (AR) coating member to prevent a lens flare or ghosting phenomenon. As shown, the lens flare refers to a phenomenon in which reflected light reflected from the object 1010 is reflected by the light receiver 1020 and then reflected back to the lens unit 1040 to reach the light receiver 1020. Specifically, the flare phenomenon often occurs when the object 1010 is located relatively close to the electronic device 800 and the intensity of the reflected light reflected from the object 1010 is greater than a threshold value. When the lens flare occurs, the depth values of the depth image acquired from the electronic device can be distorted. Figure 10

[0116] To prevent the lens flare, an anti-reflection (AR) coating can be formed on the lens unit 1040. Accordingly, the reflected light reflected from the light receiver 1020 and reaching the lens unit 1040 can not be reflected by the lens unit 1040 and can not reach the light receiver 1020.

[0117] In addition, when distortion occurs in the acquired depth image due to the lens flare or the like, the electronic device 800 can correct the depth image. Specifically, the electronic device 800 can acquire a confidence image of the depth image and correct the depth image based on the acquired confidence image. Here, the confidence image refers to an image indicating the reliability of the depth value of each pixel of the depth image. Each pixel value of the confidence image can indicate a reliability score of the depth value of each pixel of the depth image. The electronic device 800 can acquire the confidence image of the depth image by [Mathematical Formula 4].

[0118] [Mathematical Formula 4]

[0119] ​[Confidence] = abs(I2 - I4) - abs(I1 - I3)

[0120] Here, I1 to I4 denote the first image data to the fourth image data I1, I2, I3, and I4, respectively.

[0121] The electronic device 800 can determine the reliability of the depth value of each pixel of the depth image based on each pixel value of the confidence image. If each pixel value of the confidence image is greater than a certain value, the electronic device 800 can determine that the depth value of the corresponding pixel is reliable. According to an embodiment, the certain value is pre-set. At this time, the electronic device 800 can not perform an operation of correcting the acquired depth image. On the other hand, if each pixel value of the confidence image is less than the pre-set value, the electronic device 800 can determine that the depth value of the corresponding pixel is not reliable. At this time, the electronic device 800 can acquire a corrected depth image based on the depth image and the confidence image. Specifically, the electronic device 800 can acquire the corrected depth image (d c (i, j)) by [Mathematical Formula 5].

[0122] [Mathematical Formula 5]

[0123]

[0124] Here, d(i, j) is the depth image acquired through the depth image acquisition module 873, c(i, j) is the confidence image corresponding to the acquired depth image, and P is (here, f is the frequency of the emitted light signal, and C L is the speed of light). In addition, α indicates the degree of correction, and C min is a constant. According to an embodiment, α is a user-defined variable indicating the degree of correction, and C min is a user-defined constant.

[0125] In this way, the electronic device 800 can improve the accuracy of the object distance information by performing an operation of correcting the depth image.

[0126] Figure 11 is a flowchart illustrating a method for controlling an electronic device according to an embodiment.

[0127] The electronic device 100 can emit light (S1110), and acquire reflected light reflected from an object (S1120). According to an embodiment, the electronic device 100 can emit light toward an area around the electronic device 100. The electronic device 100 can acquire a first depth image based on first reflected light acquired during a first time period, and store the first reflected light in a memory (S1130). In addition, the electronic device 100 can acquire a second depth image based on second reflected light acquired during a second time period which is a time period after the first time period (S1140). At this time, when it is determined that a depth value of a depth image for each time period is unreliable, the electronic device 100 can acquire a corrected depth image by using a confidence image corresponding to the depth image. In addition, for a detailed description of the method of acquiring a depth image, refer to Figure 6A and Figure 6B .

[0128] The electronic device 100 can acquire a distance from the electronic device to an object included in the second depth image by subtracting a depth value of each pixel of the first depth image from a depth value of each pixel of the second depth image, and can correct the acquired distance by using a compensation value acquired based on time information about the second time period (S1150). For a detailed description of such an operation, refer to Figure 5 .

[0129] Figure 12 is a block diagram illustrating a configuration of an electronic device according to an embodiment. The electronic device 1100 can be implemented as a robot device capable of acquiring distance information about an object, but is not limited thereto.

[0130] Referring to Figure 12 , the electronic device 1100 can include a light source 1110, a light receiver 1120, a lens unit 1140, a memory 1130, a communication interface 1150, a driver 1160, and a processor 1170. In addition, redundant descriptions of the light source 1110, the light receiver 1120, the lens unit 1140, the memory 1130, and the processor 1170 will be omitted.

[0131] The communication interface 1150 can include at least one circuit and can communicate with various types of external devices. For example, the communication interface 1150 can communicate with an external server or a user terminal. In addition, the communication interface 1150 can communicate with external devices according to various types of communication methods. The communication interface 1150 can perform data communication in a wireless or wired manner. When performing communication with external devices in a wireless communication manner, the communication interface 1150 can include at least one of a Wi-Fi communication module, a cellular communication module, a 3rd generation (3G) mobile communication module, a 4th generation (4G) mobile communication module, a fourth generation long term evolution (LTE) communication module, or a 5th generation (5G) mobile communication module. In addition, the communication interface 1150 according to an embodiment can be implemented as a wireless communication module, but this is only an embodiment, and the communication interface 1150 can be implemented as a wired communication module (e.g., LAN, etc.).

[0132] The driver 1160 can be a component for moving the electronic device 1100. Specifically, the driver 1160 can include an actuator for driving the electronic device 1100. In addition, in addition to the driver 1160, an actuator for driving the motion of other physical components (e.g., an arm, etc.) of the electronic device 1100 can be further included.

[0133] In addition, the various embodiments described above can be implemented in a computer or a computer-readable recording medium using software, hardware, or a combination of software and hardware. In some cases, the embodiments described in the specification can be implemented by a processor itself. According to software implementation, embodiments such as processes and functions described in the present disclosure can be implemented as separate software modules. Each software module can perform one or more functions and operations described in the present disclosure.

[0134] In addition, computer instructions for performing processing operations according to the various embodiments described above can be stored in a non-transitory computer-readable medium. The computer instructions stored in the non-transitory computer-readable medium can allow a specific device to perform processing operations according to the various embodiments described above when executed by a processor.

[0135] The non-transitory computer-readable medium is not a medium that stores data for a short time, such as a register, a cache, a memory, etc., but refers to a machine-readable medium that stores data semi-permanently. Specific examples of the non-transitory computer-readable medium can include a compact disc (CD), a digital versatile disc (DVD), a hard disk, a Blu-ray disc, a universal serial bus (USB), a memory card, a read-only memory (ROM), etc.

[0136] Although the embodiments have been shown and described above, the disclosure is not limited to the specific embodiments described above, but can be variously modified by those skilled in the art to which the disclosure belongs without departing from the spirit of the disclosure as disclosed in the appended claims. Such modifications should also be understood to fall within the scope and spirit of the disclosure.

Claims

1. An electronic device comprising: a light emitter configured to emit light toward an area around the electronic device as emission light; a light receiver configured to acquire reflected light from an object, wherein the object reflects light emitted from the light emitter; a memory storing one or more instructions; and a processor configured to execute the one or more instructions to: acquire a first depth image based on first reflected light acquired through the light receiver during a first time period, and store the first depth image in the memory, acquire a second depth image based on second reflected light acquired through the light receiver during a second time period after the first time period, acquire distance information between the electronic device and the object included in the second depth image by subtracting a first depth value of each pixel of the first depth image stored in the memory from a second depth value of each pixel of the second depth image, and correct the distance information by adding a compensation value acquired based on time information corresponding to the second time period to a depth value of a pixel corresponding to the object included in the second depth image, wherein the electronic device further comprises a lens unit disposed on an emission path of the emission light, wherein the lens unit is configured to adjust an emission range of the emission light, wherein the processor is further configured to execute the one or more instructions to control the lens unit to adjust the emission range of the emission light in synchronization with the first time period and the second time period.

2. The electronic device of claim 1, wherein, The processor is further configured to execute the one or more instructions to acquire the compensation value based on the time information corresponding to the second time period and a wavelength of the emission light.

3. The electronic device of claim 1, wherein, The processor is further configured to execute the one or more instructions to: acquire image data based on the emission light and the first reflected light, acquire phase information of the first reflected light based on the image data, and acquire the first depth image based on the phase information of the first reflected light.

4. The electronic device of claim 3, wherein, The light receiver comprises a plurality of sensors, wherein the plurality of sensors include a first sensor and a second sensor, and The processor is further configured to execute the one or more instructions to: control the light receiver such that the first sensor and the second sensor are activated with a preset time difference, acquire the image data by acquiring first image data and second image data based on the first reflected light acquired by each of the first sensor and the second sensor, and acquire the phase information of the first reflected light based on the first image data and the second image data.

5. The electronic device of claim 1, wherein, The light receiver comprises a first sensor unit and a second sensor unit, wherein each of the first sensor unit and the second sensor unit comprises a plurality of sensors, and The processor is further configured to execute the one or more instructions to control the light receiver such that the first sensor unit is activated in synchronization with the first time period, and the second sensor unit is activated in synchronization with the second time period.

6. The electronic device of claim 1, wherein, The lens unit comprises a non-reflective coating member configured to prevent at least a portion of the reflected light acquired by the light receiver from being reflected by the light receiver and then reflected by the lens unit to reach the light receiver again.

7. The electronic device of claim 1, wherein, The processor is further configured to execute the one or more instructions to perform operations of: obtaining a confidence image indicating reliability of a second depth value of each pixel of a second depth image, and updating the second depth image using the confidence image. 8.A method for controlling an electronic device, the method comprising: emitting light toward an area around the electronic device as emission light; obtaining reflected light from an object, wherein the object reflects light emitted from the light emitter; obtaining and storing a first depth image based on first reflected light obtained during a first time period; obtaining a second depth image based on second reflected light obtained during a second time period after the first time period; obtaining distance information between the electronic device and the object included in the second depth image by subtracting a first depth value of each pixel of the stored first depth image from a second depth value of each pixel of the second depth image, and correcting the distance information to obtain a distance by adding a compensation value obtained based on time information corresponding to the second time period to a depth value of a pixel corresponding to the object included in the second depth image, wherein the electronic device further comprises a lens unit disposed on an emission path of the emission light, wherein the lens unit is configured to adjust an emission range of the emission light, wherein the method comprises controlling the lens unit to adjust the emission range of the emission light in synchronization with the first time period and the second time period.

9. The method of claim 8, wherein, The step of correcting the distance information comprises obtaining the compensation value based on the time information corresponding to the second time period and a wavelength of the emission light.

10. The method of claim 8, wherein, The step of obtaining the first depth image comprises: obtaining image data based on the emission light and the first reflected light, obtaining phase information of the first reflected light based on the image data, and obtaining the first depth image based on the phase information about the first reflected light.

11. The method of claim 10, wherein, In obtaining the phase information, the phase information about the first reflected light is obtained based on a plurality of image data obtained through a plurality of sensors activated with a preset time difference.

12. The method of claim 8, wherein, The step of obtaining the first depth image comprises controlling a first sensor unit and a second sensor unit so that the first sensor unit and the second sensor unit each including a plurality of sensors are activated in synchronization with the first time period and the second time period, respectively. 13.An electronic device comprising: a memory storing one or more instructions; and a processor configured to execute the one or more instructions to perform operations of: obtaining first reflected light during a first time period; obtaining second reflected light during a second time period; obtaining a first depth image based on the first reflected light; obtaining a second depth image based on the second reflected light; obtaining distance information about a first object in the second depth image by subtracting a first depth value of each pixel of the first depth image from a second depth value of each pixel of the second depth image after performing correction on the second depth image; and correcting the distance information by adding a compensation value obtained based on time information corresponding to the second time period to a depth value of a pixel corresponding to the first object included in the second depth image, The electronic device further includes a lens unit disposed on a transmission path of the transmission light, wherein the lens unit is configured to adjust a transmission range of the transmission light. The processor is further configured to execute the one or more instructions to control the lens unit to adjust the transmission range of the transmission light synchronously with the first time period and the second time period.

14. The electronic device of claim 13, wherein, The first reflected light and the second reflected light are acquired by repeatedly transmitting the transmission light in an area around the electronic device.

Images