Electronic device and control method thereof

Abstract

An electronic device and a control method thereof are disclosed. The electronic device includes a sensor and a processor configured to obtain first information related to a distance from the sensor to an object and second information related to reliability for the first information, identify a plurality of cells not including distance information among a plurality of cells corresponding to a plurality of pixels constituting the sensor based on the first information, and determine reliability of the identified plurality of cells based on the second information, change a parameter setting of the sensor based on the determined reliability, obtain third information related to the distance from the object from the sensor whose parameter setting is changed, obtain fourth information including distance information of the plurality of cells based on the first information and the third information, and obtain distance information from the object based on the distance information included in the fourth information.

Description

Technical Field

[0001] This disclosure relates to an electronic device and a control method thereof. More specifically, this disclosure relates to an electronic device and a control method thereof capable of sensing the distance to an object. Background Technology

[0002] The development of electronic technology has led to the development of various electronic devices. In recent years, in particular, electronic devices such as autonomous vehicles that drive on behalf of humans, automated guided vehicles that sort and transport goods to their destinations, and robotic cleaning machines that move through the interior of a house while cleaning it have been developed.

[0003] To prevent collisions with objects during operation, such mobile electronic devices need to sense objects located around the device or the distance to those objects. Specifically, for this purpose, electronic devices with light-based sensors (e.g., image sensors or light detection and ranging (LiDAR) sensors) have been developed.

[0004] However, electronic devices equipped with light-based sensor technologies have the following problem: when the amount of light reflected by an object is insufficient, the distance to the object may not be detected. This imposes constraints on the movement of the electronic device and, specifically, can cause collisions between the electronic device and the object. Summary of the Invention

[0005] Technical issues

[0006] This disclosure relates to electronic devices and control methods thereof designed to improve the above-mentioned problems, and the purpose of this disclosure is to provide an electronic device and control method thereof capable of receiving light with a minimum or greater threshold required for measuring distance by changing the parameter settings of a sensor.

[0007] Technical solution

[0008] An electronic device according to an embodiment includes a sensor and a processor. The processor is configured to obtain first information related to the distance from the sensor to an object and second information related to the reliability of the first information; identify multiple units that do not include distance information among multiple units corresponding to multiple pixels constituting the sensor based on the first information; determine the reliability of the identified multiple units based on the second information; change the parameter settings of the sensor based on the determined reliability; obtain third information related to the distance to the object from the sensor whose parameter settings have been changed; and obtain fourth information including the distance information of the multiple units based on the first information and the third information; and obtain the distance information to the object based on the distance information included in the fourth information.

[0009] The processor can set a region of interest (ROI) that includes multiple identified units, and change the sensor's parameter settings based on the reliability of the multiple units included in the ROI.

[0010] The processor can determine the number of units belonging to each reliability range in the ROI by the reliability range, obtain parameter setting information corresponding to the reliability distribution of multiple units included in the ROI based on the reliability parameter setting information, and change the parameter settings of the sensor based on the parameter setting information.

[0011] The processor can change at least one of the parameter values ​​used to adjust the light intensity of the sensor or the parameter values ​​used to adjust the exposure time of the light, based on a determined reliability.

[0012] The reliability of the third information can be relatively higher than that of the first information, and the multiple units included in the ROI set based on the first information may include the distance information in the third information.

[0013] The processor can determine, in the third information, multiple units included in the ROI set based on the first information, obtain distance information of the multiple units included in the ROI, and obtain fourth information based on the first information and the obtained distance information.

[0014] The sensor can determine, based on second information, the units whose reliability is equal to or less than a threshold, and generate first information that is set as a preset value of the distance information of the unit, wherein the unit has a reliability less than or equal to the threshold.

[0015] The processor can identify multiple units included in a preset ROI among multiple units corresponding to multiple pixels constituting the sensor, determine the reliability of multiple units included in the ROI based on second information, and change the parameter settings of the sensor based on the determined reliability.

[0016] The processor can determine, based on second information, the number of units with a reliability value less than or equal to a preset threshold among multiple units corresponding to multiple pixels constituting the sensor. Based on the number of units with a reliability value less than or equal to the preset threshold being greater than or equal to a preset number, the processor changes the parameter settings of the sensor and obtains third information, including distance information, from the sensor whose parameter settings have been changed.

[0017] The processor can apply weights to the reliability of a determined set of units based on the locations of the identified units, and change the sensor's parameter settings based on the reliability of the units to which the weights have been applied.

[0018] The processor can identify, based on first information, at least one of a plurality of units including distance information whose reliability is less than or equal to a threshold, and obtain, based on the first information and third information, fourth information that the distance information of the identified at least one unit exceeds the threshold.

[0019] According to an embodiment, a control method for an electronic device includes: obtaining first information related to the distance from a sensor to an object and second information related to the reliability of the first information; identifying multiple units that do not include distance information among multiple units corresponding to multiple pixels constituting the sensor based on the first information, and determining the reliability of the identified multiple units based on the second information; changing the parameter settings of the sensor based on the determined reliability, and obtaining third information related to the distance to the object from the sensor whose parameter settings have been changed; and obtaining fourth information including the distance information of the multiple units based on the first information and the third information, and obtaining the distance information to the object based on the distance information included in the fourth information.

[0020] Changing parameter settings may include: setting a region of interest (ROI) that includes multiple identified units, and changing the sensor parameter settings based on the reliability of the multiple units included in the ROI.

[0021] Changing parameter settings may include: determining the number of units in the ROI belonging to each reliability range based on the reliability range, obtaining parameter setting information corresponding to the reliability distribution of multiple units included in the ROI based on the reliability parameter setting information, and changing the parameter settings of the sensor based on the parameter setting information.

[0022] Changing parameter settings may include: altering at least one of the parameter values ​​used to adjust the luminous intensity of the sensor or the parameter values ​​used to adjust the exposure time of light, based on a determined reliability.

[0023] The reliability of the third information can be relatively higher than that of the first information, and the multiple units included in the ROI set based on the first information include the distance information in the third information.

[0024] The fourth information can be obtained by identifying multiple units included in the ROI set based on the first information from the third information, obtaining the distance information of the multiple units included in the ROI, and obtaining the fourth information based on the first information and the obtained distance information.

[0025] The sensor can determine, based on second information, a plurality of units that have a reliability equal to or less than a threshold, and generate first information that is set as a preset value of the distance information of the unit, wherein the unit has a reliability less than or equal to the threshold.

[0026] By changing the parameter settings, multiple units included in a preset ROI can be identified among multiple units corresponding to multiple pixels constituting the sensor. The reliability of the multiple units included in the ROI is determined based on second information, and the parameter settings of the sensor are changed based on the determined reliability.

[0027] The third information can be obtained by determining the number of units with a reliability value less than or equal to a preset threshold in multiple units corresponding to multiple pixels constituting the sensor based on the second information, and by changing the parameter settings of the sensor based on the number of units with a reliability value less than or equal to the preset threshold being greater than or equal to a preset number, and obtaining the third information including distance information from the sensor whose parameter settings have been changed.

[0028] Changing the parameter settings can apply weights to the reliability of a number of identified units based on their positions, and the sensor parameter settings can be changed based on the reliability of the units to which the weights have been applied.

[0029] The fourth information can be obtained by identifying at least one unit among a plurality of units including distance information whose reliability is less than or equal to a threshold based on the first information, and by obtaining the fourth information based on the first information and the third information where the distance information of the identified at least one unit exceeds the threshold.

[0030] Invention Effects

[0031] According to various embodiments, an electronic device and a method for controlling it are provided that can sense the distance to an object having a dark color (such as black). Attached Figure Description

[0032] Figure 1 This is a block diagram illustrating an electronic device according to an embodiment of the present disclosure;

[0033] Figure 2a This is a diagram illustrating first information according to an embodiment of the present disclosure;

[0034] Figure 2b This is a diagram illustrating second information according to an embodiment of the present disclosure;

[0035] Figure 3 This is a diagram illustrating an example of setting a region of interest (ROI) based on at least one unit where no distance information is detected, according to an embodiment of the present disclosure;

[0036] Figure 4a A first reliability distribution according to an embodiment of the present disclosure is shown;

[0037] Figure 4b This is a diagram illustrating a second reliability distribution according to an embodiment of the present disclosure;

[0038] Figure 4c This is a diagram illustrating reliability parameter setting information according to an embodiment of the present disclosure;

[0039] Figure 5 This is a diagram illustrating third information according to an embodiment of the present disclosure;

[0040] Figure 6a and Figure 6b This is a diagram illustrating fourth information according to an embodiment of the present disclosure;

[0041] Figure 7a This is a diagram illustrating an embodiment of changing sensor parameter settings based on weights according to an embodiment of the present disclosure;

[0042] Figure 7b This is a diagram illustrating an embodiment of changing sensor parameter settings based on weights according to an embodiment of the present disclosure;

[0043] Figure 8a This is a diagram illustrating a predetermined ROI according to an embodiment of the present disclosure;

[0044] Figure 8b This is a diagram illustrating an embodiment of changing sensor parameter settings based on the reliability of all pixels constituting the sensor, according to an embodiment of the present disclosure;

[0045] Figure 9a This is a diagram illustrating first information including distance information according to an embodiment of the present disclosure;

[0046] Figure 9b This is a diagram illustrating second information including reliability information according to an embodiment of the present disclosure;

[0047] Figure 9c This is a diagram illustrating third information including distance information according to an embodiment of the present disclosure;

[0048] Figure 9d This is a diagram illustrating fourth information, including reliability information, according to an embodiment of the present disclosure;

[0049] Figure 9e This is a diagram illustrating the fifth piece of information according to an embodiment of the present disclosure;

[0050] Figure 10a This is a detailed block diagram illustrating an electronic device according to an embodiment of the present disclosure;

[0051] Figure 10b This is a detailed block diagram illustrating an electronic device according to an embodiment of the present disclosure;

[0052] Figure 11 This is a flowchart illustrating a method for controlling an electronic device according to embodiments of the present disclosure; and

[0053] Figure 12 This is a flowchart illustrating a method for controlling an electronic device according to an embodiment of the present disclosure. Detailed Implementation

[0054] The terminology used in this specification and claims has been chosen in a general sense with respect to the function of this disclosure. However, these terms may vary according to the expectations of a person skilled in the art, legal interpretation, or the emergence of new technologies, as is known in the art. Furthermore, some terms may be used optionally by the applicant. These terms may be interpreted in the sense defined herein, and may be interpreted based on the overall content of this disclosure and by a person skilled in the art in the absence of specific definitions of the terms.

[0055] When it is determined that a detailed description of known techniques related to this disclosure may unnecessarily obscure the spirit of this disclosure, the detailed description of known techniques may be shortened or omitted.

[0056] Various exemplary embodiments will now be described in more detail with reference to the accompanying drawings; however, it should be understood that this disclosure is not limited to the various exemplary embodiments described herein.

[0057] In the following description, embodiments will be described in more detail with reference to the accompanying drawings.

[0058] Figure 1 This is a block diagram illustrating an electronic device according to an embodiment of the present disclosure.

[0059] An electronic device 100 according to an embodiment of the present disclosure may include a sensor 110, a processor 120, and a memory 130.

[0060] Sensor 110 is configured to output information about distance, and refers to... Figure 1 According to one embodiment of this disclosure, the sensor 110 may be a time-of-flight (ToF) sensor including a light emitter 10, a light receiver 20, and a microcontroller unit (MCU) 30. The ToF sensor 110 measures the distance to an object by measuring the time it takes for light emitted by the light emitter 10 to be reflected by the object and then received by the light receiver 20. As an example, the light emitter 10 may be an IR light source that emits infrared light modulated by a signal of a specific frequency, and the light receiver 20 may be an image sensor that receives light reflected from the object. However, the light emitter 10 of this disclosure may be implemented as various light sources capable of emitting light (such as ultrasound or laser), and the light receiver 20 may be implemented as various light-receiving sensors capable of receiving light (such as an RGB sensor).

[0061] The MCU 30 of sensor 110 can control the light emitter 10 to emit light based on control signals received from processor 120. The MCU 30 can output information about the distance between sensor 110 and the object based on the time it takes for light to be emitted by the light emitter 10, reflected by the object, and received by the light receiver 20. The MCU 30 can determine the distance between sensor 110 and the object by calculations based on the speed of light c and the time t it takes for light to be emitted by the light emitter 10, reflected by the object, and received at the light receiver 20.

[0062] According to an embodiment, the MCU 30 can detect the phase change relative to the time it takes for light emitted by the light emitter 10 to be reflected by the object and received by the light receiver 20, thereby outputting information about the distance between the sensor 110 and the object. The MCU 30 can determine the distance between the sensor 110 and the object based on the speed of light c, modulation frequency f, phase period n, and phase θ, by calculating distance = c / (2f)*(n+θ / 2(pi)). As described above, the distance between the sensor 110 and the object can be determined using the phase change; however, in the following, it is assumed that the MCU 30 determines the distance between the sensor 110 and the object based on the time t it takes for light emitted by the light emitter 10 to be reflected by the object and received by the light receiver 20.

[0063] The information output by sensor 110 regarding the distance between sensor 110 and the object is referred to as first information. According to an embodiment, the first information may be referred to as distance information, depth information, or a distance map or depth map. See below for further details. Figure 2a An example describing the first piece of information.

[0064] Reference Figure 1 Although MCU 30 is included Figure 1 In the sensor 110, the light emitter 10 may include a light emitter 10 and a light receiver 20, and the functions of the MCU 30 may be executed by the processor 120. In this example, the light emitter 10 may illuminate light based on a control signal received from the processor 120, and the processor 120 may determine the distance between the sensor 110 and the object based on the time it takes for the light emitted by the light emitter 10 to be reflected by the object and then received by the light receiver 20.

[0065] Electronic device 100 may include memory 130.

[0066] As described below, under the control of MCU 30 and processor 130, memory 130 can store first information, second information, and reliability parameter setting information output by sensor 110 to determine the distance between sensor 110 and the object. Details will be described later.

[0067] Figure 2aThis is a diagram illustrating first information according to an embodiment of the present disclosure.

[0068] As described above, the MCU 30 of sensor 110 can determine the distance between sensor 110 and object based on the time it takes for light emitted by light emitter 10 to be reflected by object and received by light receiver 20.

[0069] The MCU 30 can determine the distance to the object reflecting the light for each of the plurality of pixels constituting the light receiver 20. Here, each of the plurality of pixels constituting the light receiver 20 according to the embodiment may include an in-phase receiver and an out-of-phase receiver.

[0070] For example, when the light transmitter 10 emits light, the in-phase receiver is activated and the light is received, and when the light transmitter 10 does not emit light (e.g., when the light transmitter 10 is turned off), the out-of-phase receiver can be activated to receive light.

[0071] As an example, if the distance between sensor 110 and the object is 0, only the in-phase receiver can receive the light emitted by light emitter 10 and reflected by the object, while the out-of-phase receiver may not receive the light. As another example, if the distance between sensor 110 and the object is not 0, the light emitted by light emitter 10 takes a predetermined time to reach light receiver 20 after being reflected by the object, such that a portion of the light emitted by light emitter 10 and reflected by the object can be received by the in-phase receiver, while the remainder can be received by the out-of-phase receiver.

[0072] The MCU 30 can identify the distance to an object based on the time difference between the time the in-phase receiver receives light and the time difference between the time the out-of-phase receiver receives light.

[0073] like Figure 2a As shown, when light emitted by light emitter 10 is directed from the object (reference) Figure 2a When an object (such as a mouse 1, a person's arm 2, a wall surface 3, etc.) reflects light and is received at each of the multiple pixels, the timing of light reception for each of the multiple pixels can be determined.

[0074] The plurality of pixels constituting the light receiver 20 may include a timer for measuring the time of light reception and a time-to-digital (TDC) converter for converting the measured time into a digital signal. The MCU 30 may determine the time when light is received for each of the plurality of pixels based on the signal output from the TDC converter. The MCU 30 may determine the distance to an object for each of the plurality of pixels by calculations based on a time delay and the speed of light travel, and may output first information 210 including information about the determined distance, wherein the time delay is the time when light is received at each of the plurality of pixels after being illuminated by the light emitter 10. For example, the MCU 30 may identify the time when the in-phase and out-of-phase receivers constituting the pixel receive light and identify the distance to the object based on the phase difference between these two times. However, it should be understood that this is not limited to this example.

[0075] In another example, each of the in-phase and out-of-phase receivers can receive multiple beams of light and identify the distance to an object based on the phase difference of the time of each of the received beams. For example, light emitter 10 can sequentially emit multiple beams of light at predetermined time intervals (e.g., 0.1 s intervals). Each of the in-phase and out-of-phase receivers can receive multiple beams of light that are illuminated at predetermined time intervals, and MCU 30 can identify the time at which each of the in-phase and out-of-phase receivers receives multiple beams of light and can identify the distance to an object based on the phase difference of the multiple identified times. It should be understood that the specific time intervals in the above examples are for ease of description and are not limiting.

[0076] The MCU 30 can identify the distance from each of the multiple pixels constituting the light receiver 20 to the object, and stores first information 210, including information about the identified distance, in the memory 130.

[0077] like Figure 2a As shown, the first information 210 may include distance information between an object identified in each of the plurality of pixels constituting the light receiver 20 and each of the plurality of pixels. First, constituting... Figure 2a Each of the multiple units of the first information 210 shown corresponds to a pixel, and the numbers included in the multiple units are examples of distance information representing the distance from the pixel to the object as identified by the MCU 30 through the above-described calculation method, and the unit may be cm, but is not limited thereto.

[0078] Reference Figure 2a The objects may include a mouse (1), a person's arm (2), a wall surface (3), etc. The object corresponding to each of the multiple pixels constituting the light receiver 30 may be the same as or different from the object.

[0079] For example, the object corresponding to the first pixel among the plurality of pixels constituting the light receiver 20 could be the mouse 1, and the first pixel could receive light reflected by the mouse 1. In this example, the distance information of the first pixel recognized by the MCU 30 could represent the distance between the sensor 110 and the mouse 1.

[0080] As another example, the object corresponding to the second pixel among the plurality of pixels constituting the light receiver 20 is a human arm 2, and the second pixel can receive light reflected by the human arm 2. In this example, the distance information of the second pixel recognized by the MCU 30 can represent the distance between the sensor 110 and the human arm 2.

[0081] Reference Figure 2a The first information 210, each of the multiple pixels can receive light reflected from different objects (e.g., a mouse 1, a human arm 2, and a wall surface 3), and even if each of the multiple pixels receives light reflected from the same object (e.g., the upper or lower end of the wall surface 3), the distance information corresponding to the multiple units can be different.

[0082] As described above, each of the multiple units may correspond to a plurality of pixels constituting the light receiver 20, and the numbers of the multiple units may indicate the distance from the corresponding pixel to the object identified in the corresponding pixel.

[0083] Reference Figure 2a The first information 210 differs from the distance information corresponding to the multiple pixels. This is because if the distance between the sensor 110 and the object is different, the time it takes for the light emitted by the light emitter 10 to be received in each of the multiple pixels (i.e., the time of flight of the light) is different. For example, refer to Figure 2a The distance information of the unit corresponding to the pixel that receives light reflected by an object located relatively close to the sensor 110 (e.g., a person's arm, etc.) may include information about a distance that is relatively shorter than the distance information of the unit corresponding to the pixel that receives light reflected by an object located relatively far from the sensor 110 (e.g., a wall surface, etc.).

[0084] Reference Figure 2a The first piece of information 210, the distance information corresponding to some of the multiple units is 0.

[0085] As described above, since sensor 110 measures the distance to an object based on light reflected from the object, if the amount of light is less than or equal to a threshold, sensor 110 may not be able to measure the distance to the object when the amount of light reflected from the object and received at light receiver 20 is less than or equal to the threshold. For example, in objects with dark colors (such as black) (e.g., Figure 2aIn the case of the mouse 1 part, most of the light illuminating the object is absorbed on the surface of the object, and the amount of light reflected from the object and received at the light receiver 20 may be less than or equal to the threshold.

[0086] In this example, sensor 110 can output distance information of 0 for the cell corresponding to the pixel receiving light with an intensity less than or equal to a threshold. For example, as Figure 2a As shown, the distance information of the cell corresponding to the pixel that receives light reflected from object 1-1 (e.g., a portion of mouse 1) can be 0. The output value 0 is merely exemplary, and according to an embodiment, sensor 110 may output the distance information of the cell corresponding to the pixel that receives light with an intensity less than or equal to a threshold as another specific value (such as infinity).

[0087] If the distance information is 0, the distance between sensor 110 and the object may be 0, and due to insufficient light, the distance to the object may not be accurately measured. Therefore, in this disclosure, the accuracy of the distance information (i.e., the value is 0) of a unit having an output value of 0 can be determined based on the reliability information (hereinafter referred to as the second information) for each of the plurality of units. Referring below... Figure 2b .

[0088] Figure 2b This is a diagram illustrating second information according to an embodiment of the present disclosure.

[0089] Sensor 110 can output second information, which includes reliability information used for first information 210.

[0090] Specifically, the MCU 30 of sensor 110 can output second information 220 indicating the reliability of the distance information corresponding to each cell constituting the first information 210. Reliability can be a value representing the accuracy of the distance information corresponding to each cell constituting the first information 210. The MCU 30 can determine the reliability of the distance information measured for each pixel constituting the light receiver 20 based on multiple amounts of charge measured when the light receiver 20 receives multiple lights with different phase shifts. For example, when multiple lights with four phase shifts (e.g., 0 degrees, 90 degrees, 180 degrees, and 270 degrees) are sequentially illuminated by the light emitter 10 according to the control of the MCU 30, the light receiver 20 can sequentially receive the multiple lights with four phase shifts.

[0091] The MCU 30 can determine the reliability of the distance information corresponding to each of the plurality of pixels by performing a square root operation of confidence level = [(Q1-Q2)^2+(Q3-Q4)^2]. For example, the MCU 30 can perform the square root operation of [(Q1-Q2)^2+(Q3-Q4)^2] based on the charge amount Q1 measured when the first pixel of the plurality of pixels constituting the light receiver 20 receives light with a first phase shift, the charge amount Q2 measured when the first pixel receives light with a second phase shift, the charge amount Q3 measured when the first pixel receives light with a third phase shift, and the charge amount Q4 measured when the first pixel receives light with a fourth phase shift, and can obtain the confidence level for the distance information included in the cell corresponding to the first pixel. The MCU 30 can use the above method (operation) to obtain the reliability of the distance information corresponding to each of the plurality of pixels.

[0092] This is merely an example, and the reliability of the first information 210 can be determined by various methods, such as using a neural network model trained to output the reliability corresponding to each pixel by using the difference in distance information between pixels adjacent to a particular pixel or distance information measured for each pixel as input data. As another example, the light emitter 10 may sequentially illuminate multiple lights with fewer than four phase shifts (e.g., 0 degrees, 180 degrees) or more than four phase shifts (e.g., 0 degrees, 45 degrees, 90 degrees, 135 degrees, 180 degrees, 225 degrees, 270 degrees, 315 degrees, 360 degrees, etc.), and the light receiver 20 may sequentially receive multiple lights with different phases. Subsequently, the MCU 30 may measure the reliability of the corresponding pixel based on the amount of charge measured in the pixel according to the multiple received lights; thus, the MCU 30 can obtain the reliability of the distance information corresponding to each of the multiple pixels.

[0093] As described above, the second information 220 may include confidence information for the first information 210, and may be referred to as confidence information or confidence graph according to embodiments.

[0094] For example, such as Figure 2b As shown, when light shines towards an object, the multiple pixels constituting the light receiver 20 can receive the light reflected by the object, and the MCU 30 can output second information 220 including a reliability value for each unit corresponding to each pixel through the above-described calculation method. Then, the MCU 30 can store the second information 220 in the memory 130.

[0095] like Figure 2bAs shown, the second information 220 may include a unit corresponding to each of the plurality of pixels constituting the optical receiver 20 and reliability information corresponding to the respective unit. Here, the reliability information includes information including a reliability value calculated by the above calculation method, and as an example, the reliability value may have a value from 0 to 255, but is not limited thereto. A unit with a relatively high reliability value can be considered to include distance information with higher precision than a unit with a relatively low reliability value. It should be understood that, due to the... Figure 2b The shape of the second information 220, which consists of units corresponding to each of the multiple pixels shown, is an example and is not limited thereto.

[0096] Reference is made to embodiments according to this disclosure. Figure 2b The second information 220, compared to the reliability values ​​of units corresponding to other pixels, has a relatively small reliability value (e.g., a reliability value of 50 or less) for units corresponding to pixels receiving light with an intensity below a threshold. Figure 2a As shown, a pixel that receives light with a light intensity below or equal to a threshold can be a pixel that receives light reflected from an object with a dark color (such as black).

[0097] In this example, the MCU 30 can determine, based on the second information 220, which units among the plurality of units have a reliability value less than or equal to a threshold, and set the distance information of the units among the plurality of units including in the first information 210 that have a reliability value less than or equal to the threshold to 0, thereby outputting as shown in the example. Figure 2a The first piece of information shown is 210.

[0098] As described above, since sensor 110 measures the distance to an object based on light reflected from the object, it is difficult to measure the distance when the amount of light reflected from the object and received by light receiver 20 is insufficient. For example, when light shines on a black object, most of the light is absorbed onto the object's surface, potentially resulting in insufficient light received by light receiver 20.

[0099] Because the amount of light received by the light receiver 20 is insufficient, the sensor 110 may be unable to measure the distance to the black object.

[0100] This could have particular limitations on the movement of electronic devices that operate based on distance from objects (e.g., mobile robots or autonomous vehicles), and could potentially lead to collisions between electronic devices and objects.

[0101] To address this problem, the processor 120 of this disclosure can change the parameter settings of the sensor 110 when at least one unit of distance information is not detected being included in the first information 210, and additionally obtain third information including distance information from the sensor 110 whose parameter settings have been changed. Changing the parameter settings may mean changing the parameter value set in at least one of a plurality of parameters. As an example, the parameter may be a parameter for adjusting the light intensity of the light emitter 10 or a parameter for adjusting the exposure time relative to the light. However, this is merely exemplary, and the parameters of the sensor 110 may be various parameters (such as parameters for changing the modulation frequency of the light), and the technical concept of this disclosure is not limited to the embodiments described herein. This will be referred to Figure 3 It is described in detail.

[0102] Figure 3 This is a diagram illustrating an example of setting a region of interest (ROI) based on at least one unit that has not detected distance information, according to an embodiment of the present disclosure.

[0103] Processor 120 can control sensor 110 to emit light through light emitter 10. As light receiver 20 receives light as reflected from an object, processor 120 can receive first information 210 generated by MCU 30 from sensor 110. As described above, first information 210 may include distance information.

[0104] As described above, the first information 210 can be generated by the processor 120. The processor 120 can control the light emitter 10 of the sensor 110 to emit light, and can determine (or calculate) the distance between the sensor 110 and the object based on the time of flight of the light emitted by the light emitter 10 when it is reflected by the object and received by the light receiver 20.

[0105] The processor 120 can receive second information 220 from the sensor 110, which includes reliability information for distance information used in the first information 210.

[0106] As described above, sensor 110 can output second information 220 indicating the reliability of distance information corresponding to each pixel constituting light receiver 20. Reliability can be a value representing the accuracy of the distance information corresponding to each pixel constituting light receiver 20.

[0107] The second information 220 can be generated by the processor 120. The processor 120 can control the light emitter 10 of the sensor 110 to emit light, and use the charge Q and phase shift measured based on the amount of light received by the light receiver 20 when the light emitted by the light emitter 10 is reflected by the object and received by the light receiver 20, to determine the reliability of the distance information corresponding to each pixel constituting the light receiver 20. For example, if the light receiver 20 receives light with four different phase shifts, the processor 120 can determine the reliability of the distance information corresponding to each pixel by performing a square root operation of confidence = [(Q1-Q2)^2+(Q3-Q4)^2].

[0108] As described above, when the amount of light reflected by the object and received by the light receiver 20 is insufficient, that is, when the amount of light received by the light receiver 20 is less than or equal to the minimum threshold required to measure the distance, the sensor 110 may output a value of 0 as distance information for the unit corresponding to the pixel that receives the light reflected from the corresponding object.

[0109] When first information 210, including distance information, is received from sensor 110, processor 120 can identify, based on the distance information, at least one unit among multiple units corresponding to multiple pixels constituting light receiver 20 that has an output value of 0 indicating the absence of distance information. The identified at least one unit may be a unit corresponding to a pixel that has received light amounts below or equal to a minimum threshold required for distance measurement. Processor 120 can set a region of interest (ROI) including the identified at least one unit to the first information 210 and the second information 220. For example, the ROI may be a square region with a minimum size including at least one unit having an output value of 0. The ROI may have various shapes (such as circles, closed curves, and polygons including at least one unit). For example, as... Figure 3 As shown, the processor 120 can identify at least one pixel having an output value of 0 indicating the absence of distance information based on first information 210 or second information 220 of a plurality of pixels constituting the light receiver 20, and set an ROI 1000 including a unit corresponding to the identified pixel.

[0110] The processor 120 can change the parameter settings of the sensor 110 based on the reliability information corresponding to the ROI 1000.

[0111] When ROI 1000 is set, processor 120 can determine the reliability of at least one unit constituting ROI 1000 based on reliability information included in second information 220.

[0112] The processor 120 can change the parameter settings of the sensor 110 based on the reliability of at least one unit constituting the ROI 1000.

[0113] First, processor 120 can identify the reliability distribution (or histogram) of the multiple units constituting ROI 1000 to change the parameter settings of sensor 110. Then, processor 120 can obtain parameters for changing the operation and settings of sensor 110 based on the identified reliability distribution and reliability parameter setting information pre-stored in memory 130.

[0114] For example, the parameters of sensor 110 may include parameters for adjusting the light intensity of light emitter 10 or parameters for adjusting the exposure time of light. However, this is merely exemplary, and the parameters of sensor 110 may be various parameters (such as parameters for changing the modulation frequency of light), and the technical concept of this disclosure is not limited to the embodiments described herein.

[0115] Processor 120 can determine the reliability of each of the multiple units included in ROI 1000. Processor 120 can determine the number of units belonging to each reliability range.

[0116] For example, refer to Figure 4a and Figure 4b The processor 120 can determine, based on second information 220, the number of units with a reliability between 0 and 25, the number of units with a reliability between 26 and 50, ..., the number of units with a reliability between 226 and 255, in the ROI identified based on the first information 210. According to an embodiment, the reliability range of 25 units can be set differently. Figure 4a and Figure 4b An embodiment is shown that determines the number of units belonging to the corresponding reliability range for each reliability range, and when based on such... Figure 3 This technical concept will be applied when determining the number of units for the reliability range in the ROI 1000 shown.

[0117] Then, the processor 120 can obtain the parameters of the sensor 110 corresponding to the ROI 1000 based on the reliability distribution of the multiple units constituting the ROI 1000 and the reliability parameter setting information previously stored in the memory 130. The processor 130 can then change the parameter settings of the sensor 110 to correspond to the obtained parameters.

[0118] Figure 4c An example of reliability parameter setting information pre-stored in memory 130 is shown. For example, the reliability parameter setting information may include parameter setting information for changing the parameter settings of sensor 110 based on the number of units having a confidence value lower than a minimum threshold for distance measurement. For example, see reference... Figure 4cIf units with confidence values ​​lower than the minimum threshold for distance measurement are included in the ROI in a first number or more (e.g., 51 or more), parameter setting information for increasing the light intensity of the light emitter 10 by a1 times, parameter setting information for increasing the light exposure time by b1 times, or parameter setting information for increasing the light intensity of the light emitter 10 by a11 times and parameter setting information for increasing the light exposure time by b11 times can be stored.

[0119] If the number of cells with confidence values ​​lower than the minimum threshold for distance measurement is greater than or equal to a second plurality (e.g., 31) and less than a first plurality (e.g., 51), then reliability parameter setting information can be stored, which includes parameter setting information for increasing the light emission intensity of the light emitter 10 by a2 times, parameter setting information for increasing the light exposure time by b2 times, or parameter setting information for increasing the light emission intensity of the light emitter 10 by a22 times and increasing the light exposure time by b22 times.

[0120] If the number of units in the ROI with reliability values ​​below the minimum threshold for distance measurement is greater than or equal to a third (e.g., 11) and less than a second (e.g., 31), then the reliability parameter setting information can store parameter setting information for increasing the light intensity of the light emitter 10 by a3 times, parameter setting information for increasing the light exposure time by b3 times, or parameter setting information for increasing the light intensity of the light emitter 10 by a33 times and increasing the exposure time relative to the light by b33 times. When the number of units in the ROI with reliability values ​​below the minimum threshold for distance measurement is less than a third (e.g., 11), reliability parameter setting information with parameter setting information for maintaining the parameter values ​​of the light emitter 10 can be stored.

[0121] Therefore, the processor 120 can change the parameter settings of the sensor 110 based on the reliability parameter setting information and the reliability distribution of the multiple units constituting the ROI 1000.

[0122] In the reliability parameter settings, the number of units with confidence values ​​below the minimum threshold included in the ROI, the parameters for increasing intensity, and the parameters for increasing light exposure time can be proportional. For example, when the number of units with confidence values ​​below the minimum threshold included in the ROI is greater than or equal to the third (e.g., 51) number (e.g., 11) number, the parameters for increasing light intensity and the parameters for increasing light exposure time can be increased proportionally to the number of units.

[0123] For example, such as Figure 4bAs shown, when units with confidence values ​​less than or equal to the minimum threshold for distance measurement (e.g., reliability 50) are included in the ROI in a first plurality (e.g., at least 51), the processor 120 may change the parameter settings of the sensor 110 to increase the light intensity of the light emitter by a1 times (e.g., 3.5 times), increase the light exposure time by b1 times (e.g., 3.5 times), increase the light exposure time by a11 times (e.g., 2.3 times), or increase the light exposure time by b11 times (e.g., 1.5 times).

[0124] like Figure 4b As shown, when the number of units with confidence values ​​less than or equal to the minimum threshold for distance measurement (e.g., reliability 50) is greater than or equal to a second plurality (e.g., at least 31) and less than a first plurality (e.g., less than 50), the processor 120 may change the parameter settings of the sensor 110 to increase the light intensity of the light emitter 10 by a2 times (e.g., twice), increase the light exposure time by b2 times (e.g., twice), increase the light intensity of the light emitter 10 by a22 times (e.g., 1.3 times), and increase the light exposure time by b22 times (e.g., 1.5 times) based on the reliability parameter setting information.

[0125] Processor 120 can obtain third information, including information about the distance between sensor 110 and an object, from sensor 110 whose parameter settings have been modified. Processor 120 can modify the parameter settings of sensor 110 according to reliability parameter setting information and send a control signal for generating the third information including distance information to sensor 110 whose parameter settings have been modified. In this example, MCU 30 of sensor 110 can control light emitter 10 to emit light according to the control signal and output the third information including information about the distance to the object based on the time it takes for the light to be received by light receiver 20 after being reflected by the object.

[0126] The third information 510 may include information about the distance to the object sensed by the sensor 110, which increases the light intensity of the light emitter 10 or increases the light exposure time. The distance information included in the ROI 1000 of the third information 510 may have relatively high reliability compared to the distance information included in the ROI 1000 of the first information 210. This is because the light receiver 20 can receive light greater than the minimum threshold required for distance measurement by increasing the light intensity of the light emitter 10, and the light receiver 20 can receive light greater than the minimum threshold required for distance measurement by receiving light for a relatively long time relative to the increase in light exposure time.

[0127] For example, if there are multiple ROIs including multiple units with confidence values ​​below the minimum threshold for distance measurement, the processor 120 can increase the accuracy of driving path recognition or increase the light exposure time to reduce the probability of collision with the object, even if the time required to measure the distance is increased.

[0128] For example, if the time for light emission by light emitter 10 to emit light is increased, the activation time of the in-phase receiver and the out-of-phase receiver included in light receiver 20 can be increased. The in-phase receiver is activated during the time period when light emitter 10 emits light, and the out-of-phase receiver is activated during the time when light emitter 10 does not emit light. In this example, as the activation time of the out-of-phase receiver included in the pixel increases, light with a minimum threshold value or greater than that required for distance measurement can be received, and MCU 30 can obtain distance information corresponding to the respective pixel with a reliability value greater than or equal to the threshold.

[0129] For example, if there are multiple ROIs including multiple units with confidence values ​​lower than the minimum threshold for distance measurement, the processor 120 can increase the light exposure time to improve the accuracy of the electronic device 100's path recognition or reduce the probability of collision with an object, even if the time required to measure the distance increases.

[0130] As another example, if there are multiple ROIs including multiple units with confidence values ​​lower than the minimum threshold for distance measurement, it is possible to increase the luminous intensity of the light emitted by the light emitter 10 to obtain highly reliable distance information, even though it may seem impractical considering the hardware specifications of the IR light source set in the light emitter 10 (e.g., the possibility of failure). This improves the accuracy of the travel path recognition of the electronic device 100 or reduces the probability of collision with an object.

[0131] For example, referencing Figure 5 The third information 510 output by the sensor 110 whose parameters have been changed is different from the first information output by the sensor 110 before the parameters were changed. The multiple units included in the ROI 1000 include distance information.

[0132] When the sensor 110, whose parameter settings have been changed, receives third information 510, such as Figure 6b As shown, the processor 120 can obtain the fourth information 610, which includes the distance information of multiple units, based on the distance information of multiple units included in the ROI 1000 of the third information 510 and the distance information of multiple units included in the ROI 1000 of the first information 210.

[0133] When processor 120 receives third information 510 from sensor 110, processor 120 can determine, within the third information 510, multiple units included in the ROI set based on first information 210. Processor 120 can then modify the distance information of the multiple units included in the ROI 1000 of first information 210 based on the distance information of these multiple units included in the ROI 1000 of third information 510. Therefore, as... Figure 6b As shown, the electronic device 100 can obtain the modified fourth information 610 based on the distance information of multiple units. In this distance information of multiple units, the information of the multiple units included in the ROI of the first information 210 is included in the ROI of the third information 510, such as... Figure 6b As shown. Optionally, the processor 120 can generate (or obtain) a fourth information 610 including distance information of multiple units by aggregating multiple distance information corresponding to multiple units obtained from the first information 210 and multiple distance information corresponding to multiple units obtained from the third information 510. The processor 120 can generate the fourth information 610 including distance information of multiple units by aggregating the ROI 1000 of the third information 510 to the ROI 1000 of the first information 210, and by aggregating the entire region of the first information 210 and the entire region of the third information 510.

[0134] The processor 120 can perform processing (such as driving, stopping, or controlling the driving direction of the electronic device 100) based on the distance information included in the fourth information 610. As described above, since the electronic device 100 of this disclosure performs processing (such as driving) based on the fourth information 610, problems that may occur when electronic devices of the related art fail to detect the distance to a dark (such as black) object (e.g., collision with a black object, etc.) can be prevented.

[0135] According to an embodiment, this disclosure allows for changing the parameter settings of sensor 110 based on the reliability values ​​of multiple units and the weights of multiple units.

[0136] The processor 120 may apply a first weight to the reliability value of the unit located at the lower end of the plurality of units included in the ROI 1000, and apply a second weight to the reliability value of the unit located at the upper end. Here, the first weight may be higher than the second weight. This is because, compared to the unit located at the upper end, the unit located at the lower end of the plurality of units is the unit used to sense objects approaching the electronic device 100, and therefore may need to be given a higher weight to accurately measure the distance to the object.

[0137] For example, refer to Figure 7aThe processor 120 can apply a first weight to the reliability value of the unit located in the fourth row (4) among the multiple units included in the ROI 1000, a second weight to the reliability value of the unit located in the fifth row (5), and a third weight to the reliability value of the unit located in the sixth row (6). Similarly, the processor 120 can apply a fourth weight to the reliability value of the unit located in the seventh row (7) among the multiple units included in the ROI 1000, a fifth weight to the reliability value of the unit located in the eighth row (8), and a sixth weight to the reliability value of the unit located in the ninth row (9). The application of weights can be an operation of multiplying the reliability value of a unit by the weight. If the weight is w and the reliability value is k, then the reliability value with the weight applied can be the value w*k.

[0138] Processor 120 can sum the values ​​applied to each unit included in ROI 1000 and obtain a computed value by dividing the sum by the number n of multiple units included in ROI 1000. For example, when four pixels include a pixel with weight w1 and confidence value k1, a pixel with weight w1 and reliability value k2, a pixel with weight w2 and reliability value k3, and a pixel with weight w2 and reliability value k4, processor 120 can calculate (w1*k1+w1*k2+w2*k3+w2*k4) / 4.

[0139] The processor 120 can change the parameter settings of the sensor 110 based on the calculated values ​​obtained through the above calculations. For example, when a first calculated value is obtained, the processor 120 can change the parameter settings of the sensor 110 to increase the light emission intensity of the light emitter 10 by a factor of x1 or increase the light exposure time by a factor of y1. And when a second calculated value is obtained, the processor 120 can change the parameter settings of the sensor 110 to increase the light emission intensity of the light emitter 10 by a factor of x2 or increase the light exposure time by a factor of y2. Multiple parameter setting information corresponding to multiple calculated values ​​can be stored in the memory (not shown) of the electronic device 100. For example, such as... Figure 7bAs shown, the memory (not shown) can store parameter setting information, which is used to: maintain the parameter setting value of sensor 110 when the calculated value obtained by the above calculation is greater than or equal to 0 and less than the first value; and increase the light emission intensity of light emitter 10 by x1 times and increase the exposure time of light emitter 10 by y1 times when the calculated value is greater than or equal to the first value and less than the second value. For example, a memory (not shown) may store parameter setting information for increasing the luminous intensity of the light emitter 10 by x2 times and the exposure time of the light emitter 10 by y2 times, or increasing the luminous intensity of the light emitter 10 by x22 times and the exposure time by y22 times, when the calculated value is greater than or equal to a second value and less than a third value. It may also store parameter setting information for increasing the luminous intensity of the light emitter 10 by x3 times and the exposure time by y3 times, or increasing the luminous intensity of the light emitter 10 by x33 times and the exposure time by y33 times, when the calculated value is greater than or equal to a third value.

[0140] Although the ROI is set based on multiple units that do not include distance information, the ROI can also be set in advance according to the embodiments.

[0141] For example, refer to Figure 8a ROI 2000 can be configured to include a region of multiple cells in the lower center area of ​​multiple cells.

[0142] In this example, processor 120 can identify multiple units included in a predetermined ROI 2000 among multiple units, and determine the reliability of the multiple units included in ROI 2000 based on second information. Processor 120 can determine the reliability distribution (or histogram) of the multiple units constituting ROI 2000. Processor 120 can determine the reliability of each unit among the multiple units included in ROI 2000, and determine the number of units belonging to each reliability range. Processor 120 can change the parameter settings of sensor 110 based on the reliability distribution of the multiple units constituting ROI 2000 and reliability parameter setting information.

[0143] The processor 120 can change the parameter settings of the sensor 110 based on the reliability of the entire unit corresponding to the entire pixel constituting the light receiver 20. In this case, the ROI can be the region 3000 that includes all units of the second information 220, such as... Figure 8b As shown.

[0144] Specifically, the processor 120 can determine, based on the second information, the number of units among the plurality of units included in the ROI 3000 that have a reliability value less than or equal to a predetermined threshold. Here, the predetermined threshold may be the minimum reliability value required for distance measurement, such as 50, but is not limited thereto. When the number of units among the plurality of units included in the ROI 3000 that have a reliability value less than or equal to the predetermined threshold is greater than or equal to a predetermined number (e.g., 50), the processor 120 may change the parameter settings of the sensor 110.

[0145] According to an embodiment, distance information including the ROI of at least one unit having an output value of 0 is changed based on distance information obtained by sensor 110 whose parameter settings have been changed. However, this is merely exemplary, and electronic device 100 may output a value that is not 0, and distance information including the ROI of at least one unit having a reliability value less than or equal to a threshold may be changed based on distance information obtained by sensor 110 whose parameter settings have been changed.

[0146] For example, data obtained by sensor 110 such as Figure 9a The first information 910-1 shown includes distance information corresponding to multiple units, and as shown in the figure. Figure 9b The second information 910-2 shown includes reliability information corresponding to multiple units.

[0147] Reference Figure 9a and Figure 9b The processor 120 can identify at least one unit with a confidence value less than or equal to a predetermined threshold (e.g., 40) based on distance information corresponding to multiple units included in the first information 910-1 and reliability information corresponding to multiple units included in the second information 910-2. The processor 120 can set the ROI 2000 including the identified at least one unit to the first information 910-1 and the second information 910-2. For example, as... Figure 9a and Figure 9b As shown, ROI 2000 can be a rectangular area with a minimum size that includes at least one identified unit, but the embodiments are not limited to this, and ROI can be of various shapes (such as polygonal shapes).

[0148] As described above, when at least one unit with an output value of 0 is included in the first information 910-1, the processor 120 can change the parameter settings of the sensor 110, and can also obtain the third information 920-1 including distance information and the fourth information 920-2 including reliability information from the sensor 110 whose parameter settings have been changed.

[0149] The processor 120 can change (or correct, update) the output values ​​of the multiple units included in the ROI 2000 of the first information 910-1 based on the output values ​​of the multiple units included in the ROI 2000 of the third information 920-1.

[0150] Specifically, the processor 120 can apply a first weight to the output values ​​of the units included in the ROI 2000 of the first information 910-1, and can apply a second weight to the output values ​​of the units included in the ROI 2000 of the third information 920-1. The first weight can be a value obtained by dividing the confidence value of the units included in the ROI 2000 of the second information 910-2 by the sum of the confidence values ​​of the units included in the second information 910-2 and the fourth information 920-2, and the second weight can be a value obtained by dividing the confidence value of the units included in the ROI 2000 of the fourth information 920-2 by the sum of the confidence values ​​of the units included in the second information 910-2 and the fourth information 920-2. If the confidence value of the same unit is C1 in the second information 910-2 and C2 in the fourth information 920-2, then the first weight can be C1 / (C1+C2) and the second weight can be C2 / (C1+C2).

[0151] The processor 120 can change the output value of the unit included in the ROI 2000 of the first information 910-1 to the sum of the value of the output value of the unit included in the ROI 2000 of the first information 910-1 with the first weight applied to it and the value of the output value of the unit included in the ROI 2000 of the third information 920-1 with the second weight applied to it.

[0152] As an example, if the confidence value of the first unit contained in the ROI 2000 of the second information 910-2 is 25, and the confidence value of the first unit contained in the ROI 2000 of the fourth information 920-2 is 55, then the first weight can be 25 / 80, and the second weight can be 55 / 80. If the output value of the first unit contained in the ROI 2000 of the first information 910-1 is 110, and the output value of the first unit contained in the ROI 2000 of the third information 920-1 is 150, then the processor 120 can change the output value of the first unit 110 to 127 obtained by the operation 110*(25 / 80)+150*(55 / 80) (for convenience, the decimal point or smaller is rounded up). Similarly, the processor 120 can change the output values ​​of the second to nth units included in the ROI 2000 of the first information 910-1 based on multiple reliability values ​​of multiple units included in the ROI 2000 of the second information 910-2 and multiple reliability values ​​of multiple units included in the ROI 2000 of the fourth information 920-2.

[0153] like Figure 9e As shown, electronic device 100 can obtain modified fifth information 930 based on distance information obtained by sensor 110 whose parameter settings have been changed. Therefore, this disclosure can provide distance information with high quality throughout the unit.

[0154] A first weight according to the embodiment can be determined based on the standard deviation of the multiple output values ​​contained in the ROI 2000 of the first information 910-1, and a second weight can be determined based on the standard deviation of the multiple output values ​​contained in the ROI 2000 of the third information 920-1. If the average output of the N units contained in the ROI 2000 of the first information 910-1 is m1, and the output values ​​of the multiple units contained in the ROI 2000 of the first information 910-1 are x1 to xn, then the standard deviation a1 of the multiple output values ​​contained in the ROI 2000 of the first information 910-1 can be obtained by the square root operation of ((x1-m1)^2+(x2-m1)^2+…+(xn-m1)^2) / N). If the average output value of the N units contained in the ROI 2000 of the third information 920-1 is m2, and the output values ​​of the multiple units contained in the ROI 2000 of the third information 920-1 are y1 to yn, then the standard deviation a2 of the multiple output values ​​contained in the ROI 2000 of the third information 920-1 can be obtained by calculating the square root of ((y1-m2)^2+(y2-m2)^2+…+(yn-m2)^2) / N). Smaller standard deviations can be weighted higher by the processor 120. Specifically, the electronic device 100 can store information about multiple weights corresponding to multiple standard deviations (hereinafter referred to as standard deviation-weight information), where smaller standard deviations match higher weights. The processor 120 can determine the weight value corresponding to the standard deviation obtained by the above calculation from multiple pre-stored weights. For example, the processor 120 may determine a first weight corresponding to standard deviation a1 and a second weight corresponding to standard deviation a2 based on pre-stored standard deviation-weight information, wherein if standard deviation a1 is greater than standard deviation a2, the first weight may be less than the second weight.

[0155] Processor 120 can change the output value of the first unit included in the ROI 2000 of the first information 910-1 to the sum of the value of the output value of the first unit included in the ROI 2000 of the first information 910-1 with a first weight applied and the value of the output value of the first unit included in the ROI 2000 of the third information 920-1 with a second weight applied. For example, if the output value of the first unit included in the ROI 2000 of the first information 910-1 is 110, the output value of the first unit included in the ROI 2000 of the third information 920-1 is 150, the first weight is 0.4, and the second weight is 0.6, then processor 120 can change the output value of the first unit included in the ROI 2000 of the first information 910-1 from 110*0.4 to 134 obtained by the operation 110*0.4+150*0.6. Similarly, the processor 120 can change the output values ​​of the second to nth units included in the ROI 2000 of the first information 910-1 based on the first weight and the second weight.

[0156] Thus, by outputting a non-zero value, or by changing the distance information of the ROI containing at least one unit with a confidence value lower than or equal to a threshold based on the distance information obtained by sensor 110, this disclosure can obtain high-quality distance information across the entire unit.

[0157] Figure 10a This is a detailed block diagram illustrating an electronic device according to an embodiment of the present disclosure.

[0158] Reference Figure 10a An electronic device 100 according to one embodiment may include a sensor 110, a memory 130, a driving unit 140, an operation unit 150, a display 160, a communication unit 170, and a processor 120.

[0159] This is an example, and the electronic device 100 can be implemented in addition to some of the above-described configurations, and can be implemented by further including additional configurations beyond those described above. Any parts overlapping with the above description will be omitted or shortened.

[0160] The memory 130 may store the operating system (OS) for controlling the overall operation of the components of the electronic device 100, as well as instructions or data associated with the components of the electronic device 100.

[0161] The processor 120 can use various instructions or data stored in the memory 130 to control multiple hardware or software components of the electronic device 100, load instructions or data received from at least one other component into volatile memory, and store various data in non-volatile memory.

[0162] The memory 130 can store reliability parameter setting information. The reliability parameter setting information may include information about multiple parameter values ​​corresponding to multiple reliability distributions. For example, if the ROI contains more than a first plurality of pixels with confidence values ​​below a minimum threshold for distance measurement, the reliability parameter setting information may store parameter values ​​for increasing the luminous intensity of the light emitter 10 by a1 times and / or for increasing the light exposure time by b1 times. Optionally, if more than a second plurality of pixels with reliability values ​​below the minimum threshold for distance measurement are included in the ROI (but fewer than the first plurality), the reliability parameter setting information may store parameter values ​​for increasing the luminous intensity of the light emitter 20 by a2 times and / or for increasing the light exposure time by b2 times.

[0163] The memory 130 can store information about multiple parameter values ​​corresponding to multiple calculated values. The calculated values ​​can be values ​​calculated based on the reliability values ​​of multiple pixels and the weights of multiple pixels.

[0164] The driving unit 140 can move the electronic device 100. The driving unit 140 includes a drive unit (not shown) and a motor (not shown) connected to the drive unit (not shown). The drive unit (not shown) of the driving unit 140 can be implemented as the wheels or legs of a robot, and the motor (not shown) of the driving unit 140 can move the electronic device 100 by controlling the drive unit (not shown) according to the control of the processor 120.

[0165] For example, when the drive unit (not shown) is implemented as a left wheel and a right wheel, the processor 120 can send a control signal for generating a first rotational force to the motor rotating the left wheel to move the electronic device 100 in a direction that does not collide with an object in front, and send a control signal for generating a second rotational force different from the first rotational force to the motor rotating the right wheel, thereby changing the driving direction of the electronic device 100.

[0166] The operating unit 150 may include a first motor (not shown), a robotic arm (not shown) connected to the first motor (not shown), a second motor (not shown), and a robotic hand (not shown) connected to the second motor (not shown). The robotic arm (not shown) and the robotic hand (not shown) may be connected via a connector, and the robotic arm (not shown) may perform three-dimensional movement or rotation according to the drive of the first motor (not shown) connected to the robotic arm (not shown). The robotic hand (not shown) may perform three-dimensional motion, rotation, or product gripping according to the drive of the second motor (not shown) connected to the robotic hand (not shown).

[0167] Display 160 can display various screens. For example, display 160 can display information about the distance to an object or one or more objects around electronic device 100. Display 160 can display a depth map based on first information or a confidence map based on second information.

[0168] The display 160 can be implemented as a liquid crystal display (LCD) panel. According to embodiments, the display 160 can be implemented as various types of displays, such as light-emitting diodes (LEDs), organic light-emitting diodes (OLEDs), liquid crystal on silicon (LCoS), digital light processing (DLP), etc. The display 160 may also include a backlight unit and a driving circuit that can be implemented in formats such as a-si thin-film transistors (TFTs), low-temperature polycrystalline silicon (LTPS) TFTs, organic TFTs (OTFTs), etc.

[0169] The display 160 can be combined with a touch sensor and implemented as a touch screen.

[0170] The communicator 170 is configured to communicate with external devices. For example, the communicator 170 can communicate with various external devices via wireless communication methods such as Bluetooth (BT), Bluetooth Low Energy (BLE), Wi-Fi, Zigbee, etc., or infrared (IR) communication methods. The communicator 170 can be mounted on the processor 120 and can be included in the electronic device 100 as a separate configuration from the processor 120.

[0171] In addition to the above configurations, the electronic device 100 according to the embodiments may also include various configurations.

[0172] For example, the electronic device 100 may also include an input unit (not shown) capable of receiving user input. The input unit (not shown) may be implemented as a button or a touch screen and may receive various user commands, such as user commands for sensing distance to an object or user commands for moving the electronic device 100.

[0173] The electronic device 100 may also include a speaker (not shown) capable of outputting various audio data.

[0174] The electronic device 100 may also include a microphone (not shown) capable of receiving user voice. The user voice may be the user's voice used to perform tasks of the electronic device 100.

[0175] Although not in Figure 10a As shown, however, sensor 110 includes, as... Figure 10b The light transmitter 10 and light receiver 20 are shown, and the functions of the MCU 30 can be executed by the processor 120, such as... Figure 10bAs shown. In this example, the light emitter 10 can illuminate light based on a control signal received from the processor 120, and the processor 120 can determine the distance between the sensor 110 and the object based on the time it takes for the light illuminated by the light emitter 10 to be reflected by the object and then received by the light receiver 20.

[0176] Figure 11 This is a flowchart illustrating a method for controlling an electronic device according to an embodiment of the present disclosure.

[0177] In operation S1110, electronic device 100 can obtain second information related to the reliability of the first information and first information related to the distance from the object from the sensor.

[0178] The first information may include information about the distance to the object. Specifically, the first information may include information about multiple distances corresponding to multiple pixels included in the sensor. The second information may include information about the reliability of each pixel constituting the first information.

[0179] In operation S1120, the electronic device 100 can identify multiple pixels, excluding distance information, among the multiple pixels constituting the sensor based on the first information.

[0180] When receiving first information including distance information from a sensor, the electronic device 100 can identify at least one pixel among a plurality of pixels constituting the sensor that has an output value of 0 indicating the absence of distance information, based on the distance information. The identified at least one pixel may be a pixel capable of receiving light with an amount of light less than or equal to a minimum threshold required for distance measurement.

[0181] In operation S1130, the electronic device 100 can determine the reliability of the identified multiple pixels based on the second information.

[0182] The electronic device 100 can be configured to include a Region of Interest (ROI) containing at least one identified pixel. The ROI can be a square region of minimum size including at least one pixel with an output value of 0.

[0183] Electronic device 100 can determine the reliability of multiple pixels contained in the ROI based on the second information.

[0184] In operation S1140, the electronic device 100 can change the parameter settings of the sensor based on a determined reliability.

[0185] The parameters of the sensor can be, for example, parameters used to adjust the brightness of the light emitter or parameters used to adjust the exposure time of the light.

[0186] Electronic device 100 can modify sensor parameter settings based on the reliability distribution (or histogram) of multiple pixels constituting the ROI. For this purpose, electronic device 100 can determine the reliability of each pixel among the multiple pixels contained in the ROI. Electronic device 100 can determine the number of pixels belonging to a reliability range based on the reliability range. Electronic device 100 can modify sensor parameter settings based on the reliability distribution of the multiple pixels constituting the ROI and reliability parameter setting information.

[0187] In operation S1150, the electronic device 100 can obtain third information, including distance information, from a sensor whose parameter settings have been changed. The third information may include information about the distance to the object detected by a sensor that has increased the brightness of the light emitter or increased the exposure time to the light.

[0188] In operation S1160, the electronic device 100 can obtain fourth information, in which the distance information of multiple pixels identified in the first information has been changed based on the third information. The electronic device 100 can obtain the changed fourth information based on the output value included in the ROI of the first information being included in the output value of the ROI of the third information.

[0189] In operation S1170, electronic device 100 may determine the distance to an object based on the distance information contained in the fourth information. Electronic device 100 may perform processing (such as driving, stopping, and controlling the driving direction of electronic device 100) based on the distance information contained in the fourth information.

[0190] Thus, since the electronic device 100 of this disclosure performs processing (such as driving) based on fourth information, problems caused by electronic devices of the related art failing to detect the distance to a dark (such as black) object (e.g., collision with a black object, etc.) can be prevented.

[0191] Figure 12 This is a flowchart illustrating a method for controlling an electronic device according to an embodiment of the present disclosure.

[0192] In operation S1210, processor 120 may send signals requesting the creation (or transmission) of second information related to the reliability of first information and first information related to the distance to an object to sensor 110. For example, when a user command for mobile electronic device 100 is input or a user command for detecting the distance to an object is input, processor 120 may send signals requesting the creation of first and second information to sensor 110.

[0193] In operation S1220, sensor 110 can generate first information and second information according to the control of processor 120.

[0194] The MCU 30 of sensor 110 can control the light emitter 10 to emit light based on signals received from processor 120. The MCU 30 can determine the distance between sensor 110 and an object based on the time it takes for light to be received by light receiver 20 after being emitted by light emitter 10, and generate first information including information about the distance to the object through multiple units corresponding to multiple pixels. The MCU 30 of sensor 110 can determine the reliability of the distance information measured for each pixel constituting light receiver 20 based on multiple charges measured when multiple lights at different phase shifts are received in receiver 20, and generate second information including a confidence value through units corresponding to each pixel.

[0195] In operation S1230, sensor 110 can send first information and second information to processor 120, and processor 120 can identify multiple units among multiple units that do not contain distance information based on the first information.

[0196] When the processor 120 receives first information from the sensor 110, the processor 120 can identify at least one unit among a plurality of units that has an output value of 0 indicating that there is no distance information, based on the distance information of each unit included in the first information. Here, the identified at least one unit may be a unit corresponding to a pixel that receives light with a light intensity less than or equal to the minimum threshold required for distance measurement.

[0197] The processor 120 can determine the reliability of the identified multiple units based on the second information.

[0198] The processor 120 can set a Region of Interest (ROI) including at least one unit, identified based on the first information, to the second information. As an example, the ROI can be a region of minimum size square containing at least one unit with an output value of 0.

[0199] The processor 120 can determine the reliability of multiple units included in the ROI based on the second information.

[0200] Subsequently, in operation S1240, the processor 120 may send a signal requesting a change in the sensor's parameter settings to the sensor 110 based on a determined reliability.

[0201] The parameters of the sensor can be, for example, parameters used to adjust the luminescence of the light-emitting part or parameters used to adjust the exposure time of the light.

[0202] Processor 120 may send a signal requesting a change in sensor parameter settings to sensor 110 based on the reliability distribution (or histogram) of the multiple units constituting the ROI. For this purpose, processor 120 may determine the reliability of each of the multiple units included in the ROI based on second information. Processor 120 may determine the number of units in the ROI belonging to each reliability range by the reliability range, and send the signal requesting a change in sensor parameter settings based on the reliability distribution of the multiple units constituting the ROI and the reliability parameter setting information.

[0203] In operation S1250, sensor 110 can change parameter settings based on signals received from processor 120, and can send signals to processor 120 to complete the parameter setting change.

[0204] If it is determined that the parameter settings of sensor 110 have been changed, then in operation S1260, processor 120 may send a signal requesting the creation of third information to sensor 110.

[0205] In operation S1270, sensor 110 may generate third information, including information about the distance to the object, based on the signal received from processor 120, and in operation S1280, the third information may be sent to processor 120. The third information may include information about the distance to the object detected by sensor 100, which has increased the brightness of the light emitter or increased the exposure time to the light.

[0206] In operation S1290, processor 120 can generate fourth information in which the distance information of multiple units identified in the first information has been modified based on the third information. Processor 120 can obtain the fourth information in which the output value contained in the ROI of the first information is modified based on the output value contained in the ROI of the third information.

[0207] The processor 120 can obtain distance information to the object based on the distance information contained in the fourth information, and based on this, the processor 120 can perform processes (such as driving, stopping and controlling the driving direction of the electronic device 100).

[0208] Since the electronic device 100 of this disclosure performs processing (such as driving) based on fourth information, problems caused by electronic devices of the related art failing to detect the distance to a dark (such as black) object (e.g., collision with a black object, etc.) can be prevented.

[0209] The methods according to various embodiments can be implemented as software or applications that can be installed on related technology electronic devices.

[0210] The methods according to various embodiments can be implemented by software or hardware upgrades of only the relevant technical electronic devices.

[0211] The various embodiments described above can be implemented by an embedded server located in an electronic device or a server located outside the electronic device.

[0212] A non-transitory computer-readable medium may be provided, which stores a program for sequentially executing a method for controlling an electronic device according to an embodiment.

[0213] Non-transitory computer-readable media refers to media that can be read by a device. Specifically, the various applications or programs mentioned above can be stored in non-transitory computer-readable media, such as optical discs (CDs), digital versatile optical discs (DVDs), hard disks, Blu-ray discs, universal serial buses (USB), memory cards, read-only memory (ROMs), etc., and can be provided.

[0214] Although this disclosure has been shown and described with reference to various embodiments thereof, those skilled in the art will understand that various changes in form and detail may be made therein without departing from the spirit and scope of this disclosure as defined by the appended claims and their equivalents.

Claims

1. An electronic device, comprising: sensor; as well as The processor is configured as follows: The sensor obtains first information related to the distance to the object and second information related to the reliability of the first information. Based on first information, a first unit is identified among multiple units corresponding to multiple pixels constituting the sensor, including a first unit with distance information of a specific value; and based on second information, the reliability of the identified first unit is determined, wherein the specific value is 0 or an infinity value. Based on a determined level of reliability, the parameter settings of the sensor are changed, and third information related to the distance to the object is obtained from the sensor with the changed parameter settings. Based on the first and third information, a fourth information is obtained that includes distance information from multiple units, and based on the distance information included in the fourth information, the distance information to the object is obtained. The processor is further configured to: set a region of interest (ROI) including the first identified unit; determine the number of units in the ROI belonging to each reliability range as the reliability distribution of the multiple units included in the ROI by means of the reliability range; obtain parameter setting information corresponding to the reliability distribution based on the reliability parameter setting information; and change the parameter setting of the sensor based on the parameter setting information.

2. The electronic device according to claim 1, wherein, The processor is also configured to: change at least one of the parameter values ​​used to adjust the luminous intensity of the sensor or the parameter values ​​used to adjust the exposure time of light, based on the reliability of the multiple units included in the ROI.

3. The electronic device according to claim 1, wherein, The reliability of third information is relatively higher than that of first information.

4. The electronic device according to claim 1, wherein, The processor is further configured to: in the third information, determine multiple units included in the ROI set based on the first information, obtain distance information of the multiple units included in the ROI, and obtain fourth information based on the first information and the obtained distance information.

5. The electronic device according to claim 1, wherein, The processor is further configured to: identify multiple units included in a preset ROI among multiple units corresponding to multiple pixels constituting the sensor, determine the reliability of the multiple units included in the ROI based on second information, and change the parameter settings of the sensor based on the determined reliability.

6. The electronic device according to claim 1, wherein, The processor is also configured to: Based on the second information, determine the number of units with a reliability value less than or equal to a preset threshold among the multiple units corresponding to the multiple pixels constituting the sensor. Based on the fact that the number of units with a reliability value less than or equal to the preset threshold is greater than or equal to a preset number, change the parameter settings of the sensor, and obtain third information including distance information from the sensor whose parameter settings have been changed.

7. The electronic device according to claim 1, wherein, The processor is also configured to: apply weights to the reliability of the first unit based on the position of the identified first unit, and change the parameter settings of the sensor based on the reliability of the first unit to which the weights have been applied.

8. The electronic device according to claim 1, wherein, The processor is further configured to: identify at least one unit among a plurality of units including distance information whose reliability is less than or equal to a threshold based on first information, and obtain fourth information based on the first information and third information that the distance information of the identified at least one unit exceeds the threshold.

9. A control method for an electronic device, comprising: Obtain first information related to the distance to the object from the sensor and second information related to the reliability of the first information; Based on first information, a first unit is identified among multiple units corresponding to multiple pixels constituting the sensor, including a first unit with distance information of a specific value, and the reliability of the identified first unit is determined based on second information, wherein the specific value is 0 or an infinite value. Based on a determined level of reliability, the parameter settings of the sensor are modified, and third information related to the distance to the object is obtained from the sensor whose parameter settings have been modified; and Based on the first and third information, a fourth information is obtained, which includes distance information from multiple units. Furthermore, based on the distance information included in the fourth information, distance information to the object is obtained. The method of changing parameter settings includes: setting a region of interest (ROI) that includes the first unit being identified; determining the number of units in the ROI belonging to each reliability range based on the reliability range as the reliability distribution of the multiple units included in the ROI; obtaining parameter setting information corresponding to the reliability distribution based on the reliability parameter setting information; and changing the parameter settings of the sensor based on the parameter setting information.

10. The method according to claim 9, wherein, Changing parameter settings includes: altering at least one of the parameter values ​​used to adjust the luminous intensity of the sensor or the parameter values ​​used to adjust the exposure time of light, based on the reliability of the multiple units included in the ROI.

11. The method according to claim 9, wherein, The reliability of third information is relatively higher than that of first information.

Images