An electronic device including a range sensor and a method for performing auto focus

Abstract

An electronic device according to one embodiment disclosed in this document comprises a camera module including a lens unit and an image sensor, a distance sensor including a light-emitting unit and a light-receiving unit, a processor, and a memory. The processor acquires first calibration data related to an offset at a first distance shorter than a reference distance and stores it in the memory, acquires second calibration data related to crosstalk at a second distance greater than or equal to the reference distance and stores it in the memory, executes the camera module, calculates an object distance to an external object based on the first calibration data or the second calibration data, and determines the position of the lens unit for autofocus of the camera module based on the object distance. In addition to this, various other embodiments identified through the specification may be possible.

Description

Technology Field

[0001] Various embodiments disclosed in this document relate to an electronic device including a distance sensor and a method for performing autofocus using a distance sensor. Background Technology

[0002] Electronic devices such as smartphones and tablet PCs can capture images using a camera. The electronic device can support autofocus on external objects. The electronic device can calculate the distance to an object using a distance sensor (e.g., a TOF (time of flight) sensor) and perform autofocus using the calculated distance.

[0003] The distance sensor may include a light-emitting unit and a light-receiving unit. The light-emitting unit outputs light of an IR (infrared ray) pulse, and the light-receiving unit can acquire light that is reflected from an object by the IR (infrared ray) pulse. The electronic device can calculate the distance to an object by measuring the time it takes for the light output from the light-emitting unit to be reflected from an object and enter the light-receiving unit.

[0004] The electronic device can quickly determine the position of the lens for AF (autofocus) based on the calculated distance. The position of the lens for AF based on the distance to the object can be determined and stored. The problem to be solved

[0005] The electronic device can improve the accuracy of distance measurement to an object by performing calibration on the distance sensor during the manufacturing or setting stage. To improve the accuracy of distance measurement, the electronic device may simultaneously perform offset calibration and crosstalk calibration at a distance where it can avoid the influence of crosstalk, for example, at a distance of 30 cm or more from the distance sensor. In this case, at short distances (e.g., less than 30 cm), the error in distance measurement to the object may be large and the accuracy of AF may decrease.

[0006] Various embodiments of the present invention provide an electronic device that improves the accuracy of distance measurement to an object by performing offset calibration and crosstalk calibration of a distance sensor at different distances. means of solving the problem

[0007] An electronic device according to various embodiments includes a camera module including a lens unit and an image sensor, a distance sensor including a light-emitting unit and a light-receiving unit, a processor, and a memory. The processor acquires first calibration data related to an offset at a first distance shorter than a reference distance and stores it in the memory, acquires second calibration data related to crosstalk at a second distance greater than or equal to the reference distance and stores it in the memory, executes the camera module, calculates an object distance to an external object based on the first calibration data or the second calibration data, and determines the position of the lens unit for autofocus of the camera module based on the object distance. Effects of the invention

[0008] The electronic device according to the various embodiments disclosed in this document can improve the accuracy of distance measurement to an object by performing offset calibration and crosstalk calibration of the distance sensor at different distances. Through this, the accuracy of near-field AF can be improved.

[0009] The electronic device according to the various embodiments disclosed in this document may output a user interface to allow the user to remove obstructing material when crosstalk is at a level that affects object distance measurement. Brief explanation of the drawing

[0010] FIG. 1 is a block diagram of an electronic device in a network environment according to various embodiments. FIG. 2 shows an electronic device according to various embodiments. FIG. 3 shows a distance sensor according to various embodiments. FIG. 4 illustrates the calibration operation of a distance sensor according to various embodiments. FIG. 5 is a flowchart illustrating the priority storage of first calibration data according to various embodiments. FIG. 6 is a flowchart illustrating a method for performing auto-focusing according to various embodiments. Figure 7 shows the error rate according to distance according to various embodiments. FIG. 8 shows the display of a UI according to the error in object distance measurement according to various embodiments. In relation to the description of the drawings, the same or similar reference numerals may be used for identical or similar components. Specific details for implementing the invention

[0011] Hereinafter, various embodiments of this document are described with reference to the accompanying drawings. However, this is not intended to limit the technology described in this document to specific embodiments and should be understood to include various modifications, equivalents, and / or alternatives to the embodiments of this document. In relation to the description of the drawings, similar reference numerals may be used for similar components.

[0012] FIG. 1 is a block diagram of an electronic device (101) in a network environment (100) according to various embodiments. The electronic device according to the various embodiments disclosed in this document may be a device of various forms. The electronic device may include, for example, at least one of a portable communication device (e.g., a smartphone), a computer device (e.g., a personal digital assistant (PDA), a tablet PC, a laptop PC (desktop PC, workstation, or server), a portable multimedia device (e.g., an e-book reader or MP3 player), a portable medical device (e.g., a heart rate, blood glucose, blood pressure, or body temperature monitor), a camera, or a wearable device. The wearable device may include at least one of an accessory type (e.g., a watch, ring, bracelet, anklet, necklace, glasses, contact lens, or head-mounted device (HMD)), a fabric or clothing integrated type (e.g., electronic clothing), a body-attached type (e.g., a skin pad or tattoo), or a bio-implantable circuit. In some embodiments, the electronic device may include, for example, a television, a DVD (digital video disk) player, an audio device, an audio accessory device (e.g., a speaker, headphones, or a headset), a refrigerator, an air conditioner, a vacuum cleaner, an oven, a microwave oven, a washing machine, an air purifier, a set-top box, or a home automation control. It may include at least one of a panel, a security control panel, a game console, an electronic dictionary, an electronic key, a camcorder, or an electronic photo frame.

[0013] In other embodiments, the electronic device may include at least one of a navigation device, a satellite navigation system (GNSS (global navigation satellite system)), an event data recorder (EDR) (e.g., a black box for a vehicle / ship / aircraft), an automotive infotainment device (e.g., a head-up display for a vehicle), an industrial or domestic robot, a drone, an automated teller machine (ATM), a point of sales (POS) device, a measuring device (e.g., a water, electricity, or gas meter), or an Internet of Things device (e.g., a light bulb, a sprinkler system, a fire alarm, a thermostat, or a street light). The electronic device according to the embodiments of this document is not limited to the aforementioned devices, and may also provide a combination of the functions of multiple devices, such as in the case of a smartphone equipped with a function to measure personal biometric information (e.g., heart rate or blood sugar). In this document, the term "user" may refer to a person using the electronic device or a device using the electronic device (e.g., an artificial intelligence electronic device).

[0014] In a network environment (100), an electronic device (101) may communicate with an electronic device (102) through a first network (198) (e.g., a short-range wireless communication network) or with an electronic device (104) or a server (108) through a second network (199) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (101) may communicate with an electronic device (104) through a server (108). According to one embodiment, the electronic device (101) may include a processor (120), memory (130), input device (150), sound output device (155), display device (160), audio module (170), sensor module (176), interface (177), connection terminal (178), haptic module (179), camera module (180), power management module (188), battery (189), communication module (190), subscriber identification module (196), or antenna module (197). In some embodiments, the electronic device (101) may include at least one of these components (e.g., connection terminal (178) (1(1)) which may be omitted or one or more other components may be added. In some embodiments, some of these components (e.g., sensor module (176), camera module (180), or antenna module (197)) may be integrated into a single component (e.g., display device (160)).

[0015] The processor (120) can control at least one other component (e.g., a hardware or software component) of the electronic device (101) connected to the processor (120) by executing software (e.g., a program (140)), for example, and can perform various data processing or operations. According to one embodiment, as at least part of the data processing or operations, the processor (120) can store commands or data received from other components (e.g., a sensor module (176) or a communication module (190)) in volatile memory (132), process the commands or data stored in volatile memory (132), and store the resulting data in non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., a central processing unit or an application processor) or an auxiliary processor (123) that can operate independently or together with it (e.g., a graphics processing unit, an image signal processor, a sensor hub processor, or a communication processor). For example, if the electronic device (101) includes a main processor (121) and an auxiliary processor (123), the auxiliary processor (123) may be configured to use less power than the main processor (121) or to be specialized for a designated function. The auxiliary processor (123) may be implemented separately from the main processor (121) or as part thereof.

[0016] The auxiliary processor (123) can control at least some of the functions or states associated with at least one component of the electronic device (101) (e.g., display device (160), sensor module (176), or communication module (190)) on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. According to one embodiment, the auxiliary processor (123) (e.g., image signal processor or communication processor) may be implemented as part of another functionally related component (e.g., camera module (180) or communication module (190)).

[0017] The memory (130) can store various data used by at least one component of the electronic device (101) (e.g., processor (120) or sensor module (176)). The data may include, for example, software (e.g., program (140)) and input data or output data for related commands. The memory (130) may include volatile memory (132) or non-volatile memory (134).

[0018] The program (140) may be stored as software in memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0019] The input device (150) can receive commands or data to be used for a component of the electronic device (101) (e.g., processor (120)) from outside the electronic device (101) (e.g., user). The input device (150) may include, for example, a microphone, a mouse, a keyboard, or a digital pen (e.g., a stylus pen).

[0020] The sound output device (155) can output a sound signal to the outside of the electronic device (101). The sound output device (155) may include, for example, a speaker or a receiver. The speaker may be used for general purposes, such as multimedia playback or recording playback. The receiver may be used to receive incoming calls. According to one embodiment, the receiver may be implemented separately from the speaker or as part thereof.

[0021] The display device (160) can visually provide information to an external (e.g., user) of the electronic device (101). The display device (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling said device. According to one embodiment, the display device (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of the force generated by said touch.

[0022] The audio module (170) can convert sound into an electrical signal or, conversely, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through an input device (150) or output sound through an audio output device (155) or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphones) that is directly or wirelessly connected to the electronic device (101).

[0023] The sensor module (176) can detect the operating state of the electronic device (101) (e.g., power or temperature) or the external environmental state (e.g., user state) and generate an electrical signal or data value corresponding to the detected state. According to one embodiment, the sensor module (176) may include, for example, a gesture sensor, a gyroscope sensor, a barometric pressure sensor, a magnetic sensor, an accelerometer sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biosensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0024] The interface (177) may support one or more specified protocols that can be used for the electronic device (101) to be connected directly or wirelessly to an external electronic device (e.g., electronic device (102)). According to one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0025] The connection terminal (178) may include a connector through which the electronic device (101) can be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0026] The haptic module (179) can convert an electrical signal into a mechanical stimulus (e.g., vibration or movement) or an electrical stimulus that the user can perceive through tactile or kinesthetic senses. According to one embodiment, the haptic module (179) may include, for example, a motor, a piezoelectric element, or an electric stimulation device.

[0027] The camera module (180) can capture still images and video. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0028] The power management module (188) can manage the power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented, for example, as at least part of a power management integrated circuit (PMIC).

[0029] The battery (189) can supply power to at least one component of the electronic device (101). According to one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0030] The communication module (190) can support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between an electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may include one or more communication processors that operate independently of the processor (120) (e.g., application processor) and support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., cellular communication module, short-range wireless communication module, or GNSS (global navigation satellite system) communication module) or a wired communication module (194) (e.g., LAN (local area network) communication module, or power line communication module). The corresponding communication module among these communication modules can communicate with an external electronic device (104) through a first network (198) (e.g., a short-range communication network such as Bluetooth, WiFi (wireless fidelity) direct, or IrDA (infrared data association)) or a second network (199) (e.g., a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules may be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can identify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) using subscriber information (e.g., International Mobile Subscriber Identifier (IMSI)) stored in the subscriber identification module (196).

[0031] An antenna module (197) can transmit a signal or power to or from an external source (e.g., an external electronic device). According to one embodiment, the antenna module (197) may include an antenna comprising a radiator made of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). According to one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as a first network (198) or a second network (199), may be selected from the plurality of antennas, for example, by a communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device through the selected at least one antenna. According to some embodiments, in addition to the radiator, other components (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as part of the antenna module (197).

[0032] At least some of the above components can be connected to each other via a communication method between peripheral devices (e.g., bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)) and exchange signals (e.g., commands or data) with each other.

[0033] According to one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) through a server (108) connected to a second network (199). Each of the external electronic devices (102, or 104) may be the same or different type of device as the electronic device (101). According to one embodiment, all or part of the operations performed on the electronic device (101) may be performed on one or more of the external electronic devices (102, 104, or 108). For example, if the electronic device (101) needs to perform a function or service automatically or in response to a request from a user or another device, the electronic device (101) may request one or more external electronic devices to perform at least part of the function or service instead of performing the function or service itself or additionally. One or more external electronic devices that receive the above request may execute at least part of the requested function or service, or additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may provide the result as is or additionally processed as at least part of the response to the request. For this purpose, for example, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used.

[0034] The electronic device according to the various embodiments disclosed in this document may be of various forms. The electronic device may include, for example, a portable communication device (e.g., a smartphone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a consumer electronics device. The electronic device according to the embodiments of this document is not limited to the devices described above.

[0035] FIG. 2 shows an electronic device according to various embodiments.

[0036] Referring to FIG. 2, an electronic device (201) (e.g., the electronic device (101) of FIG. 1) may include a processor (210), memory (220), a display (230), a distance sensor (240), and / or a camera module (250). FIG. 2 is illustrated with a focus on configurations related to measuring distance to an object or capturing an image, but is not limited thereto.

[0037] A processor (210) (e.g., processor (120) of FIG. 1) can perform various operations necessary for the operation of an electronic device (201). For example, the processor (210) can drive a distance sensor (240) to calculate the distance to an object (hereinafter, object distance). As another example, the processor (210) can drive a camera module (250) to perform AF on an object and take an image.

[0038] The memory (220) (e.g., the memory (120) of FIG. 1) can store various information used to drive the electronic device (201). For example, the memory (220) can store calibration information of the distance sensor (240), reference data for controlling the distance sensor (240) according to noise, or data regarding the power of the distance sensor (240). As another example, the memory (220) can store information regarding the position of the lens part of the camera module (250) according to the object distance for performing AF, or information related to the capture of an image (e.g., data for light source recognition, data regarding the intensity and color temperature of the light source, or data for detecting noise of the light source).

[0039] The display (230) can display content such as images, icons, user interfaces, or text. For example, the display (230) can display an image based on image data acquired through the camera module (250). As another example, the display (230) can display a UI indicating a focused object according to the AF function.

[0040] A distance sensor (240) can be used to calculate the object distance to an external object. The distance sensor (240) can output light of a specified wavelength (e.g., IR) (hereinafter, transmitted light) and collect light reflected from an external object (hereinafter, received light). A control circuit of the processor (210) or the distance sensor (240) can calculate the object distance based on the time from the output time of the transmitted light to the arrival time of the received light. Additional information regarding the distance sensor (240) may be provided through FIG. 3.

[0041] A camera module (or camera device, image element, or image device) (250) (e.g., camera module (180) of FIG. 1) can acquire image data (e.g., RGB data). The camera module (250) may include a lens unit, an image sensor, or an image processing unit. According to one embodiment, the camera module (250) can perform AF by moving the lens unit to a position corresponding to the object distance.

[0042] FIG. 3 shows a distance sensor according to various embodiments.

[0043] Referring to FIG. 3, the distance sensor (240) may include a light-emitting part (310), a light-receiving part (320), an auxiliary light-receiving part (325), or a surface part (330).

[0044] Although not shown in FIG. 3, the distance sensor (240) may further include a control circuit that controls the light-emitting unit (310), the light-receiving unit (320), or the auxiliary light-receiving unit (325) or processes data. The control circuit of the processor (210) or the distance sensor (240) can calculate the distance (object distance) between the distance sensor (240) and the object (350) based on the time from the time of light output to the time of arrival of reflected light.

[0045] According to various embodiments, the emitting unit (310) may output transmitted light (e.g., IR) of a specified wavelength. The receiving unit (320) may detect received light that is reflected from the object (350) by the transmitted light. An auxiliary receiving unit (325) may be placed in an area adjacent to the emitting unit (310). The auxiliary receiving unit (325) may be used to detect the effect of crosstalk. A surface unit (330) may be placed on the front of the emitting unit (310) and the receiving unit (320). The surface unit (330) may include glass or a polymer. For example, at least a portion of the surface unit (330) may be treated using a paint or film so that the emitting unit (310) or the receiving unit (320) is not visible. As another example, the surface unit (330) may be included in a housing included in an electronic device (e.g., the electronic device (201) of FIG. 2).

[0046] According to various embodiments, crosstalk may occur during operation of the distance sensor (240). The crosstalk may be light entering from the light-emitting part (310) to the light-receiving part (320) via a path other than the path reflected from the object (350). For example, crosstalk may occur through a structure to which the light-emitting part (310) and the light-receiving part (320) are fixed, the space between the light-emitting part (310) (or the light-receiving part (320)) and the surface part (330), and the internal space of the surface part (330) (waveguide effect). If a paint or film that blocks visible light is applied to the light-emitting part (310) or the light-receiving part (320), the crosstalk may be further increased.

[0047] Crosstalk may be a factor that interferes with object distance recognition with the object (350) and may be excluded when calculating object distance through calibration data related to the stored crosstalk.

[0048] The graph (360) indicates the number of photons entering the light receiving unit (320) over time. The light receiving unit (320) can collect data on photons (361) caused by crosstalk arriving around the first time (t1), and photons (362) arriving around the second time (t2) caused by photons from the receiving light (Rx) reflected through the object (350). Data caused by photons from crosstalk can be removed through calibration data.

[0049] When the distance at which crosstalk calibration is performed is relatively close to the distance sensor (240), the interval between the first time (t1) and the second time (t2) may be small, and as a result, it may be difficult to distinguish between the photon (361) caused by crosstalk and the photon (362) reflected from the object (350), and the crosstalk may not be effectively removed.

[0050] The control circuit of the processor (210) or distance sensor (240) can perform crosstalk calibration at a second distance greater than a specified reference distance (e.g., about 30 cm) and store calibration data.

[0051] FIG. 4 illustrates the calibration operation of a distance sensor according to various embodiments.

[0052] Referring to FIG. 4, in operation 410, the processor (210) can start calibration of the distance sensor (240). For example, in at least one of the manufacturing operation of the distance sensor (240), the setting operation of the distance sensor (240), and the execution operation of an application related to the distance sensor (240), the processor (210) can proceed with calibration of the distance sensor (240).

[0053] In operation 420, the processor (210) can check whether the calibration is within a reference distance (e.g., 30 cm). For example, the reference distance can be set according to the degree of crosstalk occurrence.

[0054] In operation 430, if the calibration is within a reference distance, the processor (210) may perform offset calibration at a first distance (e.g., 10 cm) that is smaller than the set reference distance (e.g., about 30 cm). Offset calibration may be an operation that corrects the distance measurement value according to the manufacturing characteristics of the distance sensor (240). For example, if the actual distance to an object is 100 cm and the object distance measured using the distance sensor (240) is 98 cm, the processor (210) may store the correction value by offset calibration as 2 cm.

[0055] In operation 435, the processor (210) can store first calibration data according to offset calibration performed at a first distance in memory (220).

[0056] In operation 440, if the calibration is greater than (or exceeds) the reference distance, the processor (210) may perform crosstalk calibration at a second distance greater than or equal to the reference distance (e.g., about 30 cm). If the distance at which crosstalk calibration is performed is relatively close to the distance sensor (240), it may be difficult to distinguish between photons from crosstalk and photons reflected from an object. The processor (210) may perform crosstalk calibration at a second distance greater than the specified reference distance (e.g., about 30 cm).

[0057] In operation 445, the processor (210) can store second calibration data according to crosstalk calibration performed at a second distance in memory (220).

[0058] FIG. 5 is a flowchart illustrating the priority storage of first calibration data according to various embodiments.

[0059] Referring to FIG. 5, in operation 510, first calibration data by offset calibration may be stored in a storage element included in memory (240) or distance sensor (240). For example, the first calibration data may be transmitted from the manufacturer of the distance sensor (240) to the manufacturer of the electronic device (201). The first calibration data may be stored in memory (240) during the manufacturing operation of the electronic device (201). The first calibration data may be performed and stored at a first distance (10 cm) that is smaller than a reference distance (e.g., 30 cm).

[0060] In operation 520, the processor (210) may perform crosstalk calibration at a second distance greater than or equal to a reference distance (e.g., about 30 cm). If the distance at which crosstalk calibration is performed is relatively close to the distance sensor (240), it may be difficult to distinguish between photons from crosstalk and photons reflected from an object. The processor (210) may perform crosstalk calibration at a second distance greater than a specified reference distance (e.g., about 30 cm).

[0061] In operation 530, the processor (210) can store second calibration data according to crosstalk calibration performed at a second distance in memory (220).

[0062] FIG. 6 is a flowchart illustrating a method for performing auto-focusing according to various embodiments.

[0063] Referring to FIG. 6, in operation 610, the processor (210) can execute a camera application using the camera module (180). After the camera application is executed, the processor (210) can display a preview image on the display (230) based on image data obtained through the camera module (180).

[0064] In operation 620, the processor (210) can primarily calculate the distance to an external object (hereinafter, the measured distance) using a distance sensor (240). The processor (210) can output a transmitted light of an IR pulse through the light-emitting part (e.g., the light-emitting part (310) of FIG. 3) of the distance sensor (240), and collect data on the received light reflected from the object of the IR pulse through the light-receiving part (e.g., the light-receiving part (320) of FIG. 3) of the distance sensor (240). The processor (210) can calculate the measured distance based on the speed and travel time of the IR pulse (from the output time of the transmitted light to the reception time of the received light).

[0065] According to various embodiments, the processor (210) can calculate the object distance by correcting the measurement distance based on the first calibration data or the second calibration data. The first calibration data may be correction data related to the distance offset according to the manufacturing characteristics of the distance sensor (240). The second calibration data may be correction data related to crosstalk.

[0066] According to one embodiment, if the measured distance is smaller than a reference distance (e.g., 30 cm), the processor (210) may correct the measured distance using the first calibration data and not use the second calibration data.

[0067] According to another embodiment, the processor (210) may correct the measurement distance using the second calibration data and not use the first calibration data when the measurement distance is greater than or equal to the reference distance (e.g., 30 cm).

[0068] According to one embodiment, prior to using the distance sensor (240), the processor (210) can first calculate the object distance using a phase detector, and if the calculated distance to the object is less than or equal to a specified first reliability, the distance sensor (240) can secondarily calculate the object distance.

[0069] In operation 630, the processor (210) can determine whether the object distance calculated using the distance sensor (240) is greater than (or exceeds) a specified reliability.

[0070] According to various embodiments, if the calculated object distance is less than (or lower than) a specified reliability, the processor (210) may not use the object distance calculated using the distance sensor (240) for AF operation (NO of operation 630).

[0071] In operation 640, if the object distance calculated using the distance sensor (240) is greater than (or exceeds) the specified reliability, the processor (210) can move the lens portion of the camera module (250) using the object distance.

[0072] According to one embodiment, the memory (220) may store a table corresponding to the object distance and the position of the lens part. The processor (210) may determine the position of the lens part corresponding to the calculated object distance by referring to the table.

[0073] According to various embodiments, the processor (210) can display an AF UI on the display (230). According to one embodiment, the processor (210) can fine-tune the AF UI based on the state information (movement, rotation) of the electronic device.

[0074] FIG. 7 shows the error rate according to distance according to various embodiments. FIG. 7 is exemplary and is not limited thereto.

[0075] Referring to FIG. 7, the first graph (701) shows the error rate by distance of a distance sensor (240) that has performed both offset calibration and crosstalk calibration at a reference distance (Ls) (e.g., 30 cm). In this case, the error rate of the object distance calculated at a distance within the reference distance (Ls) is relatively large (e.g., about 20%), whereas the error rate of the object distance calculated at a distance greater than the reference distance (Ls) may be relatively small (e.g., about 5%).

[0076] The second graph (702) shows the error rate by distance of a distance sensor (240) that performs offset calibration at a first distance (e.g., about 10 cm) within a reference distance (Ls) (e.g., 30 cm) and performs crosstalk calibration at a second distance greater than or equal to the reference distance (Ls) (e.g., 30 cm). According to one embodiment, if the measured distance calculated through the distance sensor (240) is less than (or less than) the reference distance (Ls) (e.g., 30 cm), the processor (210) can correct the measured distance using the first calibration data by offset calibration. If the measured distance calculated through the distance sensor (240) is greater than (or greater than) the reference distance (Ls) (e.g., 30 cm), the processor (210) can correct the measured distance using the second calibration data by crosstalk calibration. In this case, the error rate of the object distance calculated at a distance within the reference distance (Ls) and the error rate of the object distance calculated at a distance greater than the reference distance (Ls) may both be relatively small (e.g., about 5%).

[0077] FIG. 8 shows the display of a UI according to the error in object distance measurement according to various embodiments.

[0078] Referring to FIG. 8, the processor (210) can check whether the crosstalk deviates from the reference value and an error in distance measurement occurs.

[0079] For example, if water droplets or contaminants are deposited on the surface of the distance sensor (240) (e.g., the surface (330) of FIG. 3), the crosstalk may deviate from the reference value.

[0080] According to one embodiment, the reference value may be determined based on the number of photons detected by the distance sensor (240). For example, the reference value may be one of approximately 100 kcps (kilo counts per second) to approximately 120 kcps.

[0081] The processor (210) may display a UI (810) on the display (230) when the crosstalk deviates from a reference value. For example, the UI (810) may be a pop-up window that allows the user to remove foreign matter from the surface of the distance sensor (240).

[0082] An electronic device according to various embodiments (e.g., electronic device (101) of FIG. 1, electronic device (201) of FIG. 2) comprises a camera module including a lens portion and an image sensor (e.g., camera module (180) of FIG. 1, camera module (250) of FIG. 2), a distance sensor including a light-emitting portion and a light-receiving portion (e.g., sensor module (176) of FIG. 1, distance sensor (240) of FIG. 2), a processor (e.g., processor (120) of FIG. 1, processor (210) of FIG. 2), and a memory (e.g., memory (130) of FIG. 1, memory (220) of FIG. 2), wherein the processor (e.g., processor (120) of FIG. 1, processor (210) of FIG. 2)) acquires first calibration data related to an offset at a first distance shorter than a reference distance and stores it in the memory (e.g., memory (130) of FIG. 1, memory (220) of FIG. 2), and a second calibration data related to crosstalk at a second distance greater than or equal to the reference distance Calibration data can be acquired and stored in the memory (e.g., memory (130) of FIG. 1, memory (220) of FIG. 2), and the camera module (e.g., camera module (180) of FIG. 1, camera module (250) of FIG. 2) can be executed, and the object distance to an external object can be calculated based on the first calibration data or the second calibration data, and the position of the lens part for autofocus of the camera module (e.g., camera module (180) of FIG. 1, camera module (250) of FIG. 2) can be determined based on the object distance.

[0083] According to various embodiments, the processor (e.g., processor (120) of FIG. 1, processor (210) of FIG. 2)) can calculate the object distance by correcting the distance calculated using the distance sensor (e.g., sensor module (176) of FIG. 1, distance sensor (240) of FIG. 2) using the first calibration data or the second calibration data.

[0084] According to various embodiments, the processor (e.g., processor (120) of FIG. 1, processor (210) of FIG. 2)) can correct the distance based on the first calibration data when the distance is within the reference distance.

[0085] According to various embodiments, the processor (e.g., processor (120) of FIG. 1, processor (210) of FIG. 2)) can correct the distance based on the second calibration data when the distance is greater than or equal to the reference distance.

[0086] According to various embodiments, the reference distance may be determined based on the crosstalk characteristics of the distance sensor (e.g., sensor module (176) of FIG. 1, distance sensor (240) of FIG. 2).

[0087] According to various embodiments, the electronic device (e.g., the electronic device (101) of FIG. 1, the electronic device (201) of FIG. 2) further includes a display (e.g., the display device (160) of FIG. 1, the display (230) of FIG. 2), and the processor (e.g., the processor (120) of FIG. 1, the processor (210) of FIG. 2)) can display a user interface corresponding to the autofocus on the display (e.g., the display device (160) of FIG. 1, the display (230) of FIG. 2).

[0088] According to various embodiments, the electronic device (e.g., electronic device (101) of FIG. 1, electronic device (201) of FIG. 2) further includes a display (e.g., display device (160) of FIG. 1, display (230) of FIG. 2), and the processor (e.g., processor (120) of FIG. 1, processor (210) of FIG. 2)) may display a user interface on the display (e.g., display device (160) of FIG. 1, display (230) of FIG. 2) to remove foreign matter from the surface of the distance sensor (e.g., sensor module (176) of FIG. 1, distance sensor (240) of FIG. 2) when data from crosstalk measured by the distance sensor (e.g., sensor module (176) of FIG. 1, distance sensor (240) of FIG. 2) exceeds a reference value.

[0089] According to various embodiments, the reference distance may be 30 cm from the surface of the distance sensor (e.g., sensor module (176) of FIG. 1, distance sensor (240) of FIG. 2), and the first distance may be 10 cm from the surface of the distance sensor (e.g., sensor module (176) of FIG. 1, distance sensor (240) of FIG. 2).

[0090] According to various embodiments, the first calibration data may be stored in a storage element included in the distance sensor (e.g., sensor module (176) of FIG. 1, distance sensor (240) of FIG. 2).

[0091] According to various embodiments, the first calibration data and the second calibration data may be produced during the manufacturing stage of the electronic device (e.g., the electronic device (101) of FIG. 1, the electronic device (201) of FIG. 2) and stored in the memory (e.g., the memory (130) of FIG. 1, the memory (220) of FIG. 2).

[0092] According to various embodiments, an auto focus execution method is performed in an electronic device (e.g., electronic device (101) of FIG. 1, electronic device (201) of FIG. 2), and using a distance sensor (e.g., sensor module (176) of FIG. 1, distance sensor (240) of FIG. 2) of the electronic device (e.g., electronic device (101) of FIG. 1, electronic device (201) of FIG. 2), the operation of acquiring and storing first calibration data related to an offset at a first distance shorter than a reference distance using the distance sensor (e.g., sensor module (176) of FIG. 1, distance sensor (240) of FIG. 2), the operation of acquiring and storing second calibration data related to crosstalk at a second distance greater than or equal to a reference distance using the distance sensor (e.g., sensor module (176) of FIG. 1, distance sensor (240) of FIG. 2), and the operation of executing a camera module (e.g., camera module (180) of FIG. 1, camera module (250) of FIG. 2) of the electronic device (e.g., electronic device (101) of FIG. 1, electronic device (201) of FIG. 2). The method may include an operation of calculating an object distance to an external object based on the first calibration data or the second calibration data, and an operation of determining the position of the lens portion of the camera module (e.g., the camera module (180) of FIG. 1, the camera module (250) of FIG. 2) for autofocus of the camera module (e.g., the camera module (180) of FIG. 1, the camera module (250) of FIG. 2) based on the object distance.

[0093] According to various embodiments, the operation of determining the position of the lens portion may include the operation of calculating the object distance by correcting the distance calculated using the distance sensor (e.g., sensor module (176) of FIG. 1, distance sensor (240) of FIG. 2) using the first calibration data or the second calibration data.

[0094] According to various embodiments, the operation of determining the position of the lens portion may further include an operation of correcting the distance based on the first calibration data when the distance is within the reference distance.

[0095] According to various embodiments, the operation of determining the position of the lens portion may further include an operation of correcting the distance based on the second calibration data when the distance is greater than or equal to the reference distance.

[0096] According to various embodiments, the reference distance may be determined based on the crosstalk characteristics of the distance sensor (e.g., sensor module (176) of FIG. 1, distance sensor (240) of FIG. 2).

[0097] According to various embodiments, the method may further include the operation of displaying a user interface corresponding to the autofocus on a display (e.g., display device (160) of FIG. 1, display (230) of FIG. 2) of the electronic device (e.g., electronic device (101) of FIG. 1, electronic device (201) of FIG. 2).

[0098] According to various embodiments, the method may further include an operation of displaying a user interface on the display (e.g., the display device (160) of FIG. 1, the display (230) of FIG. 2) of the electronic device (e.g., the electronic device (101) of FIG. 1, the electronic device (201) of FIG. 2) to remove foreign matter from the surface of the distance sensor (e.g., the sensor module (176) of FIG. 1, the distance sensor (240) of FIG. 2) when data from crosstalk measured by the distance sensor (e.g., the sensor module (176) of FIG. 1, the distance sensor (240) of FIG. 2) exceeds a reference value.

[0099] According to various embodiments, the reference distance may be 30 cm from the surface of the distance sensor (e.g., sensor module (176) of FIG. 1, distance sensor (240) of FIG. 2), and the first distance may be 10 cm from the surface of the distance sensor (e.g., sensor module (176) of FIG. 1, distance sensor (240) of FIG. 2).

[0100] According to various embodiments, the operation of acquiring and storing the 1 calibration data may include the operation of loading the 1 calibration data stored in a storage element included in the distance sensor (e.g., sensor module (176) of FIG. 1, distance sensor (240) of FIG. 2).

[0101] According to various embodiments, the first calibration data and the second calibration data may be produced during the manufacturing stage of the electronic device (e.g., the electronic device (101) of FIG. 1, the electronic device (201) of FIG. 2) and stored in the memory (e.g., the memory (130) of FIG. 1, the memory (220) of FIG. 2) of the electronic device (e.g., the electronic device (101) of FIG. 1, the electronic device (201) of FIG. 2).

[0102] The electronic device according to the various embodiments disclosed in this document may be of various forms. The electronic device may include, for example, a portable communication device (e.g., a smartphone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a consumer electronics device. The electronic device according to the embodiments of this document is not limited to the devices described above.

[0103] The various embodiments of this document and the terms used therein are not intended to limit the technical features described in this document to specific embodiments, and should be understood to include various modifications, equivalents, or substitutions of said embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of said items unless the relevant context clearly indicates otherwise. In this document, phrases such as “A or B,” “at least one of A and B,” “at least one of A or B,” “A, B or C,” “at least one of A, B and C,” and “at least one of A, B, or C” each may include any one of the items listed together in the corresponding phrase, or all possible combinations thereof. Terms such as “first,” “second,” or “first” or “second” may be used simply to distinguish a component from another corresponding component and do not limit the components in any other aspect (e.g., importance or order). Where any (e.g., 1st) component is referred to as “coupled” or “connected” to another (e.g., 2nd) component, with or without the terms “functionally” or “communicationly,” it means that said any component may be connected to said other component directly (e.g., via a wire), wirelessly, or through a third component.

[0104] As used herein, the term "module" may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be a component formed integrally, or a minimum unit of said component or a part thereof that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0105] Various embodiments of the present document may be implemented as software (e.g., program (140)) comprising one or more instructions stored in a storage medium (e.g., internal memory (136) or external memory (138)) readable by a machine (e.g., electronic device (101)). For example, a processor (e.g., processor (120)) of a machine (e.g., electronic device (801)) may call at least one of the one or more instructions stored from the storage medium and execute it. This enables the machine to be operated to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code that can be executed by an interpreter. The storage medium readable by the machine may be provided in the form of a non-transitory storage medium. Here, 'non-temporary' simply means that the storage medium is a tangible device and does not contain a signal (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently and cases where it is stored temporarily.

[0106] According to one embodiment, the method according to the various embodiments disclosed herein may be provided by being included in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)) or an application store (e.g., Play Store). TM It can be distributed online (e.g., downloaded or uploaded) through ) or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily created on a device-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0107] According to various embodiments, each component (e.g., module or program) of the components described above may include a singular or multiple entities. According to various embodiments, one or more of the components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Generally or additionally, multiple components (e.g., module or program) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each of the components of the multiple components in the same or similar manner as those performed by the corresponding component among the multiple components prior to the integration. According to various embodiments, operations performed by the module, program, or other components may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

Claims

Claim 1 An electronic device comprising: a camera module including a lens unit and an image sensor; a distance sensor including a light-emitting unit and a light-receiving unit; a processor; and a memory; wherein the processor acquires first calibration data related to an offset at a first distance shorter than a reference distance and stores it in the memory, acquires second calibration data related to crosstalk at a second distance greater than or equal to the reference distance and stores it in the memory, executes the camera module, calculates an object distance to an external object based on the first calibration data or the second calibration data, determines the position of the lens unit for autofocus of the camera module based on the object distance, and the first calibration data is stored in a storage element included in the distance sensor. Claim 2 In claim 1, the processor is an electronic device that calculates the object distance by correcting the distance calculated using the distance sensor using the first calibration data or the second calibration data. Claim 3 In paragraph 2, the processor is an electronic device that corrects the distance based on the first calibration data when the distance is within the reference distance. Claim 4 In paragraph 2, the processor is an electronic device that corrects the distance based on the second calibration data when the distance is greater than or equal to the reference distance. Claim 5 An electronic device according to claim 1, wherein the reference distance is determined based on the crosstalk characteristics of the distance sensor. Claim 6 An electronic device according to claim 1, further comprising a display, wherein the processor displays a user interface corresponding to the autofocus on the display. Claim 7 An electronic device according to claim 1, further comprising a display, wherein the processor displays a user interface on the display to remove foreign matter from the surface of the distance sensor when data from crosstalk measured by the distance sensor exceeds a reference value. Claim 8 delete Claim 9 delete Claim 10 An electronic device according to claim 1, wherein the first calibration data and the second calibration data are calculated during the manufacturing stage of the electronic device and stored in the memory or the storage element. Claim 11 A method for performing autofocus in an electronic device comprises: an operation of acquiring and storing first calibration data related to an offset at a first distance shorter than a reference distance using a distance sensor of the electronic device; an operation of acquiring and storing second calibration data related to crosstalk at a second distance greater than or equal to a reference distance using the distance sensor; an operation of executing a camera module of the electronic device; an operation of calculating an object distance to an external object based on the first calibration data or the second calibration data; and an operation of determining the position of a lens portion of the camera module for autofocus of the camera module based on the object distance; wherein the first calibration data is stored in a storage element included in the distance sensor. Claim 12 delete Claim 13 delete Claim 14 delete Claim 15 delete Claim 16 delete Claim 17 delete Claim 18 delete Claim 19 delete Claim 20 delete

Images