Method and electronic device for acquiring biometric information using light from a display

Abstract

The electronic device according to the embodiment may include a housing, a display exposed through at least a portion of a first surface of the housing, an image sensor, a processor operably connected to the display and the image sensor, and a memory operably connected to the processor, wherein the memory may store instructions that, when executed, cause the processor to detect a touch of a user's finger on the image sensor while displaying a user interface using the display. In response to detecting the touch, while the touch of the finger remains on the image sensor, the user interface in a display area within a predetermined distance from the image sensor is changed, and biometric information of the user is acquired based on reflected light of light emitted from the display, wherein the reflected light is acquired using the image sensor.

Description

Technical Field

[0001] The present invention generally relates to an electronic device and method for acquiring biometric information using light from a display. Background Art

[0002] The electronic device can use the optical sensor to obtain the user's biometric information. In order to obtain the user's biometric information, the electronic device needs a light emitting device and an optical sensor. Summary of the invention

[0003] [Technical issues]

[0004] Traditionally, electronic devices may require dedicated light emitting devices and light sensors in order to provide users with various functions associated with biometric information. In addition, since the light emitting devices and light sensors are exposed on the surface of the housing of the electronic device, the design of the electronic device may be limited, increasing the cost of purchasing and installing the light emitting devices and light sensors.

[0005] Therefore, research on acquisition of biometric information without requiring a dedicated light emitting device and a light sensor has been conducted.

[0006] [Technical solution]

[0007] According to an embodiment, the electronic device may include a shell, a display exposed through at least a portion of a first surface of the shell, an image sensor exposed through at least a portion of the first surface of the shell, a processor operably connected to the display and the image sensor, and a memory operably connected to the processor, and the memory may store instructions that, when executed, cause the processor to detect a touch of a user's finger on the image sensor while displaying a user interface using the display, and in response to detecting the touch, change the user interface in an area of ​​the display within a predetermined distance from the image sensor while the touch of the finger remains on the image sensor, and obtain biometric information of the user based on reflected light of light emitted from the display, wherein the reflected light is obtained using the image sensor.

[0008] According to an embodiment, an operating method of an electronic device including a shell may include, while displaying a user interface using a display exposed through at least a portion of a first surface of the shell, detecting a touch of a user's finger on an image sensor exposed through at least a portion of the first surface of the shell, in response to detecting the touch, while the touch of the finger remains on the image sensor, changing the user interface in an area of ​​the display within a predetermined distance from the image sensor, and acquiring biometric information of the user based on reflected light of light emitted from the display, wherein the reflected light is acquired using the image sensor.

[0009] According to an embodiment, an electronic device may include a shell, a display exposed through at least a portion of a first surface of the shell, an image sensor arranged in a display area of ​​the display when the first surface of the shell is viewed, a processor operably connected to the display and the image sensor, and a memory operably connected to the processor, and the memory may store instructions, which when executed cause the processor to display an indicator indicating a preset touch request area on a user interface displayed on the display, after displaying the indicator in the user interface, detect a touch of a user's finger in at least a portion of the touch request area, in response to detecting the touch, control the display to emit light of a preset intensity in a preset second area surrounding the image sensor when viewed in a front view of the shell, and obtain biometric information of the user based on reflected light of the light of the predetermined intensity emitted from the display, wherein the reflected light is obtained using the image sensor.

[0010] The following detailed description in conjunction with the accompanying drawings discloses embodiments of the present disclosure, and other aspects, advantages, and salient features of the present disclosure will become apparent to those skilled in the art.

[0011] [Beneficial Effects of the Invention]

[0012] Electronic devices (e.g. FIG. 4A to FIG. 4C The electronic device 401) and the operating method thereof can provide the user with various functions associated with biometric information acquisition without a dedicated light emitting device and an optical sensor. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The above and other aspects, features and advantages of certain embodiments of the present disclosure will become more apparent from the following description in conjunction with the accompanying drawings, in which:

[0014] Figure 1 A block diagram of an electronic device in a network environment according to an embodiment is shown;

[0015] Figure 2 shows a block diagram of a display device according to an embodiment;

[0016] Figure 3 shows a block diagram of a camera module according to an embodiment;

[0017] Figure 4A is a view showing an electronic device according to an embodiment;

[0018] Figure 4B is a view showing an electronic device according to an embodiment;

[0019] Figure 4C is a view showing an electronic device according to an embodiment;

[0020] Figure 5Ais a view showing measurement of biometric information using an electronic device according to an embodiment;

[0021] Figure 5B is a view showing measurement of biometric information using an electronic device according to an embodiment;

[0022] Figure 5C is a view showing measurement of biometric information using an electronic device according to an embodiment;

[0023] Figure 5D is a view showing measurement of biometric information using an electronic device according to an embodiment;

[0024] Figure 5E is a view showing measurement of biometric information using an electronic device according to an embodiment;

[0025] Fig. 6A is a view showing an example of an emission area of ​​an electronic device according to an embodiment;

[0026] Figure 6B is a view showing an example of an emission area of ​​an electronic device according to an embodiment;

[0027] Figure 6C is a view showing an example of an emission area of ​​an electronic device according to an embodiment;

[0028] Fig.6D is a view showing an example of an emission area of ​​an electronic device according to an embodiment;

[0029] Fig. 6E is a view showing an example of an emission area of ​​an electronic device according to an embodiment;

[0030] Fig. 6F is a view showing an example of an emission area of ​​an electronic device according to an embodiment;

[0031] Fig. 7A is a view showing measurement of biometric information using an electronic device according to an embodiment;

[0032] Figure 7B is a view showing measurement of biometric information using an electronic device according to an embodiment;

[0033] Figure 7C is a view showing measurement of biometric information using an electronic device according to an embodiment;

[0034] Fig.7D is a view showing measurement of biometric information using an electronic device according to an embodiment;

[0035] Fig. 7Eis a view showing measurement of biometric information using an electronic device according to an embodiment;

[0036] Fig. 8A is a view showing an example of an emission area of ​​an electronic device according to an embodiment;

[0037] Figure 8B is a view showing an example of an emission area of ​​an electronic device according to an embodiment;

[0038] Figure 8C is a view showing an example of an emission area of ​​an electronic device according to an embodiment;

[0039] Fig.8D is a view showing an example of an emission area of ​​an electronic device according to an embodiment;

[0040] Fig. 8E is a view showing an example of an emission area of ​​an electronic device according to an embodiment;

[0041] Fig.8F is a view showing an example of an emission area of ​​an electronic device according to an embodiment;

[0042] Figure 8G is a view showing an example of an emission area of ​​an electronic device according to an embodiment;

[0043] Fig.9A is a view showing measurement of biometric information using an electronic device according to an embodiment;

[0044] Fig. 9B is a view showing measurement of biometric information using an electronic device according to an embodiment;

[0045] Fig. 9C is a view showing measurement of biometric information using an electronic device according to an embodiment;

[0046] Fig.9D is a view showing measurement of biometric information using an electronic device according to an embodiment;

[0047] Fig.9E is a view showing measurement of biometric information using an electronic device according to an embodiment;

[0048] Fig. 10A is a view showing an example of an emission area of ​​an electronic device according to an embodiment;

[0049] Fig. 10B is a view showing an example of an emission area of ​​an electronic device according to an embodiment;

[0050] Fig. 10Cis a view showing an example of an emission area of ​​an electronic device according to an embodiment;

[0051] Fig.11 a flowchart showing the operation of the electronic device according to an embodiment; and

[0052] Fig.12 A flowchart illustrating the operation of the electronic device according to the embodiment.

[0053] Throughout the drawings, like reference numerals will be understood to refer to like parts, components and structures. DETAILED DESCRIPTION

[0054] Figure 1 1 is a block diagram showing an electronic device 101 in a network environment 100 according to an embodiment. Figure 1 , the electronic device 101 in the network environment 100 may communicate with the electronic device 102 via the first network 198 (e.g., a short-range wireless communication network), or communicate with the electronic device 104 or the server 108 via the second network 199 (e.g., a long-range wireless communication network). According to an embodiment, the electronic device 101 may communicate with the electronic device 104 via the server 108. According to an embodiment, the electronic device 101 may include a processor 120, a memory 130, an input device 150, a sound output device 155, a display device 160, an audio module 170, a sensor module 176, an interface 177, a haptic module 179, a camera module 180, a power management module 188, a battery 189, a communication module 190, a user identification module (SIM) 196, or an antenna module 197. In some embodiments, at least one of the components (e.g., the display device 160 or the camera module 180) may be omitted from the electronic device 101, or one or more other components may be added to the electronic device 101. In some embodiments, some of the components may be implemented as a single integrated circuit.For example, the sensor module 176 (eg, a fingerprint sensor, an iris sensor, or an illumination sensor) may be implemented to be embedded in the display device 160 (eg, a display).

[0055] The processor 120 may run, for example, software (e.g., program 140) to control at least one other component (e.g., hardware component or software component) of the electronic device 101 connected to the processor 120, and may perform various data processing or calculations. According to one embodiment, as at least part of the data processing or calculation, the processor 120 may load a command or data received from another component (e.g., sensor module 176 or communication module 190) into the volatile memory 132, process the command or data stored in the volatile memory 132, and store the resultant data in the non-volatile memory 134. According to an embodiment, the processor 120 may include a main processor 121 (e.g., a central processing unit (CPU) or an application processor (AP)) and an auxiliary processor 123 (e.g., a graphics processing unit (GPU), an image signal processor (ISP), a sensor hub processor, or a communication processor (CP)) that is independent of or combined with the main processor 121 in operation. Additionally or alternatively, the auxiliary processor 123 may be adapted to consume less power than the main processor 121, or adapted to be specifically used for a specified function. The auxiliary processor 123 may be implemented separately from the main processor 121 , or as part of the main processor 121 .

[0056] When the main processor 121 is in an inactive (e.g., sleep) state, the auxiliary processor 123 may control at least some of the functions or states related to at least one component (e.g., display device 160, sensor module 176, or communication module 190) among the components of the electronic device 101 (not the main processor 121), or when the main processor 121 is in an active state (e.g., running an application), the auxiliary processor 123 may control at least some of the functions or states related to at least one component (e.g., display device 160, sensor module 176, or communication module 190) among the components of the electronic device 101 together with the main processor 121. According to an embodiment, the auxiliary processor 123 (e.g., an image signal processor or a communication processor) may be implemented as part of another component (e.g., camera module 180 or communication module 190) that is functionally related to the auxiliary processor 123.

[0057] The memory 130 may store various data used by at least one component of the electronic device 101 (e.g., the processor 120 or the sensor module 176). The various data may include, for example, software (e.g., the program 140) and input data or output data for commands related thereto. The memory 130 may include a volatile memory 132 or a nonvolatile memory 134.

[0058] The program 140 may be stored as software in the memory 130 , and may include, for example, an operating system (OS) 142 , middleware 144 , or applications 146 .

[0059] The input device 150 may receive commands or data from outside the electronic device 101 (e.g., a user) to be used by other components of the electronic device 101 (e.g., the processor 120). The input device 150 may include, for example, a microphone, a mouse, a keyboard, or a digital pen (e.g., a stylus).

[0060] The sound output device 155 can output a sound signal to the outside of the electronic device 101. The sound output device 155 may include, for example, a speaker or a receiver. The speaker can be used for general purposes such as playing multimedia or playing records, and the receiver can be used for incoming calls. Depending on the embodiment, the receiver can be implemented as a separate part from the speaker, or as a part of the speaker.

[0061] The display device 160 may visually provide information to the outside of the electronic device 101 (e.g., a user). The display device 160 may include, for example, a display, a holographic device, or a projector, and a control circuit for controlling a corresponding one of the display, the holographic device, and the projector. According to an embodiment, the display device 160 may include a touch circuit adapted to detect a touch or a sensor circuit (e.g., a pressure sensor) adapted to measure the strength of a force caused by a touch.

[0062] The audio module 170 may convert sound into an electrical signal, or vice versa. According to an embodiment, the audio module 170 may obtain sound via the input device 150, or output sound via the sound output device 155 or an earphone of an external electronic device (e.g., electronic device 102) directly (e.g., wired) or wirelessly connected to the electronic device 101.

[0063] The sensor module 176 may detect an operating state (e.g., power or temperature) of the electronic device 101 or an environmental state (e.g., a state of a user) outside the electronic device 101, and then generate an electrical signal or data value corresponding to the detected state. According to an embodiment, the sensor module 176 may include, for example, a gesture sensor, a gyro sensor, an atmospheric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an infrared (IR) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illumination sensor.

[0064] The interface 177 may support one or more specific protocols to be used to connect the electronic device 101 directly (e.g., wired) or wirelessly to an external electronic device (e.g., the electronic device 102). According to an embodiment, the interface 177 may include, for example, a high-definition multimedia interface (HDMI), a universal serial bus (USB) interface, a secure digital (SD) card interface, or an audio interface.

[0065] The connection end 178 may include a connector, wherein the electronic device 101 can be physically connected to an external electronic device (e.g., the electronic device 102) via the connector. According to an embodiment, the connection end 178 may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0066] The haptic module 179 may convert the electrical signal into mechanical stimulation (eg, vibration or motion) or electrical stimulation that can be recognized by the user via his sense of touch or kinesthetic sense. According to an embodiment, the haptic module 179 may include, for example, a motor, a piezoelectric element, or an electrical stimulator.

[0067] The camera module 180 may capture still images or moving images. According to an embodiment, the camera module 180 may include one or more lenses, an image sensor, an image signal processor, or a flash.

[0068] The power management module 188 may manage power supply to the electronic device 101. According to an embodiment, the power management module 188 may be implemented as, for example, at least a part of a power management integrated circuit (PMIC).

[0069] The battery 189 may power at least one component of the electronic device 101. According to an embodiment, the battery 189 may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0070] The communication module 190 may support establishing a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device 101 and an external electronic device (e.g., electronic device 102, electronic device 104, or server 108), and perform communication via the established communication channel. The communication module 190 may include one or more communication processors capable of operating independently from the processor 120 (e.g., an application processor (AP)) and supporting direct (e.g., wired) communication or wireless communication. According to an embodiment, the communication module 190 may include a wireless communication module 192 (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module 194 (e.g., a local area network (LAN) communication module or a power line communication (PLC) module). A corresponding one of these communication modules can communicate with an external electronic device via a first network 198 (e.g., a short-range communication network such as Bluetooth, Wireless Fidelity (Wi-Fi) Direct, or Infrared Data Association (IrDA)) or a second network 199 (e.g., a long-range communication network such as a cellular network, the Internet, or a computer network (e.g., a LAN or a wide area network (WAN))). These various types of communication modules can be implemented as a single component (e.g., a single chip), or these various types of communication modules can be implemented as multiple components separated from each other (e.g., multiple chips). The wireless communication module 192 can identify and authenticate the electronic device 101 in a communication network (such as the first network 198 or the second network 199) using user information (e.g., an International Mobile Subscriber Identity (IMSI)) stored in the user identification module 196.

[0071] The antenna module 197 may transmit or receive a signal or power to or from the outside of the electronic device 101 (e.g., an external electronic device). According to an embodiment, the antenna module 197 may include an antenna including a radiating element formed of a conductive material or a conductive pattern formed in or on a substrate (e.g., a printed circuit board (PCB)). According to an embodiment, the antenna module 197 may include a plurality of antennas. In this case, at least one antenna suitable for a communication scheme used in a communication network (such as the first network 198 or the second network 199) may be selected from the plurality of antennas by, for example, the communication module 190 (e.g., the wireless communication module 192). A signal or power may then be transmitted or received between the communication module 190 and the external electronic device via the selected at least one antenna. According to an embodiment, another component (e.g., a radio frequency integrated circuit (RFIC)) other than the radiating element may be additionally formed as a part of the antenna module 197.

[0072] At least some of the above components may be connected to each other via an inter-peripheral communication scheme (e.g., a bus, a general purpose input output (GPIO), a serial peripheral interface (SPI), or a mobile industry processor interface (MIPI)) and communicatively transmit signals (e.g., commands or data) therebetween.

[0073] According to an embodiment, a command or data may be sent or received between the electronic device 101 and the external electronic device 104 via the server 108 connected to the second network 199. Each of the electronic device 102 and the electronic device 104 may be a device of the same type as the electronic device 101, or a device of a different type from the electronic device 101. According to an embodiment, all or some operations to be executed in the electronic device 101 may be executed in one or more of the external electronic device 102, the external electronic device 104, or the server 108. For example, if the electronic device 101 should automatically execute a function or service or should execute a function or service in response to a request from a user or another device, the electronic device 101 may request the one or more external electronic devices to execute at least part of the function or service instead of executing the function or service, or the electronic device 101 may request the one or more external electronic devices to execute at least part of the function or service in addition to executing the function or service. The one or more external electronic devices that receive the request may execute the requested at least part of the function or service, or execute another function or another service related to the request, and transmit the result of the execution to the electronic device 101. The electronic device 101 may provide the result as at least a partial reply to the request with or without further processing the result. To this end, for example, cloud computing technology, distributed computing technology, or client-server computing technology may be used.

[0074] Figure 2 2 is a block diagram showing a display device 160 according to an embodiment. Figure 2, the display device 160 may include a display 210 and a display driver integrated circuit (DDI) 230 for controlling the display 210. The DDI 230 may include an interface module 231, a memory 233 (e.g., a buffer memory), an image processing module 235, or a mapping module 237. The DDI 230 may receive image information including image data or an image control signal corresponding to a command for controlling image data from another component of the electronic device 101 via the interface module 231. For example, according to an embodiment, the image information may be received from the processor 120 (e.g., the main processor 121 (e.g., an application processor)) or the auxiliary processor 123 (e.g., a graphics processing unit), wherein the auxiliary processor 123 operates independently of the function of the main processor 121. The DDI 230 may communicate with, for example, the touch circuit 250 or the sensor module 176 via the interface module 231. The DDI 230 may also store at least a portion of the received image information in the memory 233, for example, frame by frame.

[0075] The image processing module 235 may perform pre-processing or post-processing (e.g., adjustment of resolution, brightness, or size) on at least a portion of the image data. Depending on the embodiment, for example, the pre-processing or post-processing may be performed based at least in part on one or more features of the image data or one or more features of the display 210.

[0076] The mapping module 237 may generate a voltage value or a current value corresponding to the image data pre-processed or post-processed by the image processing module 235. According to an embodiment, for example, the generation of the voltage value or the current value may be performed based at least in part on one or more attributes of the pixel (e.g., an array of pixels (such as RGB stripes or a pentile structure) or a size of each sub-pixel). For example, at least some pixels of the display 210 may be driven based at least in part on the voltage value or the current value, so that visual information (e.g., text, an image, or an icon) corresponding to the image data may be displayed via the display 210.

[0077] According to an embodiment, the display device 160 may further include a touch circuit 250. The touch circuit 250 may include a touch sensor 251 and a touch sensor IC 253 for controlling the touch sensor 251. The touch sensor IC 253 may control the touch sensor 251 to sense a touch input or a hovering input for a specific position on the display 210. To this end, for example, the touch sensor 251 may detect (e.g., measure) a signal (e.g., voltage, light amount, resistance, or one or more charge amounts) corresponding to a specific position on the display 210. The touch circuit 250 may provide input information (e.g., position, area, pressure, or time) indicating a touch input or a hovering input detected via the touch sensor 251 to the processor 120. According to an embodiment, at least a portion of the touch circuit 250 (e.g., the touch sensor IC 253) may be formed as a part of the display 210 or the DDI 230, or as a part of another component (e.g., the auxiliary processor 123) located outside the display device 160.

[0078] According to an embodiment, the display device 160 may further include at least one sensor (e.g., a fingerprint sensor, an iris sensor, a pressure sensor, or an illumination sensor) of the sensor module 176 or a control circuit for the at least one sensor. In such a case, the at least one sensor or the control circuit for the at least one sensor may be embedded in a portion of a component (e.g., the display 210, the DDI 230, or the touch circuit 250) of the display device 160. For example, when the sensor module 176 embedded in the display device 160 includes a biosensor (e.g., a fingerprint sensor), the biosensor may acquire biometric information (e.g., a fingerprint image) corresponding to a touch input received via a portion of the display 210. As another example, when the sensor module 176 embedded in the display device 160 includes a pressure sensor, the pressure sensor may acquire pressure information corresponding to a touch input received via a partial area or the entire area of ​​the display 210. According to an embodiment, the touch sensor 251 or the sensor module 176 may be arranged between pixels in a pixel layer of the display 210, or above or below the pixel layer.

[0079] Figure 3 3 is a block diagram showing a camera module 180 according to an embodiment. Figure 3, the camera module 180 may include a lens assembly 310, a flash 320, an image sensor 330, an image stabilizer 340, a memory 350 (e.g., a buffer memory), or an image signal processor 360. The lens assembly 310 may collect light emitted or reflected from an object whose image is to be captured. The lens assembly 310 may include one or more lenses. According to an embodiment, the camera module 180 may include a plurality of lens assemblies 310. In this case, the camera module 180 may form, for example, a dual camera, a 360-degree camera, or a spherical camera. Some of the plurality of lens assemblies 310 may have the same lens properties (e.g., viewing angle, focal length, autofocus, f-number, or optical zoom), or at least one lens assembly may have one or more lens properties different from those of another lens assembly. The lens assembly 310 may include, for example, a wide-angle lens or a telephoto lens.

[0080] The flashlight 320 may emit light, wherein the emitted light is used to enhance the light reflected from the object. According to an embodiment, the flashlight 320 may include one or more light emitting diodes (LEDs) (e.g., red, green, blue (RGB) LEDs, white LEDs, infrared (IR) LEDs, or ultraviolet (UV) LEDs) or xenon lamps. The image sensor 330 may acquire an image corresponding to the object by converting light emitted or reflected from the object and transmitted through the lens assembly 310 into an electrical signal. According to an embodiment, the image sensor 330 may include one image sensor selected from a plurality of image sensors having different properties (e.g., an RGB sensor, a black and white (BW) sensor, an IR sensor, or a UV sensor), a plurality of image sensors having the same properties, or a plurality of image sensors having different properties. Each image sensor included in the image sensor 330 may be implemented using, for example, a charge coupled device (CCD) sensor or a complementary metal oxide semiconductor (CMOS) sensor.

[0081] The image stabilizer 340 may move the image sensor 330 or at least one lens included in the lens assembly 310 in a specific direction, or control an operational property of the image sensor 330 (e.g., adjust the readout timing) in response to the movement of the camera module 180 or the electronic device 101 including the camera module 180. In this way, at least a portion of the negative effects (e.g., image blur) generated due to the movement of the image being captured is allowed to be compensated. According to an embodiment, the image stabilizer 340 may sense such movement of the camera module 180 or the electronic device 101 using a gyro sensor (not shown) or an acceleration sensor (not shown) arranged inside or outside the camera module 180. According to an embodiment, the image stabilizer 340 may be implemented as, for example, an optical image stabilizer.

[0082] The memory 350 may at least temporarily store at least a portion of an image acquired via the image sensor 330 for subsequent image processing tasks. For example, if multiple images are captured quickly or image capture is delayed due to shutter lag, the acquired original image (e.g., Bayer pattern image, high-resolution image) may be stored in the memory 350, and its corresponding copy image (e.g., low-resolution image) may be previewed via the display device 160. Then, if a specified condition is met (e.g., by a user's input or a system command), at least a portion of the original image stored in the memory 350 may be acquired and processed by, for example, the image signal processor 360. According to an embodiment, the memory 350 may be configured as at least a portion of the memory 130, or the memory 350 may be configured as a separate memory that operates independently of the memory 130.

[0083] The image signal processor 360 may perform one or more image processing on an image acquired via the image sensor 330 or an image stored in the memory 350. The one or more image processing may include, for example, depth map generation, three-dimensional (3D) modeling, panorama generation, feature point extraction, image synthesis, or image compensation (e.g., noise reduction, resolution adjustment, brightness adjustment, blurring, sharpening, or softening). Additionally or alternatively, the image signal processor 360 may perform control (e.g., exposure time control or readout timing control) on at least one of the components included in the camera module 180 (e.g., image sensor 330). The image processed by the image signal processor 360 may be stored back to the memory 350 for further processing, or the image may be provided to an external component outside the camera module 180 (e.g., memory 130, display device 160, electronic device 102, electronic device 104, or server 108). According to an embodiment, the image signal processor 360 may be configured as at least a part of the processor 120, or the image signal processor 360 may be configured as a separate processor that operates independently of the processor 120. If the image signal processor 360 is configured as a separate processor from the processor 120, at least one image processed by the image signal processor 360 may be displayed as it is by the processor 120 via the display device 160, or may be displayed after being further processed.

[0084] According to an embodiment, the electronic device 101 may include a plurality of camera modules 180 having different properties or functions. In this case, at least one camera module 180 of the plurality of camera modules 180 may form, for example, a wide-angle camera, and at least another camera module 180 of the plurality of camera modules 180 may form a telephoto camera. Similarly, at least one camera module 180 of the plurality of camera modules 180 may form, for example, a front camera, and at least another camera module 180 of the plurality of camera modules 180 may form a rear camera.

[0085] The electronic device according to the embodiment may be one of various types of electronic devices. The electronic device may include, for example, a portable communication device (e.g., a smart phone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, a household appliance, etc., without limitation. According to an embodiment of the present disclosure, the electronic device is not limited to those electronic devices described above.

[0086] It should be understood that the embodiments of the present disclosure and the terms used therein are not intended to limit the technical features set forth herein to specific embodiments, but include various changes, equivalent forms or alternative forms for corresponding embodiments. For the description of the accompanying drawings, similar reference numerals may be used to refer to similar or related elements. It will be understood that the nouns in the singular form corresponding to the term may include one or more things unless the relevant context clearly indicates otherwise. As used herein, each of the phrases such as "A or B", "at least one of A and B", "at least one of A or B", "A, B or C", "at least one of A, B and C" and "at least one of A, B or C" may include any one or all possible combinations of the items listed together with the corresponding one of the multiple phrases. As used herein, terms such as "1st" and "2nd" or "first" and "second" may be used to simply distinguish the corresponding component from another component, and do not limit the component in other aspects (e.g., importance or order). It will be understood that if an element (e.g., a first element) is referred to as being “coupled with another element (e.g., a second element)”, “coupled to another element (e.g., the second element)”, “connected with another element (e.g., the second element)” or “connected to another element (e.g., the second element)” when the terms “operably” or “communicatively” are used or when the terms “operably” or “communicatively” are not used, the element may be directly (e.g., wired) connected to the other element, wirelessly connected to the other element, or connected to the other element via a third element.

[0087] As used herein, the term "module" may include a unit implemented in hardware, software, firmware, or any combination thereof, and may be used interchangeably with other terms (e.g., "logic," "logic block," "portion," or "circuit"). A module may be a single integrated component adapted to perform one or more functions or a minimum unit or portion of the single integrated component. For example, according to an embodiment, a module may be implemented in the form of an application specific integrated circuit (ASIC).

[0088] The embodiments described herein may be implemented as software (e.g., program 140) including one or more instructions stored in a storage medium (e.g., internal memory 136 or external memory 138) that can be read by a machine (e.g., electronic device 101). For example, under the control of a processor, a processor (e.g., processor 120) of the machine (e.g., electronic device 101) may call at least one of the one or more instructions stored in the storage medium and execute the at least one instruction with or without the use of one or more other components. This enables the machine to operate to perform at least one function according to the at least one instruction called. The one or more instructions may include code generated by a compiler or code that can be run by an interpreter. A machine-readable storage medium may be provided in the form of a non-transitory storage medium. Among them, the term "non-transitory" only means that the storage medium is a tangible device and does not include a signal (e.g., an electromagnetic wave), but the term does not distinguish between data being semi-permanently stored in a storage medium and data being temporarily stored in a storage medium.

[0089] According to an embodiment, the method according to an embodiment of the present disclosure may be included and provided in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be released in the form of a machine-readable storage medium (e.g., a compact disk read-only memory (CD-ROM)), or may be downloaded via an application store (e.g., Play Store TM ) The computer program product may be published (e.g., downloaded or uploaded) online, or the computer program product may be distributed (e.g., downloaded or uploaded) directly between two user devices (e.g., smart phones). If published online, at least part of the computer program product may be temporarily generated, or at least part of the computer program product may be at least temporarily stored in a machine-readable storage medium (such as a memory of a manufacturer's server, an application store's server, or a forwarding server).

[0090] According to an embodiment, each component (e.g., module or program) in the above-mentioned components may include a single entity or multiple entities. According to an embodiment, one or more components in the above-mentioned components may be omitted, or one or more other components may be added. Alternatively or additionally, multiple components (e.g., module or program) may be integrated into a single component. In this case, according to an embodiment, the integrated component may still perform the one or more functions of each component in the multiple components in the same or similar manner as a corresponding component in the multiple components before integration. According to an embodiment, the operations performed by a module, program or another component may be performed sequentially, in parallel, repeatedly or in a heuristic manner, or one or more operations in the operations may be run or omitted in different orders, or one or more other operations may be added.

[0091] Figure 4A , Figure 4B and Figure 4C is a view showing an electronic device 401 according to an embodiment. Figure 4A An electronic device 401 according to an embodiment is shown. Figure 4B 4 is a view showing when the electronic device 401 according to the embodiment acquires biometric information using light of the display 410.

[0092] Figure 4C A side view of an electronic device 401 according to an embodiment is shown, which is along Figure 4B In the embodiment, FIG. 4A to FIG. 4C The electronic device 401 may correspond to Figure 1 In the embodiment, reference will be made to the electronic device 101. Figure 1 The configuration of the electronic device 101 is described FIG. 4A to FIG. 4C Electronic device 401.

[0093] Reference Figure 4A In an embodiment, the electronic device 401 may include a display 410 and / or a camera 420. In an embodiment, the display 410 and the camera 420 may correspond to Figure 1 In an embodiment, the electronic device 401 may further include a processor (eg, Figure 1Processor 120). Processor 120 may include a microprocessor or any suitable type of processing circuit, such as one or more general-purpose processors (e.g., ARM-based processors), digital signal processors (DSPs), programmable logic devices (PLDs), application-specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), graphics processing units (GPUs), video card controllers, etc. In addition, it can be recognized that when a general-purpose computer accesses the code for implementing the processing shown herein, the execution of the code converts the general-purpose computer into a special-purpose computer for performing the processing shown herein. Certain functions and steps provided in the accompanying drawings may be implemented in hardware, software, or a combination of both, and may be executed in whole or in part within the programming instructions of a computer. Unless an element is expressly stated using the phrase "means for...", a claim element should not be interpreted under the provisions of 35 U.S.C. §112(f). In addition, the skilled person believes and understands that in the disclosure claimed for protection, a "processor" or "microprocessor" may be hardware. Under the broadest reasonable interpretation, the appended claims are statutory subject matter consistent with 35 U.S.C. §101.

[0094] Reference Figure 4A In an embodiment, the display 410 may be exposed through, for example, a majority of the front plate 430. In an embodiment, a recess or opening may be formed in a portion of the screen display area of ​​the display 410, and another electronic component, such as a camera 420, a proximity sensor (not shown), and / or a light sensor (not shown) may be aligned with the recess or opening.

[0095] In an embodiment, a processor (e.g., Figure 1 The processor 120 of the embodiment may display a first user interface on the display 410 to receive input for obtaining biometric information of the user. In an embodiment, during the operation of the biometric information measurement application, the first user interface may receive input (e.g., touch input) to select the type of user biometric information to be measured. In an embodiment, the processor 120 may identify the type of user biometric information to be measured based on the input received through the first user interface. In an embodiment, the first user interface may be generated by another running application that is not a biometric information measurement application. In an embodiment, the input for obtaining user biometric information may include a touch covering the camera 420 with the user's finger 402. In an embodiment, the biometric information may include, but is not limited to, heart rate, oxygen saturation, stress index, blood pressure, blood sugar, tissue water content, tissue dehydration, fingerprints, or a combination thereof.

[0096] Reference Figure 4B and Figure 4CIn an embodiment, the processor 120 may receive an input for acquiring biometric information of the user through the display 410. In an embodiment, the processor 120 may detect a touch of the user's finger 402 covering the camera 420 on the display 410.

[0097] In an embodiment, if a touch covering the camera 420 with the user's finger 402 is detected, the processor 120 may display a second user interface on the display 410. In an embodiment, the second user interface may be associated with the measurement of the user's biometric information during the execution of the biometric information measurement application.

[0098] In an embodiment, during display of the second user interface, the processor 120 may set the emission area 450 in response to detecting the touch area 440 originating from the touch covering the camera 420 with the user's finger 402, and control the display 410 to emit light 461 and 463 from the set emission area 450. In an embodiment, the touch area 440 may be detected by using a touch sensor (not shown) and / or a pressure sensor (not shown) of the display 410. In an embodiment, the intensity of the light 461 and 463 emitted from the emission area 450 may be higher than the intensity of the light emitted from other areas of the second user interface. In an embodiment, the intensity of the light 461 and 463 emitted from the emission area 450 may be, for example, higher than the intensity of the light originating from the display 410 when the first user interface is displayed. Alternatively, the intensity of the light 461 and 463 emitted from the emission area 450 may be higher than or equal to the intensity of the light originating from the display 410 when the first user interface is displayed. In an embodiment, when the first user interface is displayed, light may be emitted from the display 410 at a first intensity, and lights 461 and 463 may be emitted from the emission area 450 at a second intensity higher than the first intensity.

[0099] In an embodiment, the processor 120 may set the emission area 450 within the touch area 440 corresponding to the touch input covering the camera 420 with the finger 402. In an embodiment, the processor 120 may set the edge of the emission area 450 within a set distance from the camera 420 based on the touch area 440 covering the camera 420 with the finger 402. In an embodiment, the processor 120 may set the size of the emission area 450 to correspond to the touch area 440. In an embodiment, the boundary of the emission area 450 may be within the boundary of the touch area 440. In an embodiment, the processor 120 may change the size and / or boundary of the emission area 450 as the touch area 440 changes.

[0100] In an embodiment, the processor 120 may set the emission area 450 based on the type of biometric information to be acquired. In an embodiment, the processor 120 may adjust the shape of the emission area 450 and / or the position of the emission area 450 on the display 410 in response to the type of biometric information to be acquired. In an embodiment, the processor 120 may store the emission area 450 in a memory (e.g., Figure 1 The information related to the shape of the emission area 450, the position of the emission area 450 on the display 410, or a combination thereof for each type of biometric information to be acquired is preset in the memory 130 of the display 410. In an embodiment, the shape of the emission area 450 may include a polygon, a circle, an ellipse, or a combination thereof. In an embodiment, the position of the emission area 450 may include a position on the display 410 at a predetermined distance from the camera 420, a position around the camera 420, or a combination thereof.

[0101] In an embodiment, the processor 120 may adjust the color of the lights 461 and 463, the intensity of the lights 461 and 463, or a combination thereof in response to the type of biometric information to be acquired. In an embodiment, the processor 120 may adjust the color of the lights 461 and 463, the intensity of the lights 461 and 463, or a combination thereof in response to the touch area of ​​the finger 402. In an embodiment, the processor 120 may adjust the intensity of the lights 461 and 463 to increase as the touch area of ​​the finger 402 becomes narrower. In an embodiment, the memory (e.g., Figure 1 Information related to the color of light 461 and 463, the intensity of light 461 and 463, or a combination thereof for each type of biometric information to be acquired is preset in the memory 130).

[0102] Reference Figure 4B and Figure 4C In an embodiment, the processor 120 may receive reflected lights 471 and 473 of the lights 461 and 463 emitted from the emission area 450 through the camera 420. In an embodiment, the processor 120 may acquire biometric information of the user based on the received reflected lights 471 and 473.

[0103] In an embodiment, the processor 120 may reset the emission area 450 based on the reflected lights 471 and 473. In an embodiment, if the asymmetry of the reference direction of the image obtained based on the reflected lights 471 and 473 is recognized, the processor 120 may reset the emission area 450 based on the degree of asymmetry of the reference direction of the image. In an embodiment, the processor 120 may reset the emission area 450 to generate a symmetrical image obtained based on the reflected lights 471 and 473. In an embodiment, the processor 120 may acquire the biometric information of the user based on the reset emission area.

[0104] In an embodiment, as an example, light 461 and 463 are lights of display 410, but light 461 and 463 may be emitted by a light source emitting infrared light (e.g., an infrared light emitting diode (LED)). In an embodiment, if light 461 and 463 are infrared lights, camera 420 may be a time of flight (TOF) sensor or a fingerprint sensor.

[0105] Figures 5A to 5E 5 is a diagram illustrating biometric information measurement using an electronic device 501 according to an embodiment. Figures 6A to 6F 601 according to an embodiment of the present invention. FIG. 5A to FIG. 5E The electronic device 501 or 6A to 6F The electronic device 601 may correspond to Figure 1 The electronic device 101 or FIG. 4A to FIG. 4C In the embodiment, reference is made to the electronic device 401. Figure 1 The configuration of the electronic device 101 is described Figures 5A to 5E and Figures 6A to 6F .

[0106] In the embodiments, reference Figures 5A to 5E , if the electronic device 501 is viewed from a first direction (e.g., the front direction of the electronic device 501), the camera 520 of the electronic device 501 may overlap with the display 510. In an embodiment, the overlapping area between the camera 520 of the electronic device 501 and the display 510 may not include the light emitting device of the display 510. In an embodiment, the display 510 may be exposed through, for example, a majority of the front plate 530.

[0107] refer to Figure 5A In an embodiment, a processor (e.g., processor 120) may display a first user interface 511 on a display 510. In an embodiment, during the execution of, for example, a biometric information measurement application, the first user interface 511 may receive an input (e.g., a touch input) to select the type of user biometric information to be measured. In an embodiment, the first user interface 511 may be generated by another executing application instead of the biometric information measurement application.

[0108] refer to Figure 5B In an embodiment, during displaying the first user interface 511 on the display 510, the processor 120 may recognize that a finger of the user (eg, Figure 4C The touch of the finger 402) covering the camera 520. In an embodiment, if the touch covering the camera 520 is recognized, the processor 120 may display a second user interface 513 different from the first user interface 511 on the display 510.

[0109] In an embodiment, the processor 120 may recognize a touch covering the camera 520 based on a touch area corresponding to the touch area 540 of the finger 402 detected by a touch sensor (not shown) of the display 510. In an embodiment, if the detected touch area surrounds the camera 520, the processor 120 may recognize that the touch covers the camera 520. In an embodiment, if the touch of the finger 402 is detected, the processor 120 may recognize a touch covering the camera 520 based on an image acquired through the camera 520. In an embodiment, in response to a touch input in a preset area of ​​the display 510 (e.g., a preset area surrounding the camera 520), the processor 120 may enable the camera 520 and recognize a touch covering the camera 520 based on an image acquired through the camera 520.

[0110] In an embodiment, if a touch covering the camera 520 is recognized, the processor 120 may recognize whether the camera 520 is completely covered by the finger 402. In an embodiment, the processor 120 may recognize whether the camera 520 is completely covered based on light acquired by an image sensor (not shown) of the camera 520. In an embodiment, if the light acquired by the image sensor (not shown) of the camera 520 includes light other than reflected light of light emitted from the display 510, the processor 120 may recognize that the camera 520 is not completely covered by the finger 402. In an embodiment, if the light acquired by the image sensor (not shown) of the camera 520 does not include light other than reflected light of light emitted from the display 510, the processor 120 may recognize that the camera 520 is completely covered by the finger 402.

[0111] like Figure 5C As shown, in an embodiment, if the camera 520 is completely covered by the finger 402, the processor 120 may display a second user interface 513 for biometric measurement. Figure 5D As shown, in an embodiment, if the camera 520 is not completely covered by the finger 402, the processor 120 may display a third user interface 515 including an indicator 514 that guides the user to re-touch the display 510 with his or her finger 402. In an embodiment, the second user interface 513 may be displayed when measuring the user's biometric information during the operation of the biometric information measurement application. In an embodiment, the third user interface 515 may indicate to the user that a re-touch is required in order to measure the biometric information.

[0112] refer to Figure 5CIn an embodiment, the processor 120 may display a second user interface 513 on the display 510. In an embodiment, the processor 120 may set a portion of the second user interface 513 as an emission area 550. In an embodiment, the processor 120 may set the emission area 550 based on the touch area 540 of the finger 402. In an embodiment, the processor 120 may set the emission area 550 so that its boundary is within the touch area 540 of the finger 402. In an embodiment, the processor 120 may set the emission area 550 to surround the camera 520. In an embodiment, the processor 120 may determine the shape, pattern, size, position, or a combination thereof of the emission area 550 based on the type of biometric information to be acquired, which may be selected by the user during the display of the first user interface 511.

[0113] In an embodiment, the processor 120 may emit a preset light (eg, Figure 4C In an embodiment, the processor 120 may determine the color, intensity, or combination of the lights 461 and 463 based on the type of biometric information to be acquired. In an embodiment, the processor 120 may determine the color, intensity, or combination of the lights 461 and 463 during display of the first user interface 511.

[0114] In an embodiment, the processor 120 may emit lights 461 and 463 in the emission area 550 on the display 510 , and measure biometric information of the user based on reflected lights 471 and 473 of the lights 461 and 463 acquired by the camera 520 .

[0115] Although in this example, Figure 5C The emission area 550 is shown in a ring shape, but the present disclosure is not limited thereto, and the emission area 550 may take various shapes. Fig. 6A In an embodiment, the electronic device 601 may include two or more emission areas 651 and 652 on the display 610, wherein the display 610 is exposed through a majority of the front plate 630 of the electronic device 601. In an embodiment, the emission areas 651 and 652 may have a circular shape together. In an embodiment, for example, the emission areas 651 and 652 may be symmetrically arranged around the center of the camera 620. Alternatively, the emission areas 651 and 652 may be asymmetrically arranged around the center of the camera 620.

[0116] In an embodiment, the processor 120 may control the emission areas 651 and 652 to be illuminated at different times. In an embodiment, the processor 120 may illuminate the emission area 651 for a first duration, and illuminate the emission area 652 for a second duration after the first duration expires. In an embodiment, the first duration and the second duration may be durations for biometric measurement, respectively.

[0117] In an embodiment, the processor 120 may control the emission areas 651 and 652 to be lit in different colors. In an embodiment, as an example, the processor 120 may emit light of a first color (e.g., red) from the emission area 651, and emit light of a second color (e.g., green) from the emission area 652. Alternatively, the processor 120 may control the emission areas 651 and 652 to be lit in the same color.

[0118] In an embodiment, the processor 120 may control to emit light of a first color (e.g., red) from the emission areas 651 and 652 for a first duration, and to emit light of a second color (e.g., green) from the emission areas 651 and 652 for a second duration. In an embodiment, the processor 120 may control the emission areas 651 and 652 to emit light of a color (e.g., yellow) mixed with the first color (e.g., red) and the second color (e.g., green).

[0119] 6B, in an embodiment, the emission area 653 of the display 610 exposed through the majority of the front plate 630 of the electronic device 601 may include two or more emission sub-areas 654 and 655. In an embodiment, in order to emit light in a preset pattern, the processor 120 may divide the emission area 653 into two or more emission sub-areas 654 and 655. In an embodiment, the processor 120 may control the display 610 to emit light corresponding to the preset pattern from the emission sub-areas 654 and 655.

[0120] In an embodiment, the processor 120 may control the emission sub-areas 654 and 655 to be lit at different times. In an embodiment, the processor 120 may control the emission sub-areas 654 and 655 to be lit in different colors.

[0121] refer to Figure 6CIn an embodiment, the emission areas 656 and 657 of the display 610 exposed through the majority of the front plate 630 of the electronic device 601 may include the emission area 656 and the emission area 657 directly adjacent to the camera 620, wherein the emission area 657 is interposed between the emission area 656 and the camera 620. In an embodiment, the processor 120 may control the emission areas 656 and 657 to be illuminated at different times. In an embodiment, the processor 120 may control the emission areas 656 and 657 to be illuminated in different colors. In an embodiment, the processor 120 may control the emission areas 656 and 657 so that the light source close to the camera 620 emits light of a relatively shorter wavelength. For example, the emission area 656 far from the camera 620 may emit light of a first color (e.g., red), and the emission area 657 close to the camera 620 may emit light of a second color (e.g., green).

[0122] In an embodiment, the processor 120 may control the emission areas 656 and 657 to be illuminated at different intensities. In an embodiment, the processor 120 may control the emission area 656 to be illuminated with a first intensity of light and the emission area 657 to be illuminated with a second intensity of light. In different embodiments, the first intensity may be higher or lower than the second intensity.

[0123] refer to Fig.6D In an embodiment, two or more emission areas 661 and 662 of the display 610 may be configured to be exposed through a majority of the front plate 630 of the electronic device 601. In an embodiment, the emission areas 661 and 662 together may form a quadrilateral shape.

[0124] refer to Fig. 6E In an embodiment, a quadrilateral emission area 663 of the display 610 exposed through a majority of the front plate 630 of the electronic device 601 may include two or more emission sub-areas 664 and 665 .

[0125] refer to Fig. 6F In an embodiment, the emission areas 666 and 667 of the display 610 of the front panel 630 of the electronic device 601 may include a quadrilateral emission area 667 directly adjacent to the camera 620 and a quadrilateral emission area 666, wherein the emission area 667 is inserted between the quadrilateral emission area 666 and the camera 620.

[0126] Emission areas 651, 652, 653, 656, 657, 661, 662, 663, 666, and 667 are shown here as circles or quadrilaterals. However, these are just examples, and the emission areas may be other shapes, such as polygons.

[0127] refer to Figure 5DIn an embodiment, if the processor 120 recognizes that the camera 520 is not completely covered by the finger 402, the processor 120 may display an indicator 514 on the third user interface 515, which guides the user to re-touch the display 510 with his or her finger 402. In an embodiment, as an example, the indicator 514 may be an image object having a shape for indicating the camera 520. The indicator 514 may include a phrase, an image, or a combination thereof for requesting a re-touch.

[0128] In an embodiment, the processor 120 may determine the position of the indicator 514 based on the position of light other than reflected light acquired by an image sensor (not shown) of the camera 520. The position of the light other than reflected light may be a position corresponding to an area exposed to light other than reflected light. In an embodiment, for example, if the position of the light other than reflected light is in a first direction from the center of the camera 520, the processor 120 may determine the position of the indicator 514 as a position at a set distance from a preset reference position in the first direction. In an embodiment, the processor 120 may determine the set distance in response to an area exposed to light other than reflected light on an image acquired by the image sensor (not shown) of the camera 520. In an embodiment, the processor 120 may determine the set distance in response to the width of the area exposed to light other than reflected light.

[0129] refer to Figure 5E In an embodiment, if the indicator 514 is displayed and then a touch covering the camera 520 is recognized, the processor 120 may recognize whether the camera 520 is completely covered by the finger 402. In an embodiment, if it is recognized that the camera 520 is completely covered by the re-touch of the finger 402, the processor 120 may store the position information of the indicator 514 in a memory (e.g., Figure 1 in the memory 130).

[0130] like Figure 5C As shown, in an embodiment, if the camera 520 is completely covered by the re-touch of the finger 402, the processor 120 may display the second user interface 513 for biometric measurement. Figure 5D As shown, in an embodiment, if the camera 520 is not completely covered by the re-touch of the finger 402 , the processor 120 may again display the third user interface 515 including the indicator 514 guiding the user to touch the display 510 again.

[0131] In an embodiment, the processor 120 may emit light 461 and 463 from the emission area of ​​the display 510 after the re-touch of the finger 402 during display of the second user interface 513, and measure the user's biometric information based on reflected light 471 and 473 of the light 461 and 463 acquired by the camera 520.

[0132] In an embodiment, during display of the third user interface 515, the processor 120 may re-display the indicator 514 requesting a re-touch of the finger 402. In an embodiment, the processor 120 may display the indicator 514 at a re-determined position based on the touch area 541 of the finger 402.

[0133] Figures 7A to 7E They are views respectively showing biometric information measurement using an electronic device 701 according to an embodiment. Figures 8A to 8G 7 and 8 respectively illustrate examples of emission areas of an electronic device 801 according to an embodiment. FIG. 8A to FIG. 8G The electronic device 801 may correspond to Figure 1 The electronic device 101 or FIG. 4A to FIG. 4C In the embodiment, by referring to Figure 1 8A to 8G and Figures 7A to 7E . In 8A to 8G and Figures 7A to 7E In the description of FIG. 5A to FIG. 5E and 6A to 6F Redundant description of .

[0134] If the user views the electronic device 701 in a first direction (eg, the front direction of the electronic device 701), then Figures 7A to 7E The display 710 and the camera 720 may overlap each other. In an embodiment, the light emitting device of the display 570 may be disposed in an overlapping area of ​​the camera 720 of the electronic device 701 in the display 710. In an embodiment, the display 710 may be exposed through, for example, a majority of the front plate 730.

[0135] In an embodiment, if the camera 720 is disabled, the user may not be able to see the camera 720. In an embodiment, if the camera 720 is disabled, the light emitting device of the overlapping area of ​​the camera 720 in the display 710 may emit light. Alternatively, if the camera 720 is enabled, the user may be able to see the camera 720. In an embodiment, if the camera 720 is enabled, the light emitting device of the overlapping area of ​​the camera 720 in the display 710 may not emit light.

[0136] refer to Fig. 7A In an embodiment, a processor (eg, processor 120 ) may display a first user interface 711 on a display 710 .

[0137] refer to Figure 7B In an embodiment, during displaying the first user interface 711 on the display 710, the processor 120 may recognize that a finger of the user (eg, Figure 4C The touch of the finger 402) covering the camera 720. In an embodiment, if the touch covering the camera 720 is recognized, the processor 120 may display a second user interface 713 different from the first user interface 711 on the display 710.

[0138] In an embodiment, the processor 120 may recognize a touch covering the camera 720 based on a touch area detected by a touch sensor (not shown) of the display 710 corresponding to the touch area 740 of the finger 402. In an embodiment, if the detected touch area corresponds to an overlapping area of ​​the camera 720 in the display 710, the processor 120 may recognize a touch covering the camera 720. In an embodiment, if the detected touch area surrounds the camera 720, the processor 120 may recognize a touch covering the camera 720. In an embodiment, if the touch of the finger 402 is detected, the processor 120 may recognize a touch covering the camera 720 based on an image acquired through the camera 720. In an embodiment, in response to a touch input in a preset area of ​​the display 710 (e.g., a preset area surrounding the camera 720), the processor 120 may enable the camera 720 and recognize a touch covering the camera 720 based on an image acquired through the camera 720.

[0139] In an embodiment, if a touch covering the camera 720 is recognized, the processor 120 may recognize whether the camera 720 is completely covered by the finger 402. In an embodiment, the processor 120 may recognize whether the camera 720 is completely covered based on light acquired by an image sensor (not shown) of the camera 720. In an embodiment, if the light acquired by the image sensor (not shown) of the camera 720 includes light other than reflected light of light emitted from the display 710, the processor 120 may recognize that the camera 720 is not completely covered by the finger 402. In an embodiment, if the light acquired by the image sensor (not shown) of the camera 720 does not include light other than reflected light of light emitted from the display 710, the processor 120 may recognize that the camera 720 is completely covered by the finger 402.

[0140] like Figure 7C As shown, in an embodiment, if the camera 720 is completely covered by the finger 402, the processor 120 may display a second user interface 713 for biometric measurement. Fig.7DAs shown, in an embodiment, if the camera 720 is not completely covered by the finger 402 , the processor 120 may display a third user interface 715 including an indicator 714 , which guides the user to re-touch the display 710 with his or her finger 402 .

[0141] refer to Figure 7C In an embodiment, the processor 120 may display a second user interface 713 on the display 710. In an embodiment, the processor 120 may set a portion of the second user interface 713 as a launch area 750. In an embodiment, the processor 120 may set the launch area 750 based on the touch area 740 of the finger 402. In an embodiment, the processor 120 may set the launch area 750 so that its boundary is within the touch area 740 of the finger 402.

[0142] In an embodiment, the processor 120 may emit a preset light (eg, Figure 4C In an embodiment, the processor 120 may determine the color, intensity, or a combination thereof of the lights 461 and 463 based on the type of biometric information to be acquired.

[0143] In an embodiment, the processor 120 may emit lights 461 and 463 from the emission area 750 on the display 710 , and measure biometric information of the user based on reflected lights 471 and 473 of the lights 461 and 463 acquired by the camera 720 .

[0144] Although in this example, Figure 7C The emission area 750 is shown to be annular, but the present disclosure is not limited thereto, and the emission area 750 may take various shapes. Fig. 8A In an embodiment, the electronic device 801 may include two or more emission areas 851 and 852 on the display 810, wherein the display 810 is exposed through a majority of the front plate 830 of the electronic device 801. In an embodiment, the emission areas 851 and 852 may have a circular shape together. In an embodiment, as an example, the emission areas 851 and 852 may be symmetrically arranged around the center of the camera 820. Alternatively, the emission areas 851 and 852 may be asymmetrically arranged around the center of the camera 820.

[0145] In an embodiment, the processor 120 may control the emission areas 851 and 852 to be lit at different times. In an embodiment, the processor 120 may control the emission areas 851 and 852 to be lit in different colors.

[0146] In an embodiment, the processor 120 may control the emission regions 851 and 852 to emit light of a first color (e.g., red) for a first duration, and to emit light of a second color (e.g., green) for a second duration. In an embodiment, the processor 120 may control the emission regions 851 and 852 to be lit according to a color (e.g., yellow) mixed with the first color (e.g., red) and the second color (e.g., green).

[0147] refer to Figure 8B In an embodiment, the emission area 853 of the display 810 exposed through the majority of the front plate 830 of the electronic device 801 may include two or more emission sub-areas 854 and 855. In an embodiment, in order to emit light in a preset pattern, the processor 120 may divide the emission area 853 into two or more emission sub-areas 854 and 855. In an embodiment, the processor 120 may control the display 810 to emit light corresponding to the preset pattern from the emission sub-areas 854 and 855.

[0148] In an embodiment, the processor 120 may control the emission sub-areas 854 and 855 to be illuminated at different times. In an embodiment, the processor 120 may control the emission sub-areas 854 and 855 to be illuminated in different colors.

[0149] refer to Figure 8C In an embodiment, the emission areas 856 and 857 of the display 810 exposed through the majority of the front plate 830 of the electronic device 801 may include an emission area 857 directly adjacent to the camera 820 and an emission area 856, wherein the emission area 857 is interposed between the emission area 856 and the camera 820. In an embodiment, the processor 120 may control the emission areas 856 and 857 to be illuminated at different times. In an embodiment, the processor 120 may control the emission areas 856 and 857 to be illuminated in different colors. In an embodiment, the processor 120 may control the emission areas 856 and 857 to be illuminated so that the light source close to the camera 820 emits light of a relatively shorter wavelength. For example, the emission area 856 far from the camera 820 may emit light of a first color (e.g., red), while the emission area 857 close to the camera 820 may emit light of a second color (e.g., green).

[0150] In an embodiment, the processor 120 may control the emission areas 856 and 857 to be illuminated at different intensities. In an embodiment, the processor 120 may control the emission area 856 to be illuminated with a first intensity of light and the emission area 857 to be illuminated with a second intensity of light. In different embodiments, the first intensity may be higher or lower than the second intensity.

[0151] refer to Fig.8DIn an embodiment, two or more emission areas 861 and 862 of the display 810 may be configured to be exposed through a majority of the front panel 830 of the electronic device 801. In an embodiment, the emission areas 861 and 862 together may form a quadrilateral shape.

[0152] refer to Fig. 8E In an embodiment, a quadrilateral emission area 863 of the display 810 exposed through a majority of the front plate 830 of the electronic device 801 may include two or more emission sub-areas 864 and 865.

[0153] refer to Fig.8F In an embodiment, the emission areas 866 and 867 of the display 810 exposed through a majority of the front panel 830 of the electronic device 801 may include a quadrilateral emission area 867 directly adjacent to the camera 820 and a quadrilateral emission area 866, wherein the emission area 867 is inserted between the quadrilateral emission area 866 and the camera 820.

[0154] refer to Figure 8G In an embodiment, the emission area 871 of the display 810 may be configured as an area having the same size as the camera 820. Therefore, in this embodiment, the camera 820 is not set in a recess or opening of the display 810, but overlaps with the pixels of the display 810. In an embodiment, the emission area 871 of the display 810 may periodically emit light. In an embodiment, in response to the periodic emission of the emission area 871 of the display 810, the camera 280 may receive reflected light of the light emitted from the emission area 871 of the display 810. In an embodiment, the light emission of the emission area 871 and the light detection of the camera 820 may be separated in time. In such an embodiment, the processor 120 may control the emission area 871 not to emit light when the camera 820 receives light. In an embodiment, the processor 120 may set a shadow area (not shown) around the emission area 871. In an embodiment, the shadow area (not shown) may emit light during biometric measurement. In an embodiment, the shadow area (not shown) may be set within the touch area 740 of the finger 402.

[0155] Emission areas 851, 852, 853, 856, 857, 861, 862, 863, 866, 867, and 871 are shown here as circles or quadrangles. However, these are just examples, and the emission areas may be other shapes, such as polygons.

[0156] refer to Fig.7DIn an embodiment, if the processor 120 recognizes that the camera 720 is not completely covered by the finger 402, the processor 120 may display an indicator 714 on the third user interface 715, which guides the user to re-touch the display 510 with his or her finger 402. In an embodiment, as an example, the indicator 714 may be an image object having a shape for indicating the camera 720. The indicator 714 may include a phrase, an image, or a combination thereof for requesting a re-touch.

[0157] refer to Fig. 7E In an embodiment, if the indicator 714 is displayed and a touch covering the camera 720 is subsequently recognized, the processor 120 may recognize whether the camera 720 is completely covered by the finger 402. In an embodiment, if it is recognized that the camera 720 is completely covered by the heavy touch of the finger 402, the processor 120 may store the position information of the indicator 714 in a memory (e.g., Figure 1 in the memory 130).

[0158] like Figure 7C As shown, in an embodiment, if the camera 720 is completely covered by the re-touch of the finger 402, the processor 120 may display the second user interface 713 for biometric measurement. Fig.7D As shown, in an embodiment, if the camera 720 is not completely covered by the re-touch of the finger 402 , the processor 120 may again display the third user interface 715 including the indicator 714 , which guides the user to touch the display 710 again.

[0159] In an embodiment, the processor 120 may emit light 461 and 463 from the emission area of ​​the display 510 after the finger 402 re-touches during display of the second user interface 713, and measure the user's biometric information based on reflected light 471 and 473 of the light 461 and 463 acquired by the camera 520.

[0160] In an embodiment, during display of the third user interface 715, the processor 120 may redisplay the indicator 714 requesting to re-touch the finger 402. In an embodiment, the processor 120 may display the indicator 714 at a re-determined position based on the touch area 741 of the finger 402.

[0161] Figures 9A to 9E They are views respectively showing biometric information measurement using an electronic device 901 according to an embodiment. Fig. 10A , 10B 10C and 10D are views showing examples of emission areas of the electronic device 1001 according to the embodiment. 9A to 9E Electronic device 901 or FIG. 10A to FIG. 10CThe electronic device 1001 may correspond to Figure 1 The electronic device 101 or FIG. 4A to FIG. 4C In the embodiment, by referring to Figure 1 The configuration of the electronic device 101 is described Figures 9A to 9E as well as Fig. 10A , 10B and 10C.

[0162] refer to Figures 9A to 9E In an embodiment, if the electronic device 701 is viewed in a first direction (e.g., the front direction of the electronic device 901), the camera 920 of the electronic device 901 may not overlap with the display 910. In an embodiment, the camera 920 may be disposed in an area of ​​the front panel 930 where the display 910 is not exposed.

[0163] refer to Fig.9A In an embodiment, a processor (eg, processor 120 ) may display a first user interface 911 on the display 910 .

[0164] refer to Fig. 9B In an embodiment, during displaying the first user interface 911 on the display 910, the processor 120 may recognize that a finger of the user (eg, Figure 4C The touch of the finger 402) covering the camera 920. In an embodiment, if the touch covering the camera 920 is recognized, the processor 120 may display a second user interface 913 different from the first user interface 911 on the display 910.

[0165] In an embodiment, the processor 120 may recognize a touch covering the camera 920 based on a touch area corresponding to the touch area 940 of the finger 402 detected by a touch sensor (not shown) of the display 910. In an embodiment, if the touch of the finger 402 is detected, the processor 120 may recognize a touch covering the camera 920 based on an image acquired through the camera 920. In an embodiment, in response to a touch input in a preset area of ​​the display 910 (e.g., an area on the display 910 adjacent to the camera 920), the processor 120 may enable the camera 920 and recognize a touch covering the camera 920 based on an image acquired through the camera 920.

[0166] In an embodiment, if a touch covering the camera 920 is recognized, the processor 120 may recognize whether the camera 920 is completely covered by the finger 402. In an embodiment, the processor 120 may recognize whether the camera 920 is completely covered based on light acquired by an image sensor (not shown) of the camera 920. In an embodiment, if the light acquired by the image sensor (not shown) of the camera 920 includes light other than reflected light of light emitted from the display 910, the processor 120 may recognize that the camera 920 is not completely covered by the finger 402. In an embodiment, if the light acquired by the image sensor (not shown) of the camera 920 does not include light other than reflected light of light emitted from the display 910, the processor 120 may recognize that the camera 920 is completely covered by the finger 402.

[0167] like Fig. 9C As shown, in an embodiment, if the camera 920 is completely covered by the finger 402, the processor 120 may display a second user interface 913 for biometric measurement, such as Fig.9D As shown, in an embodiment, if the camera 920 is not completely covered by the finger 402 , the processor 120 may display a third user interface 915 including an indicator 914 that guides the user to re-touch the display 910 with his or her finger 402 .

[0168] refer to Fig. 9C In an embodiment, the processor 120 may display a second user interface 913 on the display 910. In an embodiment, the processor 120 may set a portion of the second user interface 913 as an emission area 950. In an embodiment, the processor 120 may set the emission area 950 based on the touch area 940 of the finger 402. In an embodiment, the processor 120 may set the emission area 950 so that its boundary is within the touch area 940 of the finger 402. In an embodiment, the processor 120 may set the emission area 950 to be close to the camera 920. In an embodiment, the processor 120 may determine the shape, pattern, size, position, or a combination thereof of the emission area 950 based on the type of biometric information to be acquired, wherein the type may be selected by the user during display of the first user interface 911.

[0169] In an embodiment, the processor 120 may emit a preset light (eg, Figure 4C In an embodiment, the processor 120 may determine the color, intensity, or a combination thereof of the lights 461 and 463 based on the type of biometric information to be acquired.

[0170] In an embodiment, the processor 120 may emit lights 461 and 463 from the emission area 950 on the display 910 , and measure biometric information of the user based on reflected lights 471 and 473 of the lights 461 and 463 acquired by the camera 920 .

[0171] Although in this example, Fig. 9C The emission area 950 is shown in an elliptical shape, but the present disclosure is not limited thereto, and the emission area 950 may take various shapes. Fig. 10A In an embodiment, as an example, the emission area 1051 of the display 1010 may be in the shape of a quadrilateral. The emission area 1051 may have a shape such as a polygon, a circle, an ellipse, and the like.

[0172] Reference Fig. 10B , the emission areas 1053 and 1053 of the display 1010 may include two or more emission areas. In an embodiment, the processor 120 may control the emission areas 1052 and 1053 to be lit at different times. Therefore, in this embodiment, the processor 120 may control the emission area 1052 to be lit during the first duration and the emission area 1053 to be lit during the second duration.

[0173] In an embodiment, the processor 120 may control the emission areas 1052 and 1053 to be lit in different colors. In an embodiment, as an example, the processor 120 may emit light of a first color (e.g., red) from the emission area 1052, and emit light of a second color (e.g., green) from the emission area 1053. Alternatively, the processor 120 may control the emission areas 1052 and 1053 to be lit in the same color.

[0174] In an embodiment, the processor 120 may control the emission regions 1052 and 1053 to emit light of a first color (e.g., red) for a first duration, and to emit light of a second color (e.g., green) for a second duration. In an embodiment, the processor 120 may control the emission regions 1052 and 1053 to be illuminated with a color (e.g., yellow) mixed with the first color (e.g., red) and the second color (e.g., green).

[0175] refer to Fig. 10C In an embodiment, the emission area 1054 of the display 1010 may include two or more emission sub-areas 1055 to 1058. In an embodiment, in order to emit light in a preset pattern, the processor 120 may divide the emission area 1054 into two or more emission sub-areas 1055 to 1058. In an embodiment, the processor 120 may control the display 1010 to emit light corresponding to the preset pattern from the emission sub-areas 1055 to 1058.

[0176] In an embodiment, the processor 120 may control the emission sub-areas 1055 to 1058 at different times. In an embodiment, the processor 120 may control the emission sub-areas 1055 to 1058 to be lit in different colors.

[0177] refer to Fig.9D In an embodiment, if the processor 120 recognizes that the camera 920 is not completely covered by the finger 402, the processor 120 may display an indicator 914 on the third user interface 915, which guides the user to re-touch the display 910 with his or her finger 402. In an embodiment, as an example, the indicator 914 may be an image object for indicating the shape of the camera 920. The indicator 914 may include a phrase, an image, or a combination thereof for requesting a re-touch.

[0178] In an embodiment, the processor 120 may determine the position of the indicator 914 based on the position of light other than reflected light acquired by an image sensor (not shown) of the camera 920. The position of the light other than reflected light may be a position corresponding to an area exposed to light other than reflected light. In an embodiment, for example, if the position of the light other than reflected light is in a first direction from the center of the camera 920, the processor 120 may determine the position of the indicator 914 as a position at a set distance from a preset reference position in the first direction. In an embodiment, the processor 120 may determine the set distance in response to an area exposed to light other than reflected light on an image acquired by the image sensor (not shown) of the camera 920. In an embodiment, the processor 120 may determine the set distance in response to the width of the area exposed to light other than reflected light.

[0179] refer to Fig.9E In an embodiment, if the indicator 914 is displayed and a touch covering the camera 920 is subsequently recognized, the processor 120 may recognize whether the camera 920 is completely covered by the finger 402. In an embodiment, if it is recognized that the camera 920 is completely covered by the re-touch of the finger 402, the processor 120 may store the position information of the indicator 914 in a memory (e.g., Figure 1 in the memory 130).

[0180] like Fig. 9C As shown, in an embodiment, if the camera 920 is completely covered by the re-touch of the finger 402, the processor 120 may display a second user interface 913 for biometric measurement. Fig.9D As shown, in an embodiment, if the camera 920 is not completely covered by the re-touch of the finger 402 , the processor 120 may again display the third user interface 915 including the indicator 914 guiding the user to re-touch the display 910 .

[0181] In an embodiment, the processor 120 may emit light 461 and 463 from the emission area of ​​the display 910 after the finger 402 re-touches during display of the second user interface 913, and measure the user's biometric information based on reflected light 471 and 473 of the light 461 and 463 acquired by the camera 920.

[0182] In an embodiment, during display of the third user interface 915, the processor 120 may re-display the indicator 914 requesting a re-touch of the finger 402. In an embodiment, the processor 120 may display the indicator 914 at a re-determined position based on the touch area 941 of the finger 402.

[0183] Fig.11 An electronic device (eg, FIG. 4A to FIG. 4C 401) of the electronic device. Figure 1 The electronic device 101 and FIG. 4A to FIG. 4C The configuration of the electronic device 401 is described Fig.11 operation.

[0184] Reference Fig.11 In operation 1110, a processor (eg, Figure 1 The processor 120 of the embodiment may display a user interface on a display (e.g., display 410). In an embodiment, for example, during the operation of the biometric information measurement application, the user interface related to the sensor (e.g., touch sensor) displayed in operation 1110 may receive an input (e.g., touch input) to select the type of biometric information of the user to be measured. In an embodiment, the user interface may be generated by another running application different from the biometric information measurement application.

[0185] In an embodiment, the processor 120 may determine the type of biometric information to be acquired through the user interface displayed in operation 1110. In an embodiment, the biometric information may include, for example, heart rate, oxygen saturation, pressure index, blood pressure, blood sugar, tissue water content, tissue dehydration, fingerprints, or a combination thereof. The biometric information acquired using reflected light is not limited thereto.

[0186] In operation 1120, the processor 120 may recognize that a finger of a user (eg, Figure 4CIn an embodiment, the processor 120 may recognize a touch covering the camera 420 based on a touch area 440 corresponding to the touch area 440 of the finger 402 detected by a touch sensor (not shown) of the display 410. In an embodiment, if the detected touch area surrounds the camera 420, the processor 120 may recognize a touch covering the camera 420. In an embodiment, if the touch of the finger 402 is detected, the processor 120 may recognize a touch covering the camera 420 based on an image acquired through the camera 420. In an embodiment, in response to a touch input in a preset area of ​​the display 410 (e.g., a preset area surrounding the camera 420), the processor 120 may enable the camera 420 and recognize a touch covering the camera 420 based on an image acquired through the camera 420.

[0187] In an embodiment, if a touch covering the camera 420 is recognized ("Yes"), the processor 120 may perform operation 1130. In an embodiment, if a touch covering the camera 420 is not recognized ("No"), the processor 120 may return to operation 1120.

[0188] In operation 1130, the processor 120 may change the user interface to a user interface for biometric information acquisition. In an embodiment, during the execution of the biometric information measurement application, the user interface displayed in operation 1130 may be associated with the measurement of the biometric information of the user.

[0189] In operation 1140, the processor 120 may acquire biometric information using light emitted from the display 410. In an embodiment, the processor 120 may acquire biometric information by emitting light from an emission area (e.g., Figure 4B and Figure 4C The emission region 450) emits light (e.g., Figure 4C 461 and 463), and obtaining reflected light of light 461 and 463 through camera 420 (for example, Figure 4C The biometric information is obtained by analyzing the reflected light 471 and 473.

[0190] In an embodiment, the processor 120 may set a portion of the user interface as the emission area 450. In an embodiment, the processor 120 may set the emission area 450 based on the touch area 440 of the finger 402. In an embodiment, the processor 120 may set the emission area 450 so that its boundary is within the touch area 440 of the finger 402. In an embodiment, the processor 120 may set the emission area 450 to surround the camera 420. In an embodiment, during display of the user interface in operation 1110, the processor 120 may determine the shape, pattern, size, position, or a combination thereof of the emission area 450 based on the type of biometric information to be acquired.

[0191] In an embodiment, the processor 120 may emit a preset light (eg, Figure 4C In an embodiment, during display of the user interface in operation 1110, the processor 120 may determine the color, intensity, or a combination thereof of the lights 461 and 463 based on the type of biometric information to be acquired.

[0192] Fig.12 An electronic device (eg, FIG. 4A to FIG. 4C 401) of the electronic device. Figure 1 The electronic device 101 and FIG. 4A to FIG. 4C The configuration of the electronic device 401 is described Fig.12 operation.

[0193] In an embodiment, Fig.12 Operation 1210 may correspond to Fig.11 Operation 1120. In an embodiment, Fig.12 Operation 1235 may correspond to Fig.11 Operation 1140.

[0194] Reference Fig.12 In operation 1210, a processor (eg, Figure 1 The processor 120 can recognize the user's finger (for example, Figure 1 The touch of finger 402) covering a camera (e.g., camera 420).

[0195] In operation 1220, the processor 120 may identify whether the touch position needs to be changed. In an embodiment, if the camera 420 is not completely covered by the finger 402, the processor 120 may identify that it is necessary to change the touch position.

[0196] In an embodiment, if the light acquired by the image sensor (not shown) of the camera 420 includes light other than that from the display (e.g., FIG. 4A to FIG. 4CIf the camera 420 is light other than the reflected light of the light emitted by the display 410), the processor 120 can recognize that the camera 420 is not completely covered by the finger 402.

[0197] In an embodiment, if the processor 120 identifies that the touch position needs to be changed ("Yes"), the processor 120 may perform operation 1231. In an embodiment, if the processor 120 identifies that the touch position does not need to be changed ("No"), the processor 120 may perform operation 1235.

[0198] In operation 1231, the processor 120 may guide the user to re-touch the display 410. In an embodiment, the processor 120 may display an indicator on the user interface requesting re-touch of the finger 402. In an embodiment, the processor 120 may display an indicator on the user interface requesting re-touch of the finger 402. Figure 1 The position of the indicator requesting a re-touch is determined based on information in the memory 130 of the controller.

[0199] In an embodiment, the processor 120 may display an indicator requesting a re-touch on the user interface, return to operation 1210, and thus re-detect the user touch. In an embodiment, if the user touch is released and then detected again, if the touch area 440 is changed, or a combination thereof occurs, the processor 120 may recognize that the user touch is re-detected. In an embodiment, if a re-detected user touch is detected in operation 1220 and the processor 120 recognizes that an additional re-touch is not required, the processor 120 may store the position information of the indicator displayed in operation 1231 in a memory (e.g., Figure 1 in the memory 130).

[0200] In operation 1235, the processor 120 may acquire biometric information using light emitted from the display 410. In an embodiment, the processor 120 may acquire biometric information by emitting light from an emission area (e.g., Figure 4B and Figure 4C The emission region 450) emits light (e.g., Figure 4C 461 and 463), and obtaining reflected light of light 461 and 463 through camera 420 (for example, Figure 4C The biometric information is obtained by analyzing the reflected light 471 and 473.

[0201] As mentioned above, an electronic device (e.g., FIG. 4A to FIG. 4C The electronic device 401) and the operating method thereof can provide the user with various functions associated with biometric information acquisition without a dedicated light emitting device and an optical sensor.

[0202] As mentioned above, an electronic device (e.g., FIG. 4A to FIG. 4CThe electronic device 401 may include a housing, a display exposed through at least a portion of a first surface of the housing, an image sensor exposed through at least a portion of the first surface of the housing, a processor operably connected to the display and the image sensor, and a memory operably connected to the processor. The memory may store instructions that, when executed, cause the processor to detect a touch of a user's finger on the image sensor while displaying a user interface using the display, and in response to detecting the touch, change the user interface in a display area within a predetermined distance from the image sensor while the touch of the finger remains on the image sensor, and acquire biometric information of the user based on reflected light of light emitted from the display. The reflected light is acquired using an image sensor.

[0203] In an embodiment, the instructions may cause the processor to emit light having a first intensity in an area of ​​the display within a predetermined distance from the image sensor before a touch is detected, and to emit light having a second intensity higher than the first intensity in an area of ​​the display within a predetermined distance from the image sensor in response to detecting a touch.

[0204] In an embodiment, the instructions may cause the processor to display another user interface in response to receiving light other than the reflected light at the image sensor to guide the user to re-touch the display.

[0205] In an embodiment, the instructions may cause the processor to identify an area touched by a user's finger on the display in response to detecting a touch, and based on the identified area, set an area of ​​the display within a predetermined distance from the image sensor.

[0206] In an embodiment, the image sensor may be disposed in a display area of ​​the display when viewed in a front view of the housing.

[0207] In an embodiment, when viewed in a front view of the housing, a display area within a predetermined distance from the image sensor may surround the image sensor.

[0208] In an embodiment, the instructions may cause the processor to, in response to detecting the touch, change the user interface at specified time intervals while the touch of the finger remains on the image sensor.

[0209] In an embodiment, the instructions may cause the processor to emit light of a first intensity from a first portion of the display area within a predetermined distance from the image sensor in response to detecting a touch, and to emit light of a second intensity higher than the first intensity from a second portion of the display area within a predetermined distance from the image sensor in response to detecting a touch.

[0210] In an embodiment, the instructions may cause the processor to receive input, identify first biometric information to be obtained from the user based on the received input, and in response to detecting a touch, emit light corresponding to the identified first biometric information from an area of ​​the display within a predetermined distance from the image sensor.

[0211] As mentioned above, an electronic device (e.g., FIG. 4A to FIG. 4C An operating method of an electronic device 401) may include, while displaying a user interface using a display exposed through at least a portion of a first surface of a shell, detecting a touch of a user's finger on an image sensor exposed through at least a portion of a first surface of the shell, in response to detecting the touch, while the touch of the finger remains on the image sensor, changing the user interface in an area of ​​the display within a predetermined distance from the image sensor, and acquiring biometric information of the user based on reflected light of light emitted from the display, wherein the reflected light is acquired using the image sensor.

[0212] In an embodiment, before a touch is detected, light having a first intensity is emitted in a display area within a predetermined distance from an image sensor, and in response to detecting a touch, light having a second intensity higher than the first intensity is emitted in the display area within a predetermined distance from the image sensor.

[0213] In an embodiment, the method may further include: in response to receiving light other than the reflected light at the image sensor, displaying another user interface to guide the user to touch the display again.

[0214] In an embodiment, the method may further include: in response to detecting the touch, identifying an area on the display touched by the user's finger; and setting an area of ​​the display within a predetermined distance from the image sensor based on the identified area.

[0215] In an embodiment, the image sensor may be disposed in a display region of the display when viewed in a front view of the housing, and a display region within a predetermined distance from the image sensor may surround the image sensor when viewed in a front view of the housing.

[0216] In an embodiment, the method may further include: in response to detecting a touch, emitting light of a first intensity from a first portion of the display area within a predetermined distance from the image sensor; and in response to detecting a touch, emitting light of a second intensity higher than the first intensity from a second portion of the display area within the predetermined distance from the image sensor.

[0217] In an embodiment, the method may further include receiving an input, identifying first biometric information to be acquired from the user based on the received input, and in response to detecting the touch, emitting light corresponding to the identified first biometric information from an area of ​​the display within a predetermined distance from the image sensor.

[0218] In an embodiment, the method may further include: identifying an asymmetry of an image acquired by an image sensor about a reference direction based on reflected light, identifying a degree of asymmetry of the image about the reference direction in response to identifying the asymmetry of the image about the reference direction, and changing a display area within a predetermined distance based on the identified degree of asymmetry.

[0219] As mentioned above, an electronic device (e.g., FIG. 4A to FIG. 4C The electronic device 401) may include a shell, a display exposed through at least a portion of the first surface of the shell when a first surface of the shell is viewed, an image sensor arranged in a display area of ​​the display, a processor operably connected to the display and the image sensor, and a memory operably connected to the processor, wherein the memory may store instructions that, when executed, cause the processor to display an indicator indicating a preset touch request area on a user interface displayed on the display, detect a touch of a user's finger in at least a portion of the touch request area after displaying the indicator in the user interface, and in response to detecting the touch, control the display to emit light of a preset intensity in a preset second area surrounding the image sensor when viewed in a front view of the shell, and obtain biometric information of the user based on reflected light of the light of the predetermined intensity emitted from the display, wherein the reflected light is obtained using the image sensor.

[0220] In an embodiment, a characteristic of light emitted from a first portion of the second area is different from a characteristic of light emitted from a second portion of the second area.

[0221] In an embodiment, the instructions may cause the processor to display another user interface in response to receiving light other than the reflected light at the image sensor to guide the user to re-touch the display.

[0222] In an embodiment, the instructions may cause the processor to adjust the position of the indicator based on an image acquired by the image sensor based on reflected light, display the indicator at the adjusted position on a user interface displayed on the display, detect a re-touch of a user's finger after displaying the indicator at the adjusted position, and in response to detecting the re-touch of the user's finger, store the adjusted position of the indicator on the user interface in a memory when biometric information of the user is acquired based on the reflected light.

[0223] The method according to the embodiment of the present disclosure may be implemented in hardware, software, or a combination of hardware and software.

[0224] When the method is implemented by software, a computer-readable storage medium for storing one or more programs (software modules) may be provided. One or more programs stored in the computer-readable storage medium may be configured to be executed by one or more processors in the electronic device. The one or more programs may include instructions that cause the electronic device to perform the method according to the embodiments of the present disclosure as defined in the attached claims and / or disclosed herein.

[0225] Programs (software modules or software) may be stored in non-volatile memory, including random access memory and flash memory, read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), magnetic disk storage, compact disk ROM (CD-ROM), digital versatile disk (DVD), or other types of optical storage devices, or magnetic tape cassettes. Any combination of some or all of them may form a memory in which the program is stored. In addition, a plurality of such memories may be included in an electronic device.

[0226] In addition, the program can be stored in a connectable storage device that can be accessed through a communication network such as the Internet, an intranet, a local area network (LAN), a wide area network (WAN), and a storage area network (SAN), or a combination thereof. Such a storage device can access the electronic device via an external port. In addition, a separate storage device on a communication network can access the portable electronic device.

[0227] In the above exemplary embodiments of the present disclosure, the components included in the present disclosure are represented in the singular or plural, depending on the exemplary embodiments presented. However, the singular form or plural form is selected for the convenience of description suitable for the presented situation, and the embodiments of the present invention are not limited to a single element or multiple elements thereof. In addition, multiple elements expressed in the specification may be configured as a single element, or a single element expressed in the specification may be configured as multiple elements.

[0228] Some of the above embodiments of the present disclosure may be implemented in hardware, firmware; or by executing software or computer code stored in a recording medium such as a CD ROM, a digital versatile disk (DVD), a magnetic tape, a RAM, a floppy disk, a hard disk, or a magneto-optical disk; or by executing computer code originally stored on a remote recording medium or a non-transitory machine-readable medium stored on a local recording medium and downloaded over a network, so that the methods described herein may be presented using a general-purpose computer, a dedicated processor, programmable hardware, or dedicated hardware (e.g., an ASIC or FPGA) via software stored on the recording medium. As understood in the art, a computer, a processor, a microprocessor controller, or programmable hardware includes memory components such as RAM, ROM, flash memory, etc., which can store or receive software or computer code, and when the above software or computer code is accessed and executed by a computer, a processor, or hardware, the processing methods described herein are implemented.

[0229] While the present disclosure has been illustrated and described with reference to exemplary embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the true spirit and full scope of the present disclosure.

Claims

1. An electronic device, comprising: case; a display exposed through at least a portion of the first surface of the housing, the display comprising an opening; a camera aligned with the opening in the display; processor; as well as A memory storing instructions, which, when executed by the processor, cause the electronic device to: detecting a touch of a user's finger, the touch covering the camera, In response to detecting the touch, setting a first emission area and a second emission area of ​​the display for emitting light within a predetermined distance from the camera, and acquiring biometric information of a user based on reflected light of light emitted from the first emission area and the second emission area of ​​the display, wherein the reflected light is acquired using the camera, Wherein, the first emission area and the second emission area emit light at different times and / or in different colors.

2. The electronic device according to claim 1, wherein the instruction further causes the electronic device to: Prior to detecting the touch, emitting light having a first intensity from the first emission area of ​​the display within a predetermined distance from the camera, and In response to detecting the touch, light having a second intensity higher than the first intensity is emitted from the first emission area of ​​the display within a predetermined distance from the camera.

3. The electronic device of claim 1, wherein the instruction further causes the electronic device to: In response to receiving light other than the reflected light at the camera, a user interface is displayed to guide the user to touch the camera again.

4. The electronic device of claim 1, wherein the instruction further causes the electronic device to: In response to detecting the touch, identifying an area on the display that is touched by a user's finger, and Based on the identified area, the first emission area of ​​the display is set within a predetermined distance from the camera. 5 . The electronic device according to claim 1 , wherein the camera is disposed in a display area of ​​the display when viewed in a front view of the housing.

6. The electronic device of claim 1, wherein the instruction further causes the electronic device to: In response to detecting the touch, a user interface is changed at specified time intervals while the touch of the finger is maintained on the camera.

7. The electronic device of claim 1, wherein the instruction further causes the electronic device to: In response to detecting the touch, emitting light of a first intensity from the first emission area of ​​the display within a predetermined distance from the camera; and In response to detecting the touch, light having a second intensity higher than the first intensity is emitted from the second emission area of ​​the display within a predetermined distance from the camera.

8. The electronic device of claim 1, wherein the instruction further causes the electronic device to: using the display to display a user interface, receiving input to the user interface, Based on the received input, identifying first biometric information to be obtained from the user, and In response to detecting the touch, light corresponding to the identified first biometric information is emitted from the first emission area of ​​the display within a predetermined distance from the camera.

9. A method for operating an electronic device comprising a housing, the method comprising: detecting a touch of a user's finger, the touch covering a camera aligned with an opening included in a display exposed through at least a portion of the first surface of the housing; In response to detecting the touch, setting a first emission area and a second emission area of ​​the display for emitting light within a predetermined distance from the camera; as well as acquiring biometric information of a user based on reflected light of light emitted from the first emission area and the second emission area of ​​the display, wherein the reflected light is acquired using the camera, Wherein, the first emission area and the second emission area emit light at different times and / or in different colors.

10. The method of claim 9, wherein before the touch is detected, light having a first intensity is emitted from the first emission area of ​​the display within a preset distance from the camera, and In response to detecting the touch, light having a second intensity higher than the first intensity is emitted from the first emission area of ​​the display within a predetermined distance from the camera.

11. The method according to claim 9, further comprising: In response to receiving light other than the reflected light at the camera, a user interface is displayed to guide the user to touch the camera again.

12. The method according to claim 9, further comprising: In response to detecting the touch, identifying an area on the display touched by a user's finger; as well as Based on the identified area, the first emission area of ​​the display is set within a predetermined distance from the camera.

13. The method according to claim 9, further comprising: emitting light of a first intensity from the first emission area of ​​the display within a predetermined distance from the camera in response to detecting the touch; as well as In response to detecting the touch, light having a second intensity higher than the first intensity is emitted from the second emission area of ​​the display within a predetermined distance from the camera.

14. The method according to claim 9, further comprising: displaying a user interface using the display; receiving input to the user interface; identifying first biometric information to be acquired from the user based on the received input; as well as In response to detecting the touch, light corresponding to the identified first biometric information is emitted from the first emission area of ​​the display within a predetermined distance from the camera.

15. The method according to claim 9, further comprising: identifying an asymmetry of an image acquired by the camera based on the reflected light with respect to a reference direction; In response to identifying an asymmetry of the image with respect to the reference direction, identifying a degree of asymmetry of the image with respect to the reference direction; as well as Based on the identified degree of the asymmetry, the first emissive region of the display within the predetermined distance from the camera is altered.

Images