Method for authenticating user and electronic device thereof

Abstract

Various embodiments of the present disclosure relate to devices and methods for performing user authentication on an electronic device. The electronic device includes multiple lighting devices, a camera, a display, and a processor. The processor is configured to display an object on the display, obtain multiple images through the camera based on a randomly generated image acquisition mode, obtain biometric information using at least one of the multiple images, obtain user gaze change information corresponding to the movement of the object displayed on the display using the multiple images, and perform authentication on the user based on the biometric information and the user gaze change information. Other embodiments are also available.

Description

Technical Field

[0001] The present disclosure relates to user authentication of an electronic device, and more particularly, to an apparatus and method for obtaining biometric information using a randomly emitting light source of the electronic device. Background Art

[0002] With the development of information technology (IT) and semiconductor technology, various electronic devices are developing into multimedia devices capable of providing multimedia services. The multimedia services may include at least one of voice call service, messaging service, broadcast service, wireless Internet service, camera service, electronic payment service or music playback service.

[0003] With the diversification of multimedia services, electronic devices may tend to retain more personal information. The use of payment services on electronic devices is also increasing. Therefore, electronic devices may provide various authentication services to protect personal information or payment information retained therein from attacks by others. The authentication services provided by the electronic devices may include at least one of PIN code input, image input, or biometrics using the user's biometric information.

[0004] Electronic devices may use biometric information such as the user's fingerprint, iris, or face to authenticate the user.

[0005] When using fingerprint scanning (or fingerprint recognition), the electronic device can authenticate the user based on the similarity between the user's fingerprint image obtained by the fingerprint sensor and a predefined reference fingerprint image. The fingerprint image may include information about the ridges and valleys of the human body that has contacted the fingerprint sensor.

[0006] When using iris scanning (or iris recognition), the electronic device may authenticate the user based on the similarity between an iris image of the user obtained by an infrared (IR) camera and a predefined reference iris image.

[0007] When using facial scanning (or facial recognition), the electronic device can authenticate the user based on the similarity between the features of the user's facial image obtained by the camera and a predefined reference facial image. The features of the facial image may include features such as the spacing between the eyes and the distance between the nose and lips.

[0008] However, fingerprint sensors cannot distinguish fingerprints copied in materials with shapes or properties similar to those of finger skin (or human skin). Scanners used for iris scanning cannot distinguish iris images that have been output in three dimensions (3D). Electronic devices cannot distinguish between facial images printed in two dimensions (2D) or 3D and the actual facial image of the user. In other words, electronic devices cannot determine whether the biometric information obtained for user authentication is obtained from the actual user or is forged. Therefore, electronic devices have poor security for user authentication.

[0009] The above information is presented as background information only to assist with understanding the present disclosure. No determination or assertion is made as to whether any of the above might be applicable as prior art with respect to the present disclosure. Summary of the invention

[0010] Technical issues

[0011] With the diversification of multimedia services, electronic devices may tend to retain more personal information. The use of payment services on electronic devices is also increasing. Therefore, electronic devices may provide various authentication services to protect personal information or payment information retained therein from attacks by others. The authentication services provided by the electronic devices may include at least one of PIN code input, pattern input, or biometrics using the user's biometric information.

[0012] Electronic devices may use biometric information such as the user's fingerprint, iris, or face to authenticate the user.

[0013] However, fingerprint sensors cannot distinguish fingerprints copied in materials with shapes or properties similar to those of finger skin (or human skin). Scanners used for iris scanning cannot distinguish iris images that have been output in three dimensions (3D). Electronic devices cannot distinguish between facial images printed in two dimensions (2D) or 3D and the actual facial image of the user. In other words, electronic devices cannot determine whether the biometric information obtained for user authentication is obtained from the actual user or is forged. Therefore, electronic devices have poor security for user authentication.

[0014] Technical Solution

[0015] According to an embodiment of the present disclosure, a device and method for improving the accuracy of biometrics on an electronic device may be provided.

[0016] According to an embodiment of the present disclosure, an apparatus and method for an electronic device to determine whether biometric information has been obtained from an actual user may be provided.

[0017] The objects of the present disclosure are not limited to the above contents, and other unmentioned objects will be apparent to those of ordinary skill in the art from the following description.

[0018] According to an embodiment, an electronic device includes multiple lighting devices, a camera, a display, and a processor, and the processor is configured to: display an object on the display, obtain multiple images through the camera based on a randomly generated image acquisition pattern, obtain biometric information using at least one of the multiple images, obtain user gaze change information corresponding to the movement of the object displayed on the display using the multiple images, and authenticate the user based on the biometric information and the user gaze change information.

[0019] According to an embodiment, a method for operating an electronic device includes: displaying an object on a display of the electronic device; obtaining multiple images based on a randomly generated image acquisition pattern; obtaining biometric information using at least one of the multiple images; obtaining user gaze change information corresponding to the movement of the object displayed on the display using the multiple images; and performing authentication on the user based on the biometric information and the user gaze change information.

[0020] Other aspects, advantages, and salient features of the disclosure will become apparent to those skilled in the art from the following detailed description, which, taken in conjunction with the annexed drawings, discloses exemplary embodiments of the disclosure.

[0021] Before proceeding to the following detailed description, it may be advantageous to set forth definitions of certain words and phrases used in this patent document: the terms "include" and "comprising" and their derivatives mean to include without limitation; the term "or" is inclusive, meaning and / or; the phrases "associated with" and "associated with" and their derivatives may mean to include, be included, interconnected with, contain, be contained within, connect to or be connected with, be connected to or be connected with, be communicative with, cooperate with, interleave, juxtapose, be adjacent to, be bound to or be bound with, have, have the property of, etc.; and the term "controller" means any device, system, or part thereof that controls at least one operation, such device may be implemented in hardware, firmware, or software, or a combination of at least two thereof. It should be noted that the functionality associated with any particular controller may be centralized or distributed, whether local or remote.

[0022] In addition, the various functions described below may be implemented or supported by one or more computer programs, each of which is formed by a computer-readable program code and is contained in a computer-readable medium. The terms "application" and "program" refer to one or more computer programs, software components, instruction sets, processes, functions, objects, classes, instances, related data, or parts thereof suitable for implementation in appropriate computer-readable program codes. The phrase "computer-readable program code" includes any type of computer code, including source code, object code, and executable code. The phrase "computer-readable medium" includes any type of medium that can be accessed by a computer, such as a read-only memory (ROM), a random access memory (RAM), a hard drive, a compact disk (CD), a digital video disc (DVD), or any other type of memory. "Non-transitory" computer-readable media excludes wired, wireless, optical, or other communication links that transmit temporary electrical or other signals. Non-transitory computer-readable media include media that can permanently store data and media that can store data and then rewrite data, such as rewritable optical disks or erasable storage devices.

[0023] Definitions for certain words and phrases are provided throughout this patent document, those of ordinary skill in the art should understand that in many, if not most instances, such definitions apply to prior, as well as future uses of such defined words and phrases. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] A more complete understanding of the disclosure and its attendant numerous aspects will be readily obtained as the disclosure and its attendant numerous aspects will be better understood by reference to the following detailed description when considered in conjunction with the accompanying drawings, wherein:

[0025] Figure 1 shows a perspective view of an electronic device according to an embodiment;

[0026] Figure 2 shows a block diagram of an electronic device according to an embodiment;

[0027] Figure 3 A block diagram of a processor for user authentication according to an embodiment is shown;

[0028] Figure 4A , Figure 4B and Figure 4C A block diagram of an electronic device for transmitting a synchronization signal according to an embodiment is shown;

[0029] Figure 5A and Figure 5B A diagram showing a relationship between a time captured by a camera and a time illuminated by a lighting device according to an embodiment;

[0030] Figure 6 An exemplary flow chart of user authentication on an electronic device according to an embodiment is shown;

[0031] Fig. 7A and Figure 7B A diagram showing a screen configuration for performing iris scanning on an electronic device according to an embodiment;

[0032] Figure 8 An exemplary flow chart of obtaining an image by an electronic device according to an embodiment is shown;

[0033] Fig. 9 An exemplary flow chart for performing iris scanning on an electronic device according to an embodiment is shown;

[0034] Fig.10 An exemplary flow chart for performing facial scanning on an electronic device according to an embodiment is shown;

[0035] Fig.11 An exemplary flow chart of user authentication on an electronic device according to an embodiment is shown;

[0036] Fig. 12A and Fig. 12B A view showing a screen configuration for performing area scanning on an electronic device according to an embodiment;

[0037] Fig.13A , Fig. 13B , Fig. 13C , Fig.13D and Fig.13E A diagram showing lighting time and image capturing time of a flood lighting device and a structure lighting device according to an embodiment; and

[0038] Fig.14 A diagram showing switching timing of cameras according to an embodiment.

[0039] Throughout the drawings, like reference numerals will be understood to refer to like parts, components and structures. DETAILED DESCRIPTION

[0040] Discussed below Figures 1 to 14 The various embodiments used to describe the principles of the present disclosure in this patent document are merely exemplary and should not be interpreted in any way as limiting the scope of the present disclosure. Those skilled in the art will appreciate that the principles of the present invention can be implemented in any appropriately arranged system or device.

[0041] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings.

[0042] When describing the embodiments, descriptions of technologies known in the art and not directly related to the present disclosure are omitted. This is to further clarify the main points of the present disclosure without making it unclear.

[0043] For the same reason, some elements may be exaggerated or schematically shown. The size of each element does not necessarily reflect the actual size of the element. In all the drawings, the same reference numerals are used to represent the same elements.

[0044] The advantages and features of the present disclosure and the methods for realizing the advantages and features of the present disclosure can be understood by the embodiments described below in conjunction with the accompanying drawings. However, the present disclosure is not limited to the embodiments disclosed herein, and various changes may be made thereto. The embodiments disclosed herein are only provided to inform those of ordinary skill in the art of the categories of the present disclosure. The present disclosure is limited only by the appended claims. Throughout the specification, the same reference numerals represent the same elements.

[0045] It should be understood that the combination of the blocks and flowcharts in each flowchart can be performed by computer program instructions. Since the computer program instructions can be equipped in a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, the instructions executed by the processor of the computer or other programmable data processing device generate a device for performing the functions described in conjunction with one or more blocks of each flowchart. Since the computer program instructions can be stored in a computer-usable or computer-readable memory that can be directed to a computer or other programmable data processing device to implement the functions in a specified manner, the instructions stored in the computer-usable or computer-readable memory can generate a product including an instruction device for performing the functions described in conjunction with one or more blocks in each flowchart. Since the computer program instructions can be equipped in a computer or other programmable data processing device, the instructions generated by the computer as a process executed as a series of operating steps are executed by the computer or other programmable data processing device operating the computer or other programmable data processing device, and steps for performing the functions described in conjunction with the blocks in each flowchart can be provided.

[0046] In addition, each block may represent a module, segment or part of a code including one or more executable instructions for performing a specified logical function. In addition, it should also be noted that in some alternative execution examples, the functions mentioned in the block may occur in different orders. For example, two blocks shown in succession may be executed simultaneously or in reverse order according to the corresponding functions.

[0047] As used herein, the term "module" or "unit" means a software element or a hardware element, such as a field programmable gate array (FPGA) or an application specific integrated circuit (ASIC). A module or unit plays a certain role. However, the term "module" or "unit" is not limited to mean a software or hardware element. A "module" or "unit" may be configured in a storage medium that may be addressable or configurable to be reproduced by one or more processors. Therefore, as an example, a "module" or "unit" includes elements such as software elements, object-oriented software elements, class elements and task elements, processes, functions, attributes, procedures, subroutines, program code segments, drivers, firmware, microcodes, circuits, data, databases, data architectures, tables, arrays, and variables. The functions provided in an element or "module" or "unit" may be combined with additional elements, or may be divided into sub-elements or sub-modules or sub-units. In addition, an element or "module" or "unit" may be implemented as being reproduced by one or more CPUs in a device or a secure multimedia card.

[0048] Although the terms "first" and "second" are used to describe various components, these components are not limited by these terms. These terms are provided only to distinguish one component from another component. Therefore, the first component mentioned here can also be the second component within the technical spirit of the present disclosure.

[0049] In the following description, the electronic device may be various types of devices. For example, the electronic device may include at least one of, for example, a portable communication device (e.g., a smart phone), a computer device, a portable multimedia device, a camera, a wearable device, or a household appliance. According to an embodiment of the present disclosure, the electronic device is not limited to those described above.

[0050] Figure 1 A perspective view of an electronic device according to an embodiment is shown.

[0051] Reference Figure 1 , the electronic device 100 may include a display 102, and the display 102 is exposed to the outside through a front plate facing a first direction of the housing. For example, the housing may be formed of a conductive member and / or a non-conductive member.

[0052] According to an embodiment, the electronic device 100 may include a call receiver hole 112. A speaker device for outputting a sound received from another party may be provided in the call receiver hole 112.

[0053] According to an embodiment, the electronic device 100 may include a component that is exposed through the display 102 or performs a function through the front plate without being exposed to the outside to perform various functions of the electronic device 100. For example, the component may include a first lighting device 104, which is arranged on one side of the call receiver hole 112 to emit a light beam with a unique characteristic (e.g., a pattern). As an example, the first lighting device 104 may include a structural lighting device (or structural light) to emit a unique pattern of light (e.g., infrared (IR) light), thereby allowing the camera 108 to obtain a depth image. For example, the component may include a second lighting device 106, which is arranged on the other side of the call receiver hole 112 to emit a light beam uniformly over a wide area. As an example, the second lighting device 106 may include a floodlighting device to constantly emit a light beam (e.g., IR rays) for iris or facial scanning. For example, the component may include a camera 108. As an example, the camera 108 may include a camera (e.g., an IR camera) capable of obtaining a facial image or a depth image including an iris. For example, facial scanning may include two-dimensional (2D) facial scanning and three-dimensional (3D) facial scanning.

[0054] According to an embodiment, the electronic device 100 may include at least one key input device 110. For example, the key input device 110 may include a volume control button, a power button, or a button for executing a specific function (e.g., a function of running artificial intelligence (AI) or entering a fast voice recognition execution mode) on a side surface of the housing.

[0055] Figure 2 A block diagram of an electronic device according to an embodiment is shown.

[0056] Reference Figure 2 , the electronic device 100 may include at least one of a bus 210, a processor 220, a memory 230, an input / output interface 240, a display 250, a communication interface 260, a camera 270, or a lighting module 280. In some embodiments, the electronic device 100 may exclude at least one component or may add another component. As an example, the input / output interface 240 and the communication interface 260 may be from Figure 2 The components of the electronic device 100 are omitted.

[0057] The bus 210 may include circuitry for, for example, connecting the components 220 to 280 to one another and transmitting signals (eg, control messages and / or data) between the components.

[0058] The processor 220 may include one or more of a central processing unit (CPU), an application processor (AP), a communication processor (CP), or an image signal processor (ISP). For example, the processor 220 may control at least one other component of the electronic device 100 and / or perform communication-related operations or data processing.

[0059] According to an embodiment, upon detecting the occurrence of a user authentication event, the processor 220 may perform control based on a randomly occurring image acquisition mode to allow the camera 270 and the lighting module 280 to be synchronized with each other. For example, the user authentication event may occur based on at least one of input of a power button, reception of an activation command (e.g., a voice command), holding the electronic device 100, or a change in the charging state of the electronic device 100. The image acquisition mode may include at least one of an image acquisition sequence or an image acquisition time. As an example, the image acquisition sequence and the image acquisition time may be randomly generated when a user authentication event occurs. As an example, the image acquisition sequence may be randomly generated when a user authentication event occurs, and the image acquisition time may be randomly generated when user authentication fails. As an example, the image acquisition sequence may include the sequence and time of illumination of a structured lighting device and a flood lighting device.

[0060] According to an embodiment, the processor 220 may perform user authentication based on user information obtained through the camera 270 and the lighting module 280. For example, the processor 220 may obtain a facial image through the camera 270 when the flood lighting device 280-1 is illuminated. The processor 220 may obtain a depth image through the camera 270 when the structure lighting device 280-2 is illuminated. The processor 220 may perform a biometric scan on the user using the facial image. As an example, the processor 220 may perform an iris scan using an iris image contained in the facial image. As an example, the processor 220 may perform a facial scan using feature information extracted from the facial image. When the biometric scan of the user is successful, the processor 220 may use the facial image and the depth image to detect a gaze vector corresponding to a gaze change of the user. When the gaze vector matches the movement of a gaze object displayed on the display 250, the processor 220 may determine that the user authentication is successful. For example, a gaze object may be displayed on at least a portion of the display 250 to guide the user's gaze when the occurrence of a user authentication event is detected. The gaze object may be moved randomly or based on a predefined rule. When user authentication or biometric scanning of the user fails, the processor 220 may reacquire the user's facial image and depth image based on the image acquisition mode.

[0061] The memory 230 may store commands or data related to at least one other component of the electronic device 100. For example, the memory 230 may include a volatile and / or nonvolatile memory embedded in the electronic device 100. For example, the memory 230 may include an external memory functionally connected to the electronic device 100. As an example, the external memory may include at least one of a server (e.g., a cloud) or a memory card.

[0062] The input / output interface 240 may transmit a command or data input from a user or other external device to other components of the electronic device 100, or may output a command or data received from other components of the electronic device 100 to the user or other external devices. For example, the input / output interface 240 may include at least one physical button, such as a home button, a power button, and a volume control button. For example, the input / output interface 240 may include a speaker for outputting an audio signal and a microphone for collecting an audio signal.

[0063] The display 250 may display various contents (e.g., text, images, videos, icons, or symbols) to the user. For example, the display 250 may include a touch screen. The touch screen may receive, for example, touch input, gesture input, proximity, or hovering input using an electronic pen or a body part of the user.

[0064] The communication interface 260 may establish communication between the electronic device 100 and an external device (eg, another electronic device or a server). For example, the communication interface 260 may be connected to a network through wireless or wired communication to communicate with the external device.

[0065] The camera 270 may collect image information about an object to capture still images (photos) and videos. For example, the camera 270 may transmit the collected or captured data (e.g., images) to the display 250 and the processor 220. For example, the camera 270 may include at least one of an image sensor, a lens, an image signal processor (ISP), or a flash (e.g., a light emitting diode (LED) or a xenon lamp). For example, the camera 270 may include a plurality of cameras disposed on the front surface or the rear surface of the electronic device 100. As an example, the camera 270 may include at least one infrared (IR) camera disposed on the front surface of the electronic device 100.

[0066] The lighting module 280 may emit light to help identify an object. For example, the lighting module 280 may include a floodlight 280-1 and a structured lighting device 280-2 disposed on the front surface of the electronic device 100. The floodlight 280-1 may emit light of a predetermined size to allow the camera 270 to obtain a facial image of the user. The structured lighting device 280-2 may emit a unique light (e.g., a pattern) to allow the camera 270 to obtain a depth image of the user's face. As an example, at least one of the floodlight 280-1 or the structured lighting device 280-2 may use light of a near infrared (NIR) wavelength (e.g., 850 mm or 940 mm).

[0067] Figure 3 FIG. 2 shows a block diagram of a processor for user authentication according to an embodiment. In the following description, the processor 220 may include Figure 2 The processor 220 or Figure 2 at least a portion of the processor 220.

[0068] Reference Figure 3 , the processor 220 can perform user authentication through the biometric scanning module 301 and the biometric sensing module 303.

[0069] According to an embodiment, biometric scanning module 301 may perform a biometric scan on a user using an image obtained by camera 270 when flood lighting device 280-1 is illuminated. For example, when performing an iris scan, biometric scanning module 301 may detect a facial region from an image obtained by camera 270 when flood lighting device 280-1 is illuminated. Biometric scanning module 301 may perform an iris scan on a user based on a similarity between an iris image contained in the facial region and a reference iris image pre-stored in memory 230. For example, when the similarity of the iris image exceeds a reference value, biometric scanning module 301 may determine that the iris scan on the user is successful. For example, when a facial image is not detected from an image obtained by camera 270, or when the similarity of the iris image is a reference value or less, biometric scanning module 301 may determine that the iris scan on the user fails. For example, when performing a facial scan, biometric scanning module 301 may identify whether a face exists in an image obtained by camera 270 when flood lighting device 280-1 is illuminated. As an example, the biometric scanning module 301 may extract feature points in the image and determine whether there are feature points corresponding to eyes, a nose, and a mouth. The biometric scanning module 301 may perform a facial scan on the user based on the similarity between the feature points in the face included in the image and the feature points in the reference facial image pre-stored in the memory 230. The feature points of the face may include, for example, the interval between the eyes, the outline of the face, and the distance between the nose and the lips. For example, when the similarity of the facial feature points exceeds a reference value, the biometric scanning module 301 may determine that the facial scan of the user is successful. As an example, when no face is detected from the image obtained through the camera 270, or when the similarity of the facial feature points is a reference value or less, the biometric scanning module 301 may determine that the facial scan of the user has failed.

[0070] According to an embodiment, the biometric sensing module 303 may determine whether the information used for the biometric scan in the biometric scanning module 301 is information obtained from an actual user. For example, when the occurrence of the user authentication event is detected, the biometric sensing module 303 may control the display 250 to display a gaze object on at least a portion of the display 250. The biometric sensing module 303 may detect a three-dimensional (3D) facial mesh using an image obtained by the camera 270 when illuminated by the flood lighting device 280-1 and a depth image obtained by the camera 270 when illuminated by the structured lighting device 280-2. The biometric sensing module 303 may detect a gaze vector of the user based on the 3D facial mesh and the image obtained by the camera 270 when illuminated by the flood lighting device 280-1. When the movement of the gaze object displayed on the display 250 matches the gaze vector, the biometric sensing module 303 may determine that the information used for the biometric scan in the biometric scanning module 301 has been obtained from the actual user.

[0071] According to an embodiment, bio-scanning module 301 and bio-sensing module 303 may be driven independently or in parallel. When the bio-scanning performed by bio-scanning module 301 is successful and the information used for the bio-scanning in bio-scanning module 301 is determined by bio-sensing module 303 to be information obtained from an actual user, processor 220 may determine that the user authentication is successful.

[0072] According to an embodiment, when the biometric scan by the biometric scan module 301 is successful, the biometric sensing module 303 may determine whether the information used for the biometric scan in the biometric scan module 301 is information obtained from an actual user.

[0073] According to an embodiment, when information for biometric scanning is determined as information obtained from an actual user by the biometric sensing module 303 , the biometric scanning module 301 may perform the biometric scanning.

[0074] According to an embodiment, the electronic device 100 may randomly apply the illumination mode (or image acquisition mode) of light necessary for obtaining biometric information in order to increase the security of user authentication using biometric information. Therefore, the image acquisition time (capture time) of the camera 270 needs to be synchronized with the illumination time of the illumination module 280.

[0075] Figure 4A , Figure 4B and Figure 4C A block diagram of an electronic device for transmitting a synchronization signal according to an embodiment is shown.

[0076] According to the implementation mode, reference Figure 4A, the processor 220 may send a synchronization signal to the floodlighting device 280-1 or the structured lighting device 280-2 at the illumination time of each light (or the time to obtain an image) based on the randomly generated illumination pattern (or image acquisition pattern). As an example, the processor 220 may send a synchronization signal to the floodlighting device 280-1 when the floodlighting device 280-1 illuminates (or the time to obtain a facial image), and may send a synchronization signal to the structured lighting device 280-2 when the structured lighting device 280-2 illuminates (or the time to obtain a depth image). The floodlighting device 280-1 or the structured lighting device 280-2 may emit light based on the synchronization signal received from the processor 220. As an example, the floodlighting device 280-1 may emit light of a near-infrared wavelength at a first time when the synchronization signal is received from the processor 220. In this case, the floodlighting device 280-1 may output a synchronization signal to the camera 270. As an example, the structured lighting device 280-2 may emit light of a specific pattern at a second time when the synchronization signal is received from the processor 220. In this case, the structured lighting device 280-2 may output a synchronization signal to the camera 270. Upon receiving the synchronization signal from the flood lighting device 280-1 or the structured lighting device 280-2, the camera 270 may capture an image.

[0077] According to the implementation mode, reference Figure 4B , the camera 270 may output a synchronization signal to the processor 220 when acquiring an image based on a randomly generated lighting pattern (or image acquisition pattern). The processor 220 may send a synchronization signal to the floodlighting device 280-1 or the structured lighting device 280-2 based on the synchronization signal received from the camera 270. As an example, the processor 220 may send a synchronization signal to the floodlighting device 280-1 when a facial image is acquired by the camera 270, and may send a synchronization signal to the structured lighting device 280-2 when a depth image is acquired by the camera 270. The floodlighting device 280-1 or the structured lighting device 280-2 may emit light based on the synchronization signal received from the processor 220.

[0078] According to the implementation mode, Figure 4CAs shown, the processor 220 may send a synchronization signal to any one of the flood lighting device 280-1 or the structured lighting device 280-2 and the camera 270 at the time of illumination of each light (or the time of obtaining an image) based on the randomly generated illumination pattern (or image acquisition pattern). As an example, the processor 220 may send a synchronization signal to the flood lighting device 280-1 and the camera 270 at the time of illumination of the flood lighting device 280-1 (or the time of obtaining a facial image). The processor 220 may send a synchronization signal to the structured lighting device 280-2 and the camera 270 at the time of illumination of the structured lighting device 280-2 (or the time of obtaining a depth image). The flood lighting device 280-1 or the structured lighting device 280-2 may emit light based on the synchronization signal received from the processor 220. Upon receiving the synchronization signal from the processor 220, the camera 270 may capture an image.

[0079] According to an embodiment, the camera 270 may continuously or periodically obtain images regardless of the synchronization signal. In this case, the processor 220 may determine that an image obtained when no light is emitted from any of the floodlight 280-1 and the structured light 280-2 among the images obtained by the camera 270 is unreliable. The image determined to be unreliable is excluded from the biometric verification.

[0080] Figure 5A and Figure 5B A diagram showing the relationship between the time captured by a camera and the time illuminated by a lighting device according to an embodiment. Figure 2 The camera 270 captures the time and Figure 2 Here, the capture time may refer to the time taken to obtain at least one image through the camera 270. The illumination time may refer to the duration of time that the illumination module 280 emits light.

[0081] According to the implementation mode, reference Figure 5A , when the camera 270 adopts the global shutter scheme, the illumination time of the illumination module 280 may be set to be shorter than the shooting time of the camera 270. As an example, the illumination time of the floodlighting device 280-1 and the structured illumination device 280-2 may be set to be shorter than the shooting time of the camera 270 adopting the global shutter scheme. When the global shutter scheme is adopted, reducing the illumination time of the illumination module 280 in the electronic device 100 can alleviate motion blur, thereby enhancing the performance of depth calculation and gaze tracking. Here, the global shutter scheme may indicate a scheme for obtaining an image by simultaneously exposing all pixels in the camera 270.

[0082] According to the implementation mode, reference Figure 5B, when the camera 270 adopts the rolling shutter scheme, the lighting time of the lighting module 280 may be set to be equal to or longer than the shooting time of the camera 270. As an example, the lighting time of the flood lighting device 280-1 and the structure lighting device 280-2 may be set to be equal to or longer than the shooting time of the camera 270 adopting the global shutter scheme. Here, the rolling shutter scheme may refer to a scheme of obtaining an image by sequentially exposing pixels in the camera 270 by column or by row.

[0083] Figure 6 An exemplary flow chart of user authentication on an electronic device according to an embodiment is shown. Fig. 7A and Figure 7B A view showing a screen configuration for performing iris scanning on an electronic device according to an embodiment. In the following description, the electronic device may include Figure 2 electronic device 100 or at least a portion of the electronic device 100 (e.g., processor 220).

[0084] Reference Figure 6 At operation 601, the electronic device may detect the occurrence of a user authentication event. For example, when an input of a power button is detected, an activation command (e.g., a voice command) is received, the grip of the electronic device 100 is detected, or a change in the charging state of the electronic device 100 is detected, the processor 220 may determine the occurrence of a user authentication event.

[0085] At operation 603, the electronic device may display a gaze object to guide the user's gaze. For example, when performing user authentication based on iris scanning, the processor 220 may control the display 250 to display the gaze object. Fig. 7A The gaze object 710 and the user image 700 for iris scanning are displayed as shown. The gaze object may be displayed to overlap at least a portion of the user image 700 for iris scanning. As an example, the gaze object 710 may move randomly, or as shown in FIG. Figure 7B As shown, the gaze object 710 moves according to a predetermined rule. As an example, the gaze object 710 can be fixed.

[0086] In operation 605, the electronic device may randomly generate an image acquisition pattern. The image acquisition pattern may include an order and time of obtaining a facial image and a depth image for user authentication.

[0087] In operation 607, the electronic device may obtain a facial image and a depth image based on the order of obtaining images in the randomly generated image obtaining mode. For example, the processor 220 may obtain a facial image through the camera 270 based on the image obtaining order when the flood lighting device 280-1 is illuminated, and then obtain a depth image through the camera 270 when the structure lighting device 280-2 is illuminated. For example, the processor 220 may obtain a depth image through the camera 270 based on the image obtaining order when the structure lighting device 280-2 is illuminated, and then obtain a facial image through the camera 270 when the flood lighting device 280-1 is illuminated.

[0088] At operation 609, the electronic device may determine whether the biometric scan using the facial image is successful. For example, the processor 220 may perform the biometric scan based on biometric information (e.g., iris or face) acquired from the facial image obtained through the camera 270. The processor 220 may determine whether the biometric scan is successful based on similarity between the biometric information acquired from the facial image and reference biometric information stored in the memory 230.

[0089] When the biometric scan is successful ("Yes" in operation 609), the electronic device may detect a gaze vector of the user based on the facial image and the depth image in operation 611. For example, the processor 220 may generate a gaze vector corresponding to a change in the user's gaze using eye gaze tracking based on the facial image and the depth image obtained in operation 607.

[0090] In operation 613, the electronic device may identify whether the gaze vector of the user matches the motion information of the gaze object. For example, when the similarity between the motion information about the gaze object detected by gaze tracking and the gaze vector of the user exceeds a reference value, the processor 220 may determine that the gaze vector of the user matches the motion information about the gaze object. When the similarity between the motion information about the gaze object and the gaze vector of the user is a reference value or less, the processor 220 may determine that the gaze vector of the user does not match the motion information about the gaze object.

[0091] When it is determined that the gaze vector of the user does not match the motion information of the gaze object ("No" in operation 613), the electronic device may update the gaze object in operation 617. For example, when it is determined that the gaze vector of the user does not match the motion information about the gaze object, the processor 220 may determine that the biometric information of the user for biometric scanning has not been obtained from the actual user. Therefore, the processor 220 may update the position of the gaze object to re-identify whether the object attempting user authentication is the actual user.

[0092] When the biometric scan fails ("No" in operation 609) or the gaze object is updated (operation 617), the electronic device may identify the time to obtain the next image based on the randomly generated image acquisition pattern. When the next image acquisition time arrives, in operation 607, the electronic device may obtain a facial image and a depth image based on the order of obtaining images in the randomly generated image acquisition pattern. For example, the electronic device may repeatedly perform a series of operations for user authentication (e.g., operations 607 to 613 and operations 617 to 619).

[0093] Upon determining that the gaze vector of the user matches the motion information about the gaze object (“YES” in operation 613 ), the electronic device may determine that user authentication is successful in operation 615 .

[0094] According to an embodiment, when user authentication fails a reference number of times or fails during a reference time, the electronic device may output authentication failure related information. The authentication failure related information may be output in the form of any one of a message, a sound, or an indicator (LED). When user authentication fails, the electronic device may remain locked. Here, the failure of user authentication may include Figure 6 In the case where the biometric scan fails in operation 609, or in the case where it is determined in operation 613 that the gaze vector of the user does not match the motion information of the gaze object.

[0095] According to an embodiment, the electronic device 100 may perform a biometric scanning operation using a facial image (e.g., operation 609) and an operation of determining whether a user's gaze vector matches the movement of a gaze object (e.g., operations 611 to 613) in parallel. When the biometric scanning is successful and the user's gaze vector matches the movement of the gaze object, the electronic device 100 may determine that the user authentication is successful.

[0096] According to an embodiment, upon determining that the gaze vector of the user matches the movement of the gaze object (eg, operations 611 to 613 ), the electronic device 100 may perform a biometric scan (eg, operation 609 ).

[0097] Figure 8 An exemplary flow chart of obtaining an image by an electronic device according to an embodiment is shown. The following description may include Figure 6 The operation of obtaining the facial image and the depth image in operation 607. In the following description, the electronic device may include Figure 2 electronic device 100 or at least a portion of the electronic device 100 (e.g., processor 220).

[0098] Reference Figure 8 , when the image acquisition pattern is randomly generated (e.g., Figure 6In operation 605), in operation 801, the electronic device may identify the lighting time of the lighting device based on the image acquisition mode.

[0099] In operation 803, the electronic device may determine whether the flood lighting device emits light first based on the lighting time of the lighting device.

[0100] When it is determined that the floodlight emits light first ("Yes" in operation 803), the electronic device may obtain a facial image using the floodlight in operation 805. For example, the camera 270 may obtain an image (facial image) when the floodlight 280-1 is illuminated based on a randomly generated image acquisition pattern.

[0101] In operation 807, the electronic device may obtain a depth image using the structured lighting device. For example, the camera 270 may obtain a depth image when illuminated by the structured lighting device 280-2 based on a randomly generated image acquisition pattern.

[0102] When it is determined that the structured lighting device emits light first ("No" in operation 803), the electronic device may obtain a depth image using the structured lighting device in operation 809. For example, the camera 270 may recognize the illumination time of the structured lighting device 280-2 based on a synchronization signal received from the structured lighting device 280-2 or the processor 220. The camera 270 may obtain a depth image when the structured lighting device 280-2 illuminates.

[0103] In operation 811, the electronic device may obtain a facial image using the floodlighting device. For example, the camera 270 may recognize the lighting time of the floodlighting device 280-1 based on a synchronization signal received from the floodlighting device 280-1 or the processor 220. The camera 270 may obtain an image (facial image) when the floodlighting device 280-1 is lighting.

[0104] According to an embodiment, the electronic device may sequentially obtain the face image and the depth image based on the time at which the image is obtained in the randomly generated image acquisition mode.

[0105] Fig. 9 An exemplary flow chart of performing iris scanning on an electronic device according to an embodiment is shown. The following description may include Figure 6 In the following description, the electronic device may include an operation of determining whether the biometric scan is successful in operation 609. Figure 2 electronic device 100 or at least a portion of the electronic device 100 (e.g., processor 220).

[0106] Reference Fig. 9 , after obtaining the facial image and the depth image (e.g., Figure 6In operation 607), in operation 901, the electronic device may identify whether a facial area is detected based on feature points in the facial image. For example, the processor 220 may identify whether an area including feature points corresponding to the eyes, nose, and mouth of a person is detected from the facial image.

[0107] When the face region is detected from the face image (“YES” in operation 901 ), the electronic device may identify whether an iris image is detected from the face region in operation 903 .

[0108] When an iris image is detected from the facial region in the facial image ("Yes" in operation 903), the electronic device may determine whether the biometric scan of the user using the iris image is successful in operation 905. For example, the processor 220 may perform an iris scan based on the similarity between the iris image detected from the facial image and at least one reference iris image stored in the memory 230. When the similarity between the reference iris image and the iris image exceeds a reference value, the processor 220 may determine that the iris scan of the user is successful.

[0109] When a facial region is not detected from a facial image (“No” in operation 901), an iris image is not detected from a facial region (“No” in operation 903), or a biometric scan using an iris image detected from a facial image fails (“No” in operation 905), in operation 907, the electronic device may identify a time to obtain a next image based on a randomly generated image acquisition pattern. For example, Figure 6 As shown in operation 619, the processor 220 may Figure 6 The image acquisition pattern randomly generated in operation 605 is used to identify whether the next image acquisition time has arrived.

[0110] According to an embodiment, when it is determined in operation 905 that the iris scan is successful ("Yes" in operation 905), the electronic device may identify whether an iris image for iris scan has been obtained from an actual user. For example, the electronic device may perform Figure 6 Operations 611 to 613.

[0111] Fig.10 An exemplary flow chart of performing facial scanning on an electronic device according to an embodiment is shown. The following description may include Figure 6 In the following description, the electronic device may include an operation of determining whether the biometric scan is successful in operation 609. Figure 2 electronic device 100 or at least a portion of the electronic device 100 (e.g., processor 220).

[0112] Reference Fig.10 , after obtaining the facial image and the depth image (e.g., Figure 6In operation 607), in operation 1001, the electronic device may identify whether the user's face is detected based on the feature points in the facial image. For example, the processor 220 may identify whether feature points corresponding to the eyes, nose, and mouth of a person are detected from the facial image.

[0113] When the user's face is detected from the facial image ("Yes" in operation 1001), the electronic device may determine whether the facial scan of the user is successful in operation 1003. For example, the processor 220 may perform facial scanning based on the similarity between the feature points of the user's face detected from the facial image and at least one reference facial image stored in the memory 230. When the similarity between the feature points of the reference facial image and the user's face detected from the facial image exceeds a reference value, the processor 220 may determine that the facial scan of the user is successful. Here, the facial feature points may include, for example, the interval between the eyes, the facial contour, and the distance between the nose and the lips detected based on the feature points of the facial image.

[0114] When the user's face is not detected from the facial image ("No" in operation 1001), or when the biometric scan of the user's face detected from the facial image fails ("No" in operation 1003), in operation 1005, the electronic device may identify the time to obtain the next image based on the randomly generated image acquisition pattern. For example, Figure 6 As shown in operation 619, the processor 220 may Figure 6 The image acquisition pattern randomly generated in operation 605 is used to identify whether the next image acquisition time has arrived.

[0115] According to an embodiment, when it is determined in operation 1003 that the facial scan is successful ("Yes" in operation 1003), the electronic device may identify whether the user's facial image for facial scanning has been obtained from an actual user. For example, the electronic device may perform Figure 6 Operations 611 to 613.

[0116] Fig.11 An exemplary flow chart of user authentication on an electronic device according to an embodiment is shown. Fig. 12A and Fig. 12B A view showing a screen configuration for performing face scanning on an electronic device according to an embodiment. In the following description, the electronic device may include Figure 2 electronic device 100 or at least a portion of the electronic device 100 (e.g., processor 220).

[0117] Reference Fig.11In operation 1101, the electronic device may detect the occurrence of a user authentication event. The occurrence of the user authentication event may be detected based on at least one of an input of a power button, reception of an activation command (e.g., a voice command), a change in a holding state of the electronic device 100, or a charging state of the electronic device 100.

[0118] In operation 1103, the electronic device may display a gaze object to guide the user's gaze. For example, when performing user authentication based on facial scanning, the processor 220 may control the display 250 to display a gaze object 1210 and guide information 1200 for facial scanning, such as Fig. 12A The processor 220 may apply an animation effect to the gaze object 1210, such as Fig. 12B In other words, the gaze object 1210 may be moved randomly or according to a predetermined rule.

[0119] In operation 1105, the electronic device may randomly generate an image acquisition sequence for user authentication. The image acquisition sequence may include an order of obtaining a user image (face image) obtained using the flood lighting device 280-1 and a depth image obtained using the structured lighting device 280-2.

[0120] In operation 1107, the electronic device may obtain a facial image and a depth image based on a randomly generated image acquisition sequence. For example, the processor 220 may sequentially emit light through the flood lighting device 280-1 or the structured lighting device 280-2 according to the image acquisition sequence, thereby sequentially obtaining a facial image and a depth image.

[0121] In operation 1109, the electronic device may determine whether the biometric scan using the facial image is successful. For example, the processor 220 may perform an iris scan or a facial scan using the facial image obtained through the camera 270.

[0122] When the biometric scan is successful ("Yes" in operation 1109), the electronic device may detect a gaze vector of the user based on the facial image and the depth image in operation 1111. For example, processor 220 may detect a 3D facial mesh based on the depth image and the facial area detected from the facial image. Processor 220 may generate a gaze vector corresponding to a gaze change of the user using the 3D facial mesh and the facial image. Here, the gaze vector of the user may include information about at least one of a direction, a distance, or a movement speed of the user's gaze.

[0123] In operation 1113, the electronic device may identify whether the gaze vector of the user matches the motion information of the gaze object. For example, when the similarity between the motion information about the gaze object and the gaze vector of the user exceeds a reference value, the processor 220 may determine that the gaze vector of the user does not match the motion information about the gaze object. When the similarity between the motion information about the gaze object and the gaze vector of the user is a reference value or less, the processor 220 may determine that the gaze vector of the user does not match the motion information about the gaze object.

[0124] When it is determined that the gaze vector of the user does not match the motion information about the gaze object ("No" in operation 1113), the electronic device may update the position of the gaze object in operation 1117. For example, when it is determined that the gaze vector of the user does not match the motion information about the gaze object, the processor 220 may control the display 250 to change the position of the virtual object, thereby re-identifying whether the object attempting user verification is the actual user.

[0125] When the biometric scan fails (No in operation 1109) or the position of the gaze object is updated (operation 1117), the electronic device may randomly generate an image acquisition time in operation 1119. The image acquisition time may include a time for reacquiring a user image for user authentication.

[0126] In operation 1121, the electronic device may identify whether the randomly generated image acquisition time has arrived. When the image acquisition time has arrived, the electronic device may randomly generate an image acquisition sequence for user authentication in operation 1105. For example, the electronic device may repeatedly perform a series of operations for user authentication (e.g., operations 1105 to 1113 and operations 1117 to 1121).

[0127] When it is determined that the gaze vector of the user matches the motion information about the gaze object ("Yes" in operation 1113), the electronic device may determine that the user authentication is successful in operation 1115. For example, when it is determined that the gaze vector of the user matches the motion information about the gaze object, the processor 220 may determine that the biometric information for biometric verification has been obtained from the actual user. Therefore, the processor 220 may use the biometric information to determine that the user authentication is successful.

[0128] According to an embodiment, when user authentication fails a reference number of times or fails during a reference time, the electronic device may determine that the user authentication fails. In this case, the electronic device may output authentication failure related information in the form of any one of a message, a sound or an indicator (LED).

[0129] Fig.13A , Fig. 13B , Fig. 13C , Fig.13D and Fig.13E A diagram illustrating lighting time and image capturing time of a flood lighting device and a structure lighting device according to an embodiment.

[0130] Reference FIG. 13A to FIG. 13E , the electronic device 100 may randomly set the time of obtaining the image and the lighting time of the lighting device to enhance the security of user authentication using biometric information. The image acquisition time and the lighting time of the light may match each other. For example, the user's facial image may be obtained by the camera 270 when the floodlighting device 280-1 is lighting, and the depth image may be obtained by the camera 270 when the structured lighting device 280-2 is lighting.

[0131] According to the implementation mode, Fig.13A As shown, the electronic device 100 may alternately set the lighting time of the floodlighting device 280-1 and the lighting time of the structure lighting device 280-2. The intervals between the lighting times of the lighting devices may be set to be equal to each other (Δt1=Δt2=Δt3=Δt4=Δt5).

[0132] According to the implementation mode, Fig. 13B As shown, the electronic device 100 can set the lighting time of the lighting device so that the floodlighting device 280-1 or the structure lighting device 280-2 emits light continuously. As an example, the electronic device 100 can set the lighting time of the lighting device so that the floodlighting device 280-1 emits light twice continuously and the structure lighting device 280-2 emits light twice continuously. The intervals between the lighting times of the lighting devices can be set to be equal to each other (Δt1=Δt2=Δt3=Δt4=Δt5).

[0133] According to the implementation mode, Fig. 13C As shown, the electronic device 100 can set the lighting time of the lighting device so that the floodlighting device 280-1 or the structured lighting device 280-2 emits light continuously. As an example, the electronic device 100 can set the lighting time of the lighting device so that the floodlighting device 280-1 emits light twice continuously and the structured lighting device 280-2 emits light twice continuously. However, the intervals between the lighting times of the lighting devices can be set to be different from each other (Δt1≠Δt2≠Δt3≠Δt4≠Δt5).

[0134] According to the implementation mode, Fig.13DAs shown, the electronic device 100 may set the lighting time of the lighting device so that the same type of lighting device emits light stably until the burst shot is completed. As an example, the electronic device 100 may set the lighting time of the lighting device so as to maintain the lighting of the floodlighting device 280-1 or the structured lighting device 280-2 until the burst shot is completed. In order to obtain different images, the intervals between the lighting times of the lighting devices may be set to be different from each other (Δt1≠Δt2≠Δt3≠Δt4). The intervals between the lighting times of the lighting devices during burst shooting may be set to be equal to each other (Δt5=Δt6=Δt7=Δt8).

[0135] According to the implementation mode, Fig.13E As shown, the electronic device 100 can set the lighting time of the lighting device so that the same type of lighting device emits light stably until the continuous shooting is completed. In this case, the intervals between the lighting times of the lighting devices can be set to be different from each other (Δt1≠Δt2≠Δt3≠Δt4, Δt5≠Δt6≠Δt7≠Δt8).

[0136] According to an embodiment, when the floodlighting device and the structured lighting device continuously emit light based on the randomly generated image acquisition pattern, the electronic device 100 may have a delay in acquiring an image due to the switching time between the lighting devices. Fig.14 The illumination time of the illumination device is set as shown to minimize the delay in obtaining the image.

[0137] Fig.14 A diagram showing switching timing of cameras according to an embodiment.

[0138] Reference Fig.14 , the camera 270 may obtain a first image (e.g., a facial image) based on the illumination of the first illumination device (e.g., the floodlighting device 280-1). The first illumination device may terminate the illumination at time t1 when the shooting of the first image is completed. The second illumination device (e.g., the structured illumination device 280-2) may start emitting light at the same time when the illumination of the first illumination device is terminated at time t1 when the shooting of the first image is completed. If the protection period according to the illumination of the second illumination device has passed (t2), the camera 270 may start capturing the second image (e.g., a depth image).

[0139] According to an embodiment of the present disclosure, at least a portion of a device (e.g., a module or its function) or a method (e.g., an operation) may be implemented as instructions stored in a computer-readable storage medium (e.g., memory 230), for example, in the form of a program module. The instructions, when executed by a processor (e.g., processor 220), may enable the processor to perform the corresponding function. The computer-readable medium may include, for example, a hard disk, a floppy disk, a magnetic medium (e.g., a tape), an optical recording medium (e.g., a CD-ROM, a DVD), a magneto-optical medium (e.g., an optical disc), or an embedded memory. The instruction may include a code created by a compiler or a code executable by an interpreter. Modules or programming modules according to various embodiments of the present disclosure may include at least one or more of the aforementioned components, omit some of them, or further include other additional components.

[0140] According to various embodiments of the present invention, operations performed by a module, programming module or other components may be performed sequentially, in parallel, repeatedly or heuristically, or at least some operations may be performed in a different order or omitted, or other operations may be added.

[0141] As is apparent from the above description, according to an embodiment of the present disclosure, an electronic device and an operating method thereof may use a randomly emitting light source (e.g., a floodlight and a structured light) to obtain biometric information (e.g., iris information or facial information) for user authentication, thereby preventing other users (e.g., hackers) from pre-identifying the random time of obtaining the biometric information, thereby providing enhanced security for the biometric information.

[0142] According to an embodiment of the present disclosure, an electronic device and an operating method thereof can determine whether the biometric information has been obtained from an actual user based on changes in the user's gaze when performing user authentication based on biometric information obtained using randomly emitted lights (e.g., floodlights and structured lights), thereby preventing illegal user authentication using copied biometric information and thereby providing enhanced security for biometric data.

[0143] The embodiments disclosed herein are proposed for describing and understanding the disclosed technology, and do not limit the scope of the present disclosure. Therefore, the scope of the present disclosure should be interpreted as including all changes or various embodiments based on the technical spirit of the present disclosure.

[0144] Although the present disclosure has been described with various embodiments, various changes and modifications may be suggested to one skilled in the art. The present disclosure is intended to encompass such changes and modifications as fall within the scope of the appended claims.

Claims

1. An electronic device, include: Multiple lighting fixtures; camera; monitor; as well as Processor, configured as: displaying an object on the display, Based on the randomly generated lighting pattern, a random lighting time and a random lighting order of the plurality of lighting devices are identified, wherein the random lighting time is the duration for which the plurality of lighting devices emit light, controlling the plurality of lighting devices to emit light according to the identified random lighting time and the identified random lighting order, and controlling the camera to synchronously obtain a plurality of images while the plurality of lighting devices emit light, obtaining biometric information using at least one of the plurality of images, obtaining user gaze change information corresponding to a movement of the object displayed on the display using the plurality of images, and User authentication is performed on the user based on the biometric information and the user gaze change information.

2. The electronic device according to claim 1, in, The processor is configured to: based on the randomly generated lighting pattern, control a first lighting device among the plurality of lighting devices to emit light at a first time, and control a second lighting device among the plurality of lighting devices to emit light at a second time.

3. The electronic device according to claim 2, in, The processor is configured to: obtain, through the camera, an image of the user at the first time when the first lighting device emits light, and obtain a depth image at the second time when the second lighting device emits light.

4. The electronic device according to claim 3, in, The processor is configured to obtain the biometric information of the user using the image of the user.

5. The electronic device according to claim 3, in, The processor is configured to obtain the user gaze change information corresponding to the movement of the object displayed on the display using the image of the user and the depth image.

6. The electronic device according to claim 2, in, The first lighting device comprises a flood lighting device and the second lighting device comprises a structural lighting device.

7. The electronic device according to claim 2, in, The first illumination device and the second illumination device are configured to use infrared wavelengths.

8. The electronic device according to claim 1, in, The processor is configured to recognize that the user authentication is successful when a biometric scan using the biometric information is successful and the user gaze change information matches a movement of the object displayed on the display.

9. The electronic device according to claim 1, in, The processor is configured to identify that the user authentication has failed when a biometric scan using the biometric information fails or the user gaze change information does not match the movement of the object displayed on the display, or when a biometric scan using the biometric information fails and the user gaze change information does not match the movement of the object displayed on the display.

10. The electronic device according to claim 1, in, The processor is configured to display the object on at least a portion of the display upon detecting an occurrence of a user authentication event.

11. The electronic device according to claim 1, in, The biometric information includes at least one of iris information or facial information.

12. A method for operating an electronic device, the method include: displaying an object on a display of the electronic device; Based on the randomly generated lighting pattern, identifying random lighting time and random lighting order of multiple lighting devices, wherein the random lighting time is the duration of light emission of the multiple lighting devices; Controlling the plurality of lighting devices to emit light according to the identified random lighting time and the identified random lighting order, and synchronously obtaining a plurality of images while the plurality of lighting devices emit light; acquiring biometric information using at least one of the plurality of images; obtaining user gaze change information corresponding to the movement of the object displayed on the display using the plurality of images; as well as User authentication is performed on the user based on the biometric information and the user gaze change information.

13. The method according to claim 12, in, Displaying the object includes: displaying the object on at least a portion of the display when a user authentication event is detected, and Wherein, obtaining the multiple images comprises: obtaining an image of the user at a first time when a first lighting device among a plurality of lighting devices functionally connected to the electronic device emits light based on the randomly generated lighting pattern, and At a second time when a second lighting device among the plurality of lighting devices emits light, a depth image is obtained.

14. The method according to claim 13, in, Acquiring the biometric information includes: acquiring the biometric information of the user from the image of the user, Wherein, obtaining the user gaze change information comprises: using the image of the user and the depth image, obtaining the user gaze change information corresponding to the movement of the object displayed on the display, and Wherein, the first lighting device comprises a floodlighting device, and the second lighting device comprises a structural lighting device.

15. The method according to claim 12, in, Performing the user authentication includes: recognizing that the user authentication is successful when a biometric scan using the biometric information is successful and the user gaze change information matches a movement of the object displayed on the display, wherein performing the user authentication includes: identifying that the user authentication fails when the biometric scan performed using the biometric information fails or the user gaze change information does not match the movement of the object displayed on the display, or when the biometric scan performed using the biometric information fails and the user gaze change information does not match the movement of the object displayed on the display, and The biometric information includes at least one of iris information or facial information.

Images