Method and electronic device for connecting to external display device based on biometric information

Abstract

Embodiments of the invention may include a camera module, a communication module, a display, a memory, and a processor operably connected to at least one of the camera module, the communication module, the display, and the memory. The processor may be configured to: acquire a facial image of a user through the camera module when the external display device is worn based on a request to connect to the external display device; authenticating the face by comparing the acquired face image with the face information stored in the memory; displaying, on the display, an authentication code associated with the external display device and related to the stored facial information when the facial authentication is completed; receiving, in response to the request, a request to communicate with the electronic device from the external display device through the communication module based on an authentication result of the authentication code in the external display device; and connecting to an external display device in response to the request. Various embodiments are possible.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to a method for connecting to an external display device based on biometric information and an electronic device. Background Art

[0002] With the advancement of digital technology, various types of electronic devices such as mobile terminals, personal digital assistants (PDAs), electronic notebooks, smart phones, tablet personal computers (PCs), and wearable devices have been widely used. Electronic devices have been continuously improved in terms of hardware and / or software of the electronic devices to support and increase their functions.

[0003] For example, an electronic device (e.g., a smart phone) can establish a connection with a wearable display device (e.g., AR glasses) and provide expanded reality (XR) content, such as virtual reality (VR), augmented reality (AR), and / or mixed reality (MR). The electronic device implements an AR environment, including a tethered AR mode and an independent AR mode. In the tethered AR mode, virtual content generated by the electronic device is provided through the display of the wearable display device; in the independent AR mode, the wearable display device independently generates and displays virtual content through the display without being connected to the electronic device. Summary of the invention

[0004] Technical issues

[0005] The electronic device can execute an application for connecting to an external display device (or wearable display device) (e.g., AR glasses), and identify the ID of the external display device through the executed application. When the ID of the identified external display device is selected, the electronic device can receive a password displayed on the external display device from the user. When the input password is recognized, the electronic device can establish a connection with the external display device. Thereafter, whenever the electronic device is connected to the external display device, the user may be required to identify and select the ID and enter the password.

[0006] The embodiments may provide a method and device for relatively easily connecting an electronic device and an external display device by using biometric information in a state in which a user wears the external display device.

[0007] The technical problems solved by the present disclosure are not limited to the above-mentioned technical problems, and those skilled in the art will more clearly understand other technical problems from the following description.

[0008] Solution to the problem

[0009] According to an embodiment of the present disclosure, an electronic device may include a camera module, a communication module, a display, a memory and a processor operably connected to at least one of the camera module, the communication module, the display or the memory, wherein the processor is configured to: based on a request for connection with an external display device, acquire a facial image of a user through the camera module when the external display device is worn; perform facial authentication by comparing the acquired facial image with facial information stored in the memory; when facial authentication is completed, display an authentication code associated with the external display device associated with the stored facial information on the display; based on an authentication result of the authentication code in the external display device, receive a request for communication connection with the electronic device from the external display device through the communication module; and establish a connection with the external display device in response to the request.

[0010] According to an embodiment of the present disclosure, an electronic device may include a camera module, a communication module, a display, a memory, and a processor operably connected to at least one of the camera module, the communication module, the display, or the memory, wherein the processor is configured to detect whether the electronic device is worn on a user, capture an authentication code displayed on a display of an external electronic device through the camera module when the electronic device is worn on the user, authenticate the authentication code to extract device information of the external electronic device from the authentication code, send a request for connection to the external electronic device through the communication module based on the extracted device information, and establish a connection with the external electronic device in response to the request.

[0011] According to an embodiment of the present disclosure, an operating method of an electronic device may include: an operation of acquiring a facial image of a user through a camera module of the electronic device while the external display device is worn, based on a request for connection to an external display device; an operation of performing facial authentication by comparing the acquired facial image with facial information stored in a memory of the electronic device; an operation of displaying an authentication code associated with the external display device associated with the stored facial information on a display of the electronic device when facial authentication has been completed; an operation of receiving a request for communication connection with the electronic device from the external display device through a communication module of the electronic device based on an authentication result of the authentication code in the external display device; an operation of comparing iris information stored in a secure area of ​​the memory in response to the authenticated facial information so as to perform iris authentication; and an operation of establishing a connection with the external display device when iris authentication is completed.

[0012] Advantageous Effects of the Invention

[0013] According to an embodiment, the connection operation between the electronic device and the external display device can be relatively simplified by authenticating the user's face through the electronic device while the user is wearing the external display device, displaying an authentication code (for example, a barcode or a QR code) for authenticating the external display device on the display of the electronic device when the facial authentication is completed, and using the external display device to recognize the authentication code.

[0014] According to an embodiment, security when an electronic device and an external display device are connected can be improved by using device information of the external display device to match iris information acquired through iris authentication in the external display device and facial information acquired through facial authentication in the electronic device.

[0015] According to an embodiment, by connecting the electronic device and the external display device only when the user's face is authenticated by the electronic device and iris authentication is completed by the external display device (e.g., composite authentication), user convenience can be increased while maintaining relatively high security.

[0016] It will be understood by those skilled in the art that effects that can be achieved from the present disclosure are not limited to the above-described effects, and other effects not described above will be clearly understood from the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 is a block diagram of an electronic device in a network environment according to an embodiment.

[0018] Figure 2 is a view showing an example of an external display device according to an embodiment.

[0019] Figure 3 is a block diagram illustrating an external display device according to an embodiment.

[0020] Figure 4 is a signal flow illustrating a method for matching biometric information of an electronic device and an external display device according to an embodiment.

[0021] Figure 5 is a signal flow illustrating a method for connecting an electronic device and an external display device according to an embodiment.

[0022] Figure 6 is a view illustrating an example of recognizing an authentication code displayed on an electronic device through an external display device according to an embodiment.

[0023] Figure 7 is a flowchart illustrating an operating method of an electronic device according to an embodiment.

[0024] Figure 8 is a flowchart illustrating an operating method of an external display device according to an embodiment.

[0025] Fig. 9 is a signal flow illustrating a method for iris authentication before an electronic device and an external display device are connected according to an embodiment.

[0026] Fig.10 is a signal flow illustrating a method for iris authentication before an electronic device and an external display device are connected according to an embodiment.

[0027] Fig.11 is a flowchart illustrating an iris authentication method of an electronic device according to an embodiment.

[0028] Fig.12 is a flowchart illustrating an iris authentication method of an external display device according to an embodiment. DETAILED DESCRIPTION

[0029] Figure 1 is a block diagram illustrating an electronic device 101 in a network environment 100 according to various embodiments.

[0030] Reference Figure 1 , the electronic device 101 in the network environment 100 may communicate with the electronic device 102 via the first network 198 (e.g., a short-range wireless communication network), or communicate with at least one of the electronic device 104 or the server 108 via the second network 199 (e.g., a long-range wireless communication network). According to an embodiment, the electronic device 101 may communicate with the electronic device 104 via the server 108. According to an embodiment, the electronic device 101 may include a processor 120, a memory 130, an input module 150, a sound output module 155, a display module 160, an audio module 170, a sensor module 176, an interface 177, a connection terminal 178, a haptic module 179, a camera module 180, a power management module 188, a battery 189, a communication module 190, a user identification module (SIM) 196, or an antenna module 197. In some embodiments, at least one of the above components (e.g., the connection terminal 178) may be omitted from the electronic device 101, or one or more other components may be added to the electronic device 101. In some embodiments, some of the above-described components (eg, sensor module 176, camera module 180, or antenna module 197) may be implemented as a single integrated component (eg, display module 160) 11.

[0031] The processor 120 may run, for example, software (e.g., program 140) to control at least one other component (e.g., hardware component or software component) of the electronic device 101 connected to the processor 120, and may perform various data processing or calculations. According to an embodiment, as at least part of the data processing or calculation, the processor 120 may store a command or data received from another component (e.g., sensor module 176 or communication module 190) in the volatile memory 132, process the command or data stored in the volatile memory 132, and store the resultant data in the non-volatile memory 134. According to an embodiment, the processor 120 may include a main processor 121 (e.g., a central processing unit (CPU) or an application processor (AP)) or an auxiliary processor 123 (e.g., a graphics processing unit (GPU), a neural processing unit (NPU), an image signal processor (ISP), a sensor hub processor, or a communication processor (CP)) that is operationally independent or combined with the main processor 121. For example, when the electronic device 101 includes a main processor 121 and an auxiliary processor 123, the auxiliary processor 123 may be adapted to consume less power than the main processor 121, or adapted to be dedicated to a specific function. The auxiliary processor 123 may be implemented separately from the main processor 121, or as part of the main processor 121.

[0032] When the main processor 121 is in an inactive (e.g., sleep) state, the auxiliary processor 123 (rather than the main processor 121) may control at least some of the functions or states related to at least one component among the components of the electronic device 1011 (e.g., the display module 160, the sensor module 176, or the communication module 190), or when the main processor 121 is in an active state (e.g., running an application), the auxiliary processor 123 may control at least some of the functions or states related to at least one component among the components of the electronic device 101 (e.g., the display module 160, the sensor module 176, or the communication module 190) together with the main processor 121. According to an embodiment, the auxiliary processor 123 (e.g., an image signal processor or a communication processor) may be implemented as part of another component (e.g., a camera module 180 or a communication module 190) that is functionally related to the auxiliary processor 123. According to an embodiment, the auxiliary processor 123 (e.g., a neural processing unit) may include a hardware structure dedicated to artificial intelligence model processing. The artificial intelligence model may be generated by machine learning. For example, such learning may be performed by the electronic device 101 where the artificial intelligence is executed or via a separate server (e.g., server 108). The learning algorithm may include, but is not limited to, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, for example. The artificial intelligence model may include multiple artificial neural network layers. The artificial neural network may be a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), or a deep Q network or a combination of two or more thereof, but is not limited thereto. Additionally or alternatively, the artificial intelligence model may include a software structure in addition to a hardware structure.

[0033] The memory 130 may store various data used by at least one component of the electronic device 101 (e.g., the processor 120 or the sensor module 176). The various data may include, for example, software (e.g., the program 140) and input data or output data for commands related thereto. The memory 130 may include a volatile memory 132 or a nonvolatile memory 134.

[0034] The program 140 may be stored as software in the memory 130 , and may include, for example, an operating system (OS) 142 , middleware 144 , or applications 146 .

[0035] The input module 150 may receive commands or data to be used by other components (e.g., the processor 120) of the electronic device 101 from outside (e.g., a user) of the electronic device 101. The input module 150 may include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus).

[0036] The sound output module 155 can output sound signals to the outside of the electronic device 101. The sound output module 155 may include, for example, a speaker or a receiver. The speaker can be used for general purposes such as playing multimedia or playing records. The receiver can be used to receive incoming calls. Depending on the embodiment, the receiver can be implemented as a separate part from the speaker, or as part of the speaker.

[0037] The display module 160 may visually provide information to the outside of the electronic device 101 (e.g., a user). The display device 160 may include, for example, a display, a holographic device, or a projector, and a control circuit for controlling a corresponding one of the display, the holographic device, and the projector. According to an embodiment, the display module 160 may include a touch sensor adapted to detect a touch or a pressure sensor adapted to measure the strength of a force caused by a touch.

[0038] The audio module 170 may convert sound into an electrical signal, or vice versa. According to an embodiment, the audio module 170 may obtain sound via the input module 150, or output sound via the sound output module 155 or an earphone of an external electronic device (e.g., electronic device 102) directly (e.g., wired) or wirelessly connected to the electronic device 101.

[0039] The sensor module 176 may detect an operating state (e.g., power or temperature) of the electronic device 101 or an environmental state (e.g., a state of a user) outside the electronic device 101, and then generate an electrical signal or data value corresponding to the detected state. According to an embodiment, the sensor module 176 may include, for example, a gesture sensor, a gyro sensor, an atmospheric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an infrared (IR) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illumination sensor.

[0040] The interface 177 may support one or more specific protocols to be used to connect the electronic device 101 directly (e.g., wired) or wirelessly to an external electronic device (e.g., the electronic device 102). According to an embodiment, the interface 177 may include, for example, a high-definition multimedia interface (HDMI), a universal serial bus (USB) interface, a secure digital (SD) card interface, or an audio interface.

[0041] The connection end 178 may include a connector, wherein the electronic device 101 can be physically connected to an external electronic device (e.g., the electronic device 102) via the connector. According to an embodiment, the connection end 178 may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0042] The haptic module 179 may convert the electrical signal into mechanical stimulation (eg, vibration or motion) or electrical stimulation that can be recognized by the user via his sense of touch or kinesthetic sense. According to an embodiment, the haptic module 179 may include, for example, a motor, a piezoelectric element, or an electrical stimulator.

[0043] The camera module 180 may capture still images or moving images. According to an embodiment, the camera module 180 may include one or more lenses, an image sensor, an image signal processor, or a flash.

[0044] The power management module 188 may manage power supply to the electronic device 101. According to an embodiment, the power management module 188 may be implemented as, for example, at least a part of a power management integrated circuit (PMIC).

[0045] The battery 189 may power at least one component of the electronic device 101. According to an embodiment, the battery 189 may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0046] The communication module 190 may support establishing a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device 101 and an external electronic device (e.g., electronic device 102, electronic device 104, or server 108), and perform communication via the established communication channel. The communication module 190 may include one or more communication processors capable of operating independently from the processor 120 (e.g., an application processor (AP)) and supporting direct (e.g., wired) communication or wireless communication. According to an embodiment, the communication module 190 may include a wireless communication module 192 (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module 194 (e.g., a local area network (LAN) communication module or a power line communication (PLC) module). A corresponding one of these communication modules can communicate with an external electronic device via a first network 198 (e.g., a short-range communication network such as Bluetooth, Wireless Fidelity (Wi-Fi) Direct, or Infrared Data Association (IrDA)) or a second network 199 (e.g., a long-range communication network such as a traditional cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or a wide area network (WAN))). These various types of communication modules can be implemented as a single component (e.g., a single chip), or these various types of communication modules can be implemented as multiple components separated from each other (e.g., multiple chips). The wireless communication module 192 can identify and authenticate the electronic device 101 in a communication network (such as the first network 198 or the second network 199) using user information (e.g., an International Mobile Subscriber Identity (IMSI)) stored in the user identification module 196.

[0047] The wireless communication module 192 can support 5G networks after 4G networks and next-generation communication technologies (e.g., new radio (NR) access technology). NR access technology can support enhanced mobile broadband (eMBB), massive machine type communication (mMTC), or ultra-reliable low-latency communication (URLLC). The wireless communication module 192 can support high-frequency bands (e.g., millimeter wave bands) to achieve, for example, high data transfer rates. The wireless communication module 192 can support various technologies for ensuring performance on high-frequency bands, such as, for example, beamforming, massive multiple-input multiple-output (massive MIMO), full-dimensional MIMO (FD-MIMO), array antennas, analog beamforming, or massive antennas. The wireless communication module 192 can support various requirements specified in the electronic device 101, an external electronic device (e.g., electronic device 104), or a network system (e.g., a second network 199). According to an embodiment, the wireless communication module 192 may support a peak data rate for implementing eMBB (e.g., 20 Gbps or greater), loss coverage for implementing mMTC (e.g., 164 dB or less), or U-plane delay for implementing URLLC (e.g., 0.5 ms or less for each of the downlink (DL) and uplink (UL), or 1 ms or less round trip).

[0048] The antenna module 197 may transmit or receive a signal or power to or from the outside of the electronic device 101 (e.g., an external electronic device). According to an embodiment, the antenna module 197 may include an antenna including a radiating element formed of a conductive material or a conductive pattern formed in or on a substrate (e.g., a printed circuit board (PCB)). According to an embodiment, the antenna module 197 may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication scheme used in a communication network (such as the first network 198 or the second network 199) may be selected from the plurality of antennas by, for example, the communication module 190 (e.g., the wireless communication module 192). A signal or power may then be transmitted or received between the communication module 190 and the external electronic device via the selected at least one antenna. According to an embodiment, another component (e.g., a radio frequency integrated circuit (RFIC)) other than the radiating element may be additionally formed as part of the antenna module 197.

[0049] According to various embodiments, the antenna module 197 may form a millimeter wave antenna module. According to an embodiment, the millimeter wave antenna module may include a printed circuit board, a radio frequency integrated circuit (RFIC), and a plurality of antennas (e.g., array antennas), wherein the RFIC is disposed on a first surface (e.g., bottom surface) of the printed circuit board, or adjacent to the first surface and capable of supporting a specified high frequency band (e.g., millimeter wave band), and the plurality of antennas are disposed on a second surface (e.g., top surface or side surface) of the printed circuit board, or adjacent to the second surface and capable of transmitting or receiving signals of the specified high frequency band.

[0050] At least some of the above components may be connected to each other via an inter-peripheral communication scheme (e.g., a bus, a general purpose input output (GPIO), a serial peripheral interface (SPI), or a mobile industry processor interface (MIPI)) and communicatively transmit signals (e.g., commands or data) therebetween.

[0051] According to an embodiment, a command or data may be sent or received between the electronic device 101 and the external electronic device 104 via the server 108 connected to the second network 199. Each of the electronic device 102 or the electronic device 104 may be a device of the same type as the electronic device 101, or a device of a different type from the electronic device 101. According to an embodiment, all or some operations to be executed in the electronic device 101 may be executed in one or more of the external electronic device 102, the external electronic device 104, or the server 108. For example, if the electronic device 101 should automatically execute a function or service or should execute a function or service in response to a request from a user or another device, the electronic device 101 may request the one or more external electronic devices to execute at least part of the function or service instead of executing the function or service, or the electronic device 101 may request the one or more external electronic devices to execute at least part of the function or service in addition to executing the function or service. The one or more external electronic devices receiving the request may execute at least part of the function or service requested, or execute another function or another service related to the request, and transmit the result of the execution to the electronic device 101. The electronic device 101 may provide the result as at least a partial reply to the request in the case of further processing the result or in the case of not further processing the result. To this end, cloud computing technology, distributed computing technology, mobile edge computing (MEC) technology or client-server computing technology, for example, may be used. The electronic device 101 may use, for example, distributed computing or mobile edge computing to provide ultra-low latency services. In another embodiment, the external electronic device 104 may include an Internet of Things (IoT) device. The server 108 may be an intelligent server using machine learning and / or neural networks. According to an embodiment, the external electronic device 104 or the server 108 may be included in the second network 199. The electronic device 101 may be applied to intelligent services (e.g., smart homes, smart cities, smart cars or health care) based on 5G communication technology or IoT-related technologies.

[0052] Figure 2 is a view showing an example of an external display device according to an embodiment.

[0053] refer to Figure 2 , according to the external display device 201 (or wearable display device) of the embodiment (for example, Figure 1The electronic device 102 in the figure may include a display in the form of eyewear, and may include devices of various shapes that can be worn on a part of the user's body (e.g., face or head) and provide augmented reality (AR), mixed reality (MR) and / or virtual reality (VR) services. For example, the external display device 201 may be implemented in the form of at least one of glasses, goggles, a helmet, or a hat, but is not limited thereto. The external display device 201 described below may correspond to the device described above with reference to Figure 1 Even if not mentioned in the following description, the external display device 201 according to the present disclosure may be interpreted as including at least a part of the components included in the electronic device 101. Figure 1 Describe the various components.

[0054] According to an embodiment, the external display device 201 may provide an AR service for adding virtual information (or virtual objects) to at least a portion of an actual real space (or environment). For example, the external display device 201 may overlay virtual information on an actual real space corresponding to the field of view (FOV) of the wearer to provide it to the user.

[0055] The external display device 201 may include an eyeglass component (or window component) 210 disposed at positions corresponding to the user's eyes (e.g., left and right eyes), a main frame (or main body) 240 configured to fix the eyeglass component 210, a supporting frame (or supporting component) 250 connected to both ends of the main frame 240 and mounted on at least a portion of the user's ear, and a camera module 280 (e.g., a shooting camera).

[0056] According to an embodiment, the eyeglass member 210 may include a first eyeglass 220 corresponding to the left eye of the user and a second eyeglass 230 corresponding to the right eye of the user. According to an embodiment, the eyeglass member 210 may be supported by the main frame 240. For example, the eyeglass member 210 may be fitted into an opening provided on the main frame 240. According to an embodiment, from the display module (e.g., Figure 3 The AR image emitted by the display module 320 in the glasses can be projected onto the glasses component 210.

[0057] According to an embodiment, a waveguide (eg, an optical waveguide or a transparent waveguide) may be configured on at least a portion of the area of ​​the glasses member 210. According to an embodiment, the waveguide may be used to guide the AR image emitted from the display module to the user's eyes. According to an embodiment, although Figure 2As shown, it is described that the glasses member 210 is implemented in the form of being divided into the first glasses 220 and the second glasses 230 to correspond to the left eye and the right eye of the user respectively, according to some embodiments, the glasses member 210 can be implemented in the form of a single glasses without distinguishing the first glasses 220 and the second glasses 230.

[0058] According to an embodiment, the main frame 240 and the support frame 250 may be implemented in the form of glasses.

[0059] In an embodiment, the main frame 240 may have a structure that is at least partially mounted on the user's nose. According to an embodiment, the main frame 240 may support the eyeglass member 210. According to an embodiment, the main frame 240 may be made of a synthetic resin material. According to an embodiment, the eyeglass member 210 is assembled into an opening provided on the main frame 240 so that the main frame 240 can support the eyeglass member 210.

[0060] In an embodiment, the support frame 250 may include a first support frame 260 at least partially mounted on a first direction ear (e.g., a left ear) and a second support frame 270 at least partially mounted on a second direction ear (e.g., a right ear). For example, the main frame 240 and the support frame 250 (e.g., the first support frame 260 and the second support frame 270) may be connected by a hinge portion (not shown) and coupled to be foldable.

[0061] In an embodiment, the support frame 250 may be rotatably connected to the main frame 240. According to an embodiment, the support frame 250 may include a first support frame 260 and a second support frame 270. When viewed from direction "A", the first support frame 260 may be connected to the main frame 240 on the left side (e.g., the first direction) relative to the main frame 240. When viewed from direction "A", the second support frame 270 may be connected to the main frame 240 on the right side (e.g., the second direction) relative to the main frame 240.

[0062] In an embodiment, the support frame 250 may be mounted and fixed to a portion of the main frame 240. For example, a first support frame 260 connected to the left side of the main frame 240 and a second support frame 270 connected to the right side of the main frame 240 may be configured to be connected to each other. According to an embodiment, the support frame 250 connected to both sides of the main frame 240 may be annular and may be assembled to the user's head to be worn. In addition, the support frame 250 may be deformed into various forms so that the external display device 201 may be worn on the user's face.

[0063] According to an embodiment, the support frame 250 may be configured to be at least partially worn on the user's ear. For example, the external display device 201 may be worn on the user's face in a manner such that the support frame 250 connected to the main frame 240 is at least partially worn on the user's ear. In an embodiment, the support frame 250 may be rotated relative to the main frame 240. According to an embodiment, the support frame 250 rotates in a direction closer to the main frame 240, so that the volume of the external display device 201 can be reduced.

[0064] According to an embodiment, a display module (eg, Figure 3 The display module 320 in the electronic device 101 may output the information displayed by the processor 120 of the electronic device 101 or the processor of the external display device 201 (eg, Figure 3 390 in the display module). In the case where the display module generates an AR image and projects the AR image onto the glasses component 210, AR can be achieved by combining the visible light L incident from the front (e.g., the direction the user is watching) through the glasses component 210 with the objects included in the AR image. The display module may correspond to a projector (e.g., a micro projector or a PICO projector) of a relatively small size. For example, the display module may include a laser scanning display (LSD), a digital micro-mirror display (DMD), and / or liquid crystal on silicon (LCoS). According to an embodiment, the display module may include a transparent display. In this case, the light emitting element included in the display module may be directly disposed on the glasses component 210. In addition, the display module may include various display devices for implementing AR.

[0065] In an embodiment, the eyeglass member 210, the support frame 250 and / or the display module (e.g., Figure 3 The display module 320 in the embodiment may be provided in pairs to correspond to the left eye and the right eye of the user. For example, the eyeglass member 210 may include a first eyeglass 220 and a second eyeglass 230, and the support frame 250 may include a first support frame 260 and a second support frame 270. According to some embodiments, at least some of the above components may have different configurations for the left eye and different configurations for the right eye.

[0066] According to an embodiment, the camera module 280 may represent, for example, a shooting camera (e.g., a front shooting camera). For example, the camera module 280 may be provided and implemented on the main frame 240 to capture an object in front of the external display device 201 (or in front of the user facing). For example, the camera module 280 may be provided at a substantially central portion (or central point) between the first glasses 220 and the second glasses 230 in the main frame 240 to capture the front of the main frame 240. In an embodiment, the front of the main frame 240 may represent the direction in which the user faces when the user wears the external display device 201. In an embodiment, in addition to the camera module 280, the external display device 201 may further include a plurality of other cameras.

[0067] The camera module 280 may correspond to a camera (e.g., an RGB camera) for capturing an image corresponding to the user's field of view (FoV) and / or measuring the distance to an object. The camera module 280 may also include an eye tracking camera module for identifying the direction in which the user is looking. The camera module 280 may also include a recognition camera module (gesture camera module) for identifying a predetermined space.

[0068] The shooting camera may capture the front direction of the external display device 201, and the eye tracking camera may capture the direction opposite to the capture direction of the shooting camera. For example, the eye tracking camera may include a first eye tracking camera configured to partially capture the left eye of the user and a second eye tracking camera configured to partially capture the right eye of the user. According to an embodiment, the external display device 201 may display a virtual object (or virtual information) associated with the AR service based on image information associated with the actual real space acquired by the shooting camera. According to an embodiment, the external display device 201 may display a virtual object based on a display module set to correspond to both eyes of the user (e.g., a first display module corresponding to the left eye and / or a second display module corresponding to the right eye). According to an embodiment, the external display device 201 may display a virtual object based on pre-configured configuration information (e.g., resolution, frame rate, brightness, and / or display area).

[0069] According to an embodiment, the shooting camera may include a camera with high resolution, such as a high resolution (HR) camera and / or a photo video (PV) camera. For example, the shooting camera may be used to acquire high-quality images using autofocus and image stabilization (OIS). In addition to the color camera, the shooting camera may also be implemented as a global shutter (GS) camera and a rolling shutter camera (RS).

[0070] According to an embodiment, the eye tracking camera may detect the direction in which the user is looking (e.g., eye movement). For example, the eye tracking camera may detect the user's pupil to track the gaze direction. In an embodiment, the tracked gaze direction may be used to move the center of a virtual image including a virtual object according to the gaze direction. For example, the eye tracking camera may be a global shutter (GS) camera to detect the pupil and track fast eye movements without screen smear, and the performance and specifications of each eye tracking camera may be substantially the same.

[0071] According to an embodiment, the recognition camera may detect a user gesture within a preconfigured distance (e.g., a predetermined space) and / or a predetermined space. According to an embodiment, the recognition camera may be used for head tracking of 3 depth of field (3DoF) and 6DoF, as well as hand detection and / or hand tracking. For example, the recognition camera may be used for spatial recognition of 6DoF, and perform simultaneous localization and mapping (SLAM) functions through depth shooting. According to some embodiments, the recognition camera may be used for gesture recognition functions for recognizing user gestures. The recognition camera may include a camera including a global shutter (GS). For example, the recognition camera may include a camera including a GS with relatively small screen smear (or with reduced RS phenomenon), such as a rolling shutter (RS) camera, to detect and track fast gestures and / or fine movements of fingers.

[0072] According to an embodiment, the external display device 201 may use at least one camera to detect an eye corresponding to the dominant eye and / or the supplementary eye in the left eye and / or the right eye of the user. For example, the external display device 201 may detect an eye corresponding to the dominant eye and / or the non-dominant eye based on the gaze direction of the user relative to an external object or a virtual object.

[0073] The number or position of the camera module 280 (e.g., a photographing camera, an eye tracking camera, and / or a recognition camera) may not be limited. For example, the number or position of the camera module 280 (e.g., a photographing camera, an eye tracking camera, and / or a recognition camera) may be changed differently based on the form (e.g., shape or size) of the external display device 201.

[0074] According to an embodiment, the external display device 201 may include at least one lighting member (or lighting LED) to increase the accuracy of the camera module 280 (e.g., a photographing camera, an eye tracking camera, and / or a recognition camera). For example, a first lighting member may be provided at a portion corresponding to the user's left eye, and a second lighting member may be provided at a portion corresponding to the user's right eye.

[0075] According to an embodiment, the external display device 201 may further include a microphone (e.g., a first microphone and a second microphone) configured to receive the user's voice and environmental sounds. According to an embodiment, the external display device 201 may further include a first speaker configured to transmit an audio signal to the user's left ear and a second speaker configured to transmit an audio signal to the user's right ear.

[0076] Figure 3 is a block diagram illustrating an external display device according to an embodiment.

[0077] Reference Figure 3 According to an embodiment, the external display device 201 may include glasses 310 (eg, Figure 2 , a display module 320, a sensor module 330, a communication module 340, a memory 350, an audio module 360, a camera module 370 (eg, Figure 2 2 ), a camera module 280 in the external display device 201, a battery 380, and a processor 390. According to an embodiment, components included in the external display device 201 may be understood as, for example, hardware modules (eg, circuits).

[0078] According to an embodiment, glasses 310 (e.g., Figure 2 The first glasses 220 and / or the second glasses 230 in the glasses 310 may include a condensing lens and / or a transparent waveguide. For example, the transparent waveguide may be at least partially located at a portion of the glasses 310. According to an embodiment, light emitted from the display module 320 may be incident on one end of the glasses 310, and the incident light may be transmitted to the user through a waveguide disposed inside the glasses 310. The waveguide may be made of glass or a polymer, and may include a nano-pattern, for example, a grating structure having a polygonal or curved shape disposed on a surface inside or outside.

[0079] According to an embodiment, the display module 320 may include a plurality of panels (or display areas), and the plurality of panels may be located on the glasses 310. According to an embodiment, at least a portion of the display module 320 may be made of a transparent element, and the user may recognize the actual real space behind the rear surface of the display module 320 through the display module 320. According to an embodiment, the display module 320 may display a virtual object (or virtual information) on at least a partial area of ​​the transparent element to make the user feel that the virtual object has been added to at least a portion of the actual real space.

[0080] According to an embodiment, the sensor module 330 may include a proximity sensor 331, an illumination sensor 332, and / or a gyro sensor 333. The proximity sensor 331 may detect an object adjacent to the external display device 201. The illumination sensor 332 may measure the ambient brightness level of the external display device 201. According to an embodiment, the processor 390 may identify the brightness level around the external display device 201 by using the illumination sensor 332, and change the brightness-related configuration information of the display module 320 based on the brightness level. For example, when the ambient brightness is higher than the configured brightness, the processor 390 may configure the brightness level of the display module 320 to be higher to improve visibility to the user. The gyro sensor 333 may detect the state and position of the external display device 201. For example, the gyro sensor 333 may detect whether the external display device 201 is properly worn on the user's head. For another example, the gyro sensor may detect the movement of the external display device 201 or the user wearing the external display device 201.

[0081] The communication module 340 may correspond to Figure 1 According to an embodiment, the external display device 201 can communicate with an external device (e.g., Figure 1 The external display device 201 may perform wireless communication with the server 201 and / or other electronic devices 101, 102 and 104 in the external display device 201. For example, the external display device 201 may perform wireless communication with the electronic device 101 and exchange instructions and / or data with each other. According to some embodiments, the external display device 201 may be at least partially controlled by an external electronic device (e.g., the electronic device 101) through the communication module 340. For example, the external display device 201 may perform at least one function under the control of another external electronic device. According to an embodiment, the external display device 201 may send object information (e.g., an image) located in an actual real space acquired through the camera module 370, distance information to the object, user's gaze information and / or user's gesture information to an external device (e.g., Figure 1 The server 201 and / or other electronic devices 101, 102 and 104).

[0082] According to an embodiment, the communication module 340 may include an antenna module 345. For example, the communication module 340 may support various technologies (e.g., beamforming, multiple input / output (MIMO) and / or array antenna) to ensure performance in a specified frequency band. According to an embodiment, the antenna module 345 may send a signal or power to / receive a signal or power from the outside (e.g., a peripheral device and / or a server of the external display device 201). According to some embodiments, the antenna module 345 may include a plurality of antennas (e.g., an array antenna). According to an embodiment, a signal or power may be sent / received between the communication module 340 and the outside through the antenna module 345.

[0083] The memory 350 may correspond to the above reference Figure 1 According to an embodiment, the memory 350 may store various data used by the external display device 201 when the external display device 201 provides an AR service. The data may include, for example, software (e.g., Figure 1 The memory 350 may store instructions for the processor 390 to operate. The instructions may be stored in the memory 350 as software and may be executed by the processor 390.

[0084] The audio module 360 ​​may convert sound into an electric signal or convert an electric signal into sound based on the control of the processor 390 .

[0085] The camera module 370 may include at least one camera. According to an embodiment, the camera module 370 may include at least one of a photographing camera, an eye tracking camera, or a recognition camera.

[0086] The battery 380 may provide power to at least one component of the external display device 201. The battery 380 may include a plurality of batteries. For example, the battery 380 may include a battery configured to be connected to the first support frame (eg, Figure 2 A first battery powered by a first support frame 260 in the embodiment of the present invention and a first battery configured to supply power to a second support frame (e.g., Figure 2 A second battery powered by a second support frame 270).

[0087] The processor 390 may execute a program stored in the memory 350 (eg, Figure 1The processor 390 may be operable to provide AR services to the user based solely on the external display device 201 or based on the connection with the electronic device 101. Here, the connection may represent the pairing of the external display device 201 with the electronic device 101 via near-field wireless communication (e.g., Bluetooth). The processor 390 may output at least one virtual object (or virtual information) so that at least one virtual object is added and displayed in an actual real space (e.g., FOV and angle of view (AOV)) corresponding to the gaze of the user wearing the external display device 201 through the display module 320.

[0088] According to an embodiment, the processor 390 may correspond to the above reference Figure 1 According to an embodiment, the processor 390 may execute an application (eg, an AR application) (eg, Figure 1 The processor 390 may control the display module 320 to superimpose various digital contents (e.g., AR images) on the real world provided by the display module 320 to display an image (e.g., an AR screen).

[0089] According to an embodiment, the processor 390 may be included and operated in one of the first supporting frame 260 or the second supporting frame 270 of the external display device 201. According to some embodiments, the processor 390 may include a first processor (e.g., a main processor) and a second processor (e.g., an auxiliary processor) in the first supporting frame 260 and the second supporting frame 270 of the external display device 201, respectively, and may include a dual system structure that operates synchronously with the first processor and the second processor.

[0090] According to an embodiment of the present disclosure, an electronic device (eg, Figure 1 The electronic device 101 in the embodiment may include a camera module (eg, Figure 1 , a camera module 180 in the example, Figure 1 The communication module 190 in FIG. 1 ), a display (eg, Figure 1 display module 160 in the display module 160), a memory (e.g., Figure 1130 in the memory), and a processor (e.g., Figure 1 The processor 120 in the embodiment of the present invention is configured to: when the external display device is worn, based on the external display device (for example, Figure 2 a request for connection to an external display device 201 in the external display device 201), acquiring a facial image of the user through a camera module; performing facial authentication by comparing the acquired facial image with facial information stored in a memory; displaying an authentication code associated with the external display device associated with the stored facial information on a display; when facial authentication is completed, receiving a request for communication connection with the electronic device from the external display device 201 through a communication module based on an authentication result of the authentication code in the external display device 201; and establishing a connection with the external display device 201 in response to the request.

[0091] In the secure area of ​​the memory, the user's facial information and the user's iris information may be stored in association with the device information of the external display device.

[0092] The processor may be configured to analyze the acquired facial image to extract facial feature information, and determine whether the extracted facial feature information substantially matches the facial information stored in the secure area.

[0093] In the event that the extracted facial feature information substantially matches the facial information stored in the secure area, the processor may determine that facial authentication has succeeded, and in the event that the extracted facial feature information does not substantially match the facial information stored in the secure area, the processor may determine that facial authentication has failed.

[0094] The processor may be configured to generate an authentication code based on device information of the external display device stored in the secure area of ​​the memory in response to the stored facial information.

[0095] The authentication code may correspond to a service set identifier (SSID) of the external display device corresponding to the stored facial information and the SSID of the electronic device encrypted using a public key of the external display device.

[0096] The processor may be configured to receive a request for communication connection with the electronic device from the external display device through the SSID of the electronic device included in the authentication code, the external display device making the request by authenticating the displayed authentication code.

[0097] The processor may be configured to recognize iris information stored in a secure area of ​​the memory in response to the stored facial information, receive an iris image from the external display device via a communication channel established with the external display device, and compare the recognized iris information with the received iris image to perform iris authentication.

[0098] The processor may be configured to determine that iris authentication of the user has succeeded if iris feature information extracted from the iris image substantially matches the recognized iris information, and complete connection with the external display device if iris authentication has succeeded.

[0099] The processor may be configured to, if iris feature information extracted from the iris image does not substantially match the identified iris information, determine that iris authentication of the user has failed, if iris authentication of the user has failed, request an iris image obtained by recapturing the user's iris from an external display device, receive the iris image obtained by recapturing the user's iris according to the request, and perform iris authentication again.

[0100] According to an embodiment of the present disclosure, an electronic device (eg, Figure 2 The external display device 201 in the embodiment may include a camera module (eg, Figure 3 , a camera module 370 in the embodiment of the present invention, a communication module (e.g., Figure 3 The communication module 340 in the display (e.g., Figure 3 display module 320 in the display module 320), a memory (e.g., Figure 3 350 in the memory), and a processor (e.g., Figure 3 The processor 390 in the embodiment of the present invention is configured to detect whether the electronic device is worn on the user, and to capture the image displayed on the external electronic device (for example, Figure 1 The display of the electronic device 101 (eg, Figure 1 an authentication code on the display module 160 in the display module, authenticate the authentication code to extract device information of the external electronic device from the authentication code, send a request for connection with the external electronic device through the communication module based on the extracted device information, and establish a connection with the external electronic device in response to the request.

[0101] The processor may be configured to: in a state where the user is wearing the electronic device, if facial authentication of the user by the external electronic device has succeeded, capture the authentication code.

[0102] The authentication code may correspond to the SSID of the electronic device corresponding to the authenticated facial information of the user and the SSID of the external electronic device encrypted with the public key of the electronic device.

[0103] The processor may be configured to decrypt the authentication code using a private key of the electronic device to extract the SSID of the electronic device and the SSID of the external electronic device from the authentication code.

[0104] The processor may be configured to request a communication connection with the external electronic device through the SSID of the external electronic device extracted from the authentication code if the SSID of the electronic device extracted from the authentication code matches the SSID stored in the memory.

[0105] The processor may be configured to capture the user's iris through the camera module to obtain an iris image, receive iris information corresponding to the user's authenticated facial information from the external electronic device, and compare the acquired iris image with the received iris information to perform iris authentication.

[0106] The processor may be configured to determine that iris authentication of the user has succeeded if iris feature information extracted from the iris image substantially matches the received iris information, and complete connection with the external electronic device if iris authentication has succeeded.

[0107] The processor may be configured to determine that iris authentication of the user has failed if iris feature information extracted from the iris image does not substantially match the received iris information, and to recapture the user's iris to acquire an iris image if iris authentication has failed, and to compare the acquired iris image with the received iris information to perform iris authentication again.

[0108] Figure 4 is a signal flow illustrating a method for matching biometric information of an electronic device and an external display device according to an embodiment.

[0109] In the following embodiments, each operation may be performed sequentially, but it is not necessary to perform sequentially. For example, the sequential position of each operation may be changed, or at least two operations may be performed in parallel.

[0110] refer to Figure 4 In operation 401, an electronic device according to an embodiment (eg, Figure 1 The electronic device 101 in the embodiment may be connected to an external display device (eg, Figure 2Here, connection may refer to pairing of the electronic device 101 with the external display device 201 through near-field wireless communication. The external display device 201 (or wearable display device) may include a display in the shape of glasses, and may include devices of various shapes that can be worn on a part of the user's body and display content. For example, in the case where the electronic device 101 is initially connected to the external display device 201, a conventional method may be used to establish the connection. For example, in a conventional connection method, the electronic device 101 may execute an application for connecting to the external display device 201 based on the user's input (e.g., Figure 1 The electronic device 101 may select an application 146 in the external display device 201, and the ID of the external display device 201 may be identified by the executed application. In the case where the identified ID of the external display device 201 is selected, the electronic device 101 may receive a password displayed on the external display device 201 from the user. In the case where the input password is recognized, the electronic device 101 may establish a connection with the external display device 201.

[0111] Hereinafter, operations 403 to 409 may be performed by an application for connection between the electronic device 101 and the external display device 201 .

[0112] In operation 403, the electronic device 101 may capture the user's face while the external display device 201 is worn. However, in conventional technology, in order to authenticate the user's face, it is necessary to capture an image when the external display device 201 is not worn (e.g., when the external display device 201 is removed), and the present disclosure enables facial authentication of the user when the external display device 201 is worn. The electronic device 101 may drive the camera (e.g., Figure 1 The camera module 180 in the image processing module is used to capture the user's face.

[0113] In operation 405, the electronic device 101 may acquire facial information based on the captured facial image. The facial information may include facial feature information (eg, facial feature values) of the user. The electronic device 101 may analyze the captured facial image to acquire the facial information.

[0114] In operation 404, the external display device 201 may capture the user's iris. The external display device 201 may drive a camera module (eg, Figure 3 The camera module 370 in the image processing module is used to capture the user's iris.

[0115] In operation 406, the external display device 201 may acquire iris information based on the captured iris image. The iris information may include iris feature information (e.g., iris feature values) of the user. The external display device 201 may analyze the captured iris image to acquire iris information. According to an embodiment, the external display device 201 may send the captured iris image to the electronic device 101 (not shown), and the electronic device 101 may acquire iris information based on the iris image.

[0116] Operations 403 and 405 may be performed in parallel with operations 404 and 406, or may be performed sequentially. For example, operations 403 and 405 may be performed before operations 404 and 406, or after operations 404 and 406. This is merely an implementation issue, and the present disclosure is not limited thereto.

[0117] In operation 407, the external electronic device 201 may transmit the device information and iris information of the external display device 201 acquired in operation 406 to the electronic device 101. The device information may include a public key of the external display device 201 and a service set identifier (SSID) of the external display device 201. The device information may be required to match the external display device 201 with the iris information and the facial information. The external display device 201 is connected to the electronic device 101, and thus the device information and the iris information may be transmitted to the electronic device 101 through the configured communication channel (or communication session).

[0118] In operation 409, the electronic device 101 may match the iris information and the facial information with the device information and store them. The electronic device 101 may communicate with the user through a communication module (e.g., Figure 1 The electronic device 101 may pair the acquired iris and facial information of a single user with the device information of the external display device 201 and store it in a memory (e.g., Figure 1 According to an embodiment, the electronic device 101 may store one or more of iris information and facial information about a single user. The electronic device 101 may store the iris information and facial information paired with the device information in a secure area (e.g., secure world) of the memory 130.

[0119] Figure 5 is a signal flow showing a method for connecting an electronic device and an external display device according to an embodiment. Figure 4 After the electronic device and the external display device are connected to each other at least once, the flowchart may be executed Figure 5 .

[0120] In the following embodiments, each operation may be performed sequentially, but not necessarily sequentially. For example, the sequential position of each operation may be changed, or at least two operations may be performed in parallel.

[0121] refer to Figure 5 In operation 501, an electronic device according to an embodiment (eg, Figure 1 The electronic device 101 in the embodiment may be executed for communicating with an external display device (eg, Figure 2 An application (eg, Figure 1 The electronic device 101 may receive a request from a user to execute an application and execute the application.

[0122] In operation 503, the electronic device 101 may capture the user's face in a state where the user wears the external display device 201. For example, in the case where the connection with the external display device 201 is requested by the executed application, the electronic device 101 may drive a camera (e.g., Figure 1 The camera module 180 in the external display device 201 is used to capture the user's face. When capturing the user's face, the user can be in a state of wearing the external display device 201.

[0123] In operation 505, the electronic device 101 may analyze the captured facial image. The electronic device 101 may analyze the captured facial image to extract facial feature information. The electronic device 101 may determine whether the extracted facial feature information matches the facial feature information stored in the memory (e.g., Figure 1 The electronic device 101 may match the facial information in the secure area of ​​the memory 130 in the storage device 130. In the secure area of ​​the memory 130, the iris information and the facial information may be paired and stored with the device information of the external display device 201. The electronic device 101 may determine whether the extracted facial feature information substantially matches the facial information stored in the secure area.

[0124] In operation 507, the electronic device 101 may display the Figure 1 The authentication code may be displayed on the display module 160 in the secure area. In the event that the extracted facial feature information substantially matches the facial information stored in the secure area, the electronic device 101 may generate an authentication code. The authentication code may correspond to a code required for the electronic device 101 to authenticate the external display device 201. The authentication code may be generated as an encrypted code (e.g., a Quick Response (QR) code or a barcode). The authentication code may correspond to the SSID of the external display device 201 and the SSID of the electronic device 101 corresponding to the facial information encrypted using the public key of the external display device 201. The electronic device 101 may display the authentication code on the display (e.g., Figure 1 The generated authentication code is displayed on the display module 160).

[0125] In operation 509, the external display device 201 may capture the authentication code. The user may capture the authentication code displayed on the electronic device 101 by using the external display device 201 while wearing the external display device 201. For example, the external display device 201 may execute an application for connecting with the electronic device 101 and execute a camera module (e.g., Figure 3 The camera module 370 may correspond to a photographing camera configured to capture the front of the external display device 201.

[0126] In operation 511, the external display device 201 may extract the device information of the electronic device 101 included in the authentication code. The external display device 201 may decrypt the captured authentication code with the private key of the external display device 201. The authentication code is encrypted using the public key of the external display device 201, and thus may be decrypted using the private key of the external display device 201. The external display device 201 may decrypt the authentication code to extract the device information of the electronic device 101 included in the authentication code. The external display device 201 may decrypt the authentication code to extract the SSID of the external display device 201 and the SSID of the electronic device 101. The device information of the electronic device 101 may include the SSID of the electronic device 101.

[0127] In operation 513, the external electronic device 201 may attempt to connect to the electronic device 101 based on the device information of the electronic device 101. The external display device 201 may determine whether its SSID matches the SSID of the external display device 201 extracted from the authentication code. In the event that the SSID of the external display device matches the SSID of the external display device 201 extracted from the authentication code, the external display device 201 may request to establish a communication channel through the SSID of the electronic device 101.

[0128] In operation 515, the electronic device 101 may identify device information of the external display device 201. In the case where establishment of a communication channel has been requested from the external display device 201, the electronic device 101 may identify the SSID of the external display device 201 corresponding to the facial information. The electronic device 101 may determine whether the SSID of the external display device 201 that has requested establishment of a communication channel matches the SSID of the external display device 201 corresponding to the facial information. In the case where the SSID of the external display device 201 that has requested establishment of a communication channel matches the SSID of the external display device 201 corresponding to the facial information, the electronic device 101 may allow connection with the external display device 201 that has requested establishment of a communication channel.

[0129] According to an embodiment, the electronic device 101 may display an authentication code and then wait for a connection with the external display device 201 for a configured time (e.g., 1 minute or 5 minutes). The electronic device 101 may determine whether establishment of a communication channel has been requested from the external display device 201 within the configured time. In the case where establishment of a communication channel has been requested from the external display device 201 within the configured time, the electronic device 101 may determine whether the SSID of the external display device 201 that has requested establishment of a communication channel matches the SSID of the external display device 201 corresponding to the facial information. In the case where establishment of a communication channel has been requested from the external display device 201 after the configured time has passed, the electronic device 101 may not approve the connection with the external display device 201 that has requested establishment of a communication channel.

[0130] In operation 517, the electronic device 101 may establish a connection with the external display device 201. In the case where facial authentication is performed in a state where the user is wearing the external display device 201 and the facial authentication has succeeded (or has been completed), the electronic device 101 may display (or output) an authentication code to perform a connection operation with the external display device 201. According to an embodiment, the electronic device 101 may authenticate the iris of the user wearing the external display device 201 before connecting with the external display device 201 (for example, after operation 515). In the case where both facial authentication and iris authentication have succeeded (or have been completed) in a state where the user is wearing the external display device 201, the electronic device 101 may be connected to the external display device 201. Reference will be made to the following. Figures 9 to 12 Describe the iris authentication method in detail.

[0131] Figure 6 is a view illustrating an example of recognizing an authentication code displayed on an electronic device through an external display device according to an embodiment.

[0132] refer to Figure 6 , the user can wear an external display device (e.g., Figure 2 The state of capturing the display on the electronic device (eg, Figure 1 Reference numeral 601 may represent the line of sight of a user wearing the external display device 201, or may be a camera of the external display device 201 (eg, Figure 3 The external display device 201 may capture and identify the authentication code displayed on the electronic device 101 to connect with the electronic device 101.

[0133] Figure 7 is a flowchart 700 illustrating an operating method of an electronic device according to an embodiment.

[0134] In the following embodiments, each operation may be performed sequentially, but not necessarily sequentially. For example, the sequential position of each operation may be changed, or at least two operations may be performed in parallel.

[0135] According to an embodiment, it can be understood that operations 701 to 713 are performed by an electronic device (eg, Figure 1 A processor (eg, Figure 1 Executed by processor 120 in.

[0136] refer to Figure 7 In operation 701, an electronic device according to an embodiment (eg, Figure 1 A processor (eg, Figure 1 The processor 120 in the embodiment may execute a function for communicating with an external display device (eg, Figure 2 An application (eg, Figure 1 The external display device 201 (or wearable display device) may include a display in the shape of glasses, and may include devices of various shapes that can be worn on a part of the user's body and display content. Operation 701 may be performed after the electronic device 101 has been connected to the external display device 201 at least once. The processor 120 may receive a request to execute an application from the user and execute the application.

[0137] In operation 703, the processor 120 may use a camera (e.g., Figure 1 The processor 120 may capture the user's face through the camera module 180 when the user wears the external display device 201. The processor 120 may obtain the user's facial image when the user wears the external display device 201.

[0138] In operation 705, the processor 120 may authenticate the face. The processor 120 may compare the acquired facial image of the user with the facial image stored in the memory (e.g., Figure 1 The processor 120 may extract facial feature information from the acquired facial image of the user. The processor 120 may determine whether the extracted facial feature information corresponds to the facial information stored in the secure area. In the secure area, the iris information and the facial information may be paired and stored with the device information (e.g., a public key and an SSID) of the external display device 201. The processor 120 may determine whether the extracted facial feature information substantially matches the facial information stored in the secure area.

[0139] In the case where the extracted facial feature information substantially matches the facial information stored in the secure area, the processor 120 may determine that facial authentication has succeeded and perform operation 707. In the case where the extracted facial feature information does not substantially match the facial information stored in the secure area, the processor 120 may determine that facial authentication has failed and return to operation 703. When returning to operation 703, the processor 120 may recapture the user's face. According to an embodiment, in the case where facial authentication has failed even after performing facial authentication a configured number of times (e.g., 3 times, 5 times, etc.), the processor 120 may end the operation.

[0140] In the case where facial authentication is successful, in operation 707, the processor 120 may generate an authentication code corresponding to the facial information. The authentication code may correspond to a code required for the electronic device 101 to authenticate the external display device 101. The authentication code may be generated as an encrypted code (e.g., a QR code or a barcode). The authentication code may correspond to the SSID of the external display device 201 corresponding to the facial information and the SSID of the electronic device 101 encrypted using the public key of the external display device 201.

[0141] According to an embodiment, the processor 120 may generate an authentication code in advance and store the authentication code in a secure area of ​​the memory 130. The processor 120 may store the generated authentication code in response to facial information. In the case where the authentication code is stored in the secure area in response to facial information, the processor 120 may omit operation 707. In addition, in the case where the authentication code is stored in the secure area in response to facial information, the processor 120 may perform an operation of identifying the authentication code stored in the secure area instead of an operation of generating the authentication code. This is merely an implementation issue, and the present disclosure is not limited to this description.

[0142] In operation 709, the processor 120 may display the Figure 1The processor 120 may display the authentication code so that the external display device 201 recognizes the authentication code. The external display device 201 may capture the authentication code to extract the device information of the electronic device 101 included in the authentication code. The authentication code is encrypted using the public key of the external display device 201, and thus may be decrypted using the private key of the external display device 201. The external display device 201 may decrypt the authentication code to extract the device information of the electronic device 101 included in the authentication code. The external display device 201 may decrypt the authentication code to extract the SSID of the external display device 201 and the SSID of the electronic device 101. The device information of the electronic device 101 may include the SSID of the electronic device 101. The external electronic device 201 may attempt to connect to the electronic device 101 based on the device information of the electronic device 101. The external display device 201 may request to establish a communication channel by using the SSID of the electronic device 101.

[0143] In operation 711, the processor 120 may receive a connection request from the external display device 201. The processor 120 may receive a connection request from the external display device 201 through a communication module (eg, Figure 1 The processor 120 may further include a communication module 190 in the processor 120 to receive a connection request from the external display device 201. In the case where establishment of a communication channel has been requested from the external display device 201, the processor 120 may identify the SSID of the external display device 201 corresponding to the facial information. The processor 120 may determine whether the SSID of the external display device 201 that has requested establishment of a communication channel matches the SSID of the external display device 201 corresponding to the facial information. In the case where the SSID of the external display device 201 that has requested establishment of a communication channel matches the SSID of the external display device 201 corresponding to the facial information, the processor 120 may allow connection with the external display device 201 that has requested establishment of a communication channel.

[0144] According to an embodiment, the processor 120 may display an authentication code and then wait for a connection with the external display device 201 for a configured time (e.g., 1 minute or 5 minutes). The processor 120 may determine whether establishment of a communication channel has been requested from the external display device 201 within the configured time. In the case where establishment of a communication channel has been requested from the external display device 201 within the configured time, the processor 120 may determine whether the SSID of the external display device 201 that has requested establishment of a communication channel matches the SSID of the external display device 201 corresponding to the facial information. In the case where establishment of a communication channel has been requested from the external display device 201 after the configured time has passed, the processor 120 may not approve the connection with the external display device 201 that has requested establishment of a communication channel.

[0145] In operation 713, the processor 120 may establish a connection with the external display device 201. In the case where facial authentication is performed in a state where the user wears the external display device 201 and the facial authentication has succeeded (or has been completed), the processor 120 may display (or output) an authentication code to perform a connection operation with the external display device 201. According to an embodiment, the processor 120 may authenticate the iris of the user wearing the external display device 201 before connecting with the external display device 201 (for example, after operation 711). In the case where both facial authentication and iris authentication have succeeded (or have been completed) in a state where the user wears the external display device 201, the processor 120 may connect to the external display device 201. Reference will be made to the following. Figures 9 to 12 Describes the iris authentication method.

[0146] Figure 8 is a flowchart 800 illustrating an operating method of an external display device according to an embodiment.

[0147] In the following embodiments, each operation may be performed sequentially, but not necessarily sequentially. For example, the sequential position of each operation may be changed, or at least two operations may be performed in parallel.

[0148] According to an embodiment, it can be understood that operations 801 to 811 are performed by an electronic device (eg, Figure 2 A processor (eg, Figure 3 Processor 390 in executes.

[0149] refer to Figure 8 In operation 801, an external display device (eg, Figure 3 A processor (eg, Figure 2 The processor 390 in the embodiment may identify whether the user is wearing the external display device. For example, the processor 390 may detect whether the user is wearing the external display device through a sensor module (e.g., Figure 3 The processor 390 may determine whether the user is wearing the external display device 201 based on the sensing value acquired from the sensor module 330.

[0150] In operation 803, the processor 390 may capture the image displayed on the electronic device (eg, Figure 1 The user can capture the authentication code displayed on the electronic device 101 by using the external display device 201 while wearing the external display device 201. For example, the processor 390 can execute an application for connecting with the electronic device 101 based on the user input, and execute the camera module (for example, Figure 3The camera module 370 may correspond to a photographing camera configured to capture the front of the external display device 201.

[0151] In operation 805, the processor 390 may determine whether the authentication of the authentication code has succeeded. The authentication code is encrypted using the public key of the external display device 201, and thus may be decrypted using the private key of the external display device 201. The processor 390 may decrypt the authentication code using the private key of the external display device 201. In the case where the authentication code is decrypted using the private key of the external display device 201, the processor 390 may determine that the authentication has succeeded and perform operation 807. Alternatively, the processor 390 may determine whether the SSID of the processor matches the SSID of the external display device 201 extracted from the authentication code. In the case where the SSID of the processor matches the SSID of the external display device 201 extracted from the authentication code, the processor 390 may determine that the authentication has succeeded.

[0152] In the event that the authentication code is not decrypted with the private key of the external display device 201, the processor 390 may determine that the authentication has failed and return to operation 803. Alternatively, in the event that the SSID of the processor (e.g., the SSID stored in the memory of the external display device 201) does not match the SSID of the external display device 201 extracted from the authentication code, the processor 390 may determine that the authentication has failed. In the event of returning to operation 803, the processor 390 may recapture the authentication code displayed on the electronic device 101. According to an embodiment, in the event that the authentication code displayed on the electronic device 101 is not authenticated within a configured number of times, the processor 390 may end the connection operation with the electronic device 101.

[0153] In the case where the authentication of the authentication code has succeeded, the processor 390 may extract the device information of the electronic device 101 included in the authentication code at operation 807. The processor 390 may decrypt the authentication code to extract the SSID of the external display device 201 and the SSID of the electronic device 101. The device information of the electronic device 101 may include the SSID of the electronic device 101.

[0154] At operation 809, the processor 390 may send a connection request to the electronic device 101 based on the device information of the electronic device 101. The processor 390 may transmit a connection request to the electronic device 101 through a communication module (eg, Figure 3The communication module 340 in the electronic device 101 uses the SSID of the electronic device 101 to request (or attempt) connection with the electronic device 101 (for example, establish a communication channel). The electronic device 101 may display an authentication code and then wait for connection with the external display device 201 for a configured time (for example, 1 minute or 5 minutes). The electronic device 101 may determine whether a connection request has been received from the external display device 201 within the configured time. In the case where the connection request has been received from the external display device 201 within the configured time, the electronic device 101 may determine whether the SSID of the external display device 201 that has requested connection matches the SSID of the external display device 201 corresponding to the facial information. In the case where the SSID of the external display device 201 that has requested connection matches the SSID of the external display device 201 that corresponds to the facial information, the electronic device 101 may allow connection with the external display device 201 that has requested connection. In the case where establishment of a communication channel has been requested from the external display device 201 after the configured time has passed, the electronic device 101 may not approve the connection with the external display device 201 that has requested establishment of a communication channel.

[0155] In operation 811, the processor 390 may establish a connection with the electronic device 101. The processor 390 may establish a connection with the electronic device 101 if the electronic device 101 allows the connection request. According to an embodiment, the processor 390 may authenticate the iris of the user wearing the external display device 201 before connecting with the electronic device 101 (for example, before operation 811). In the case where both facial authentication and iris authentication have succeeded (or have been completed) in a state where the user is wearing the external display device 201, the processor 390 may connect to the electronic device 101. Figures 9 to 12 Describes the iris authentication method.

[0156] Fig. 9 is a signal flow illustrating a method for iris authentication before an electronic device and an external display device are connected according to an embodiment. Fig. 9 Can be Figure 5 Executed before operation 517 in, or included in operation 517 and executed.

[0157] In the following embodiments, each operation may be performed sequentially, but it is not necessary to perform sequentially. For example, the sequential position of each operation may be changed, or at least two operations may be performed in parallel.

[0158] refer to Fig. 9 In operation 901, an electronic device according to an embodiment (eg, Figure 1 The electronic device 101 in the embodiment and the external display device (eg, Figure 2The electronic device 101 may establish a communication channel with the external display device 201 before connecting (eg, pairing) with the external display device 201. The communication channel establishment may be used to exchange iris information for iris authentication.

[0159] In operation 903, the electronic device 101 may identify a data file stored in a memory (eg, Figure 1 The electronic device 101 may identify iris information corresponding to facial information for which facial authentication has succeeded in a state where the user wears the external display device 201.

[0160] In operation 905, the external display device 201 may capture the iris of the user. Since the user wears the external display device 201, the external display device 201 may capture the iris of the user by using an eye tracking camera (e.g., Figure 3 The camera module 370 in the display device 201 can capture the user's iris to identify the user's gaze direction. The external display device 201 can capture the user's iris and obtain the user's iris image.

[0161] In the drawings, operation 903 is described as being performed before operation 905, but operations 903 and 905 may be performed in parallel, or operation 903 may be performed after first performing operation 905. This is merely an implementation issue, and the description does not limit the present disclosure.

[0162] In operation 907, the electronic device 101 can communicate with the user through a communication module (eg, Figure 1 The electronic device 101 may send the recognized iris information to the external display device 201 through the communication channel established in the external display device 201. Operation 907 may be performed while operation 905 is performed.

[0163] In operation 909, the external display device 201 may authenticate the iris. For example, the external display device 201 may authenticate the iris through a communication module (e.g., Figure 3 The communication module 340 in the electronic device 101 receives iris information from the electronic device 101. The external display device 201 can analyze the acquired iris image to extract iris feature information. The external display device 201 can compare the iris feature information extracted from the iris image with the received iris information to authenticate the user's iris.

[0164] In operation 911, when the authentication of the user's iris has been completed, the electronic device 101 and the external display device 201 can be connected to each other. The external display device 201 can send an iris authentication result (e.g., authentication success) to the electronic device 101, and when the iris authentication result has been received, the electronic device 101 can allow the connection with the external display device 201.

[0165] Fig.10 is a signal flow illustrating a method for iris authentication before an electronic device and an external display device are connected according to an embodiment. Fig.10 Can be Figure 5 Executed before operation 517 in, or included in operation 517 and executed.

[0166] In the following embodiments, each operation may be performed sequentially, but it is not necessary to perform sequentially. For example, the sequential position of each operation may be changed, or at least two operations may be performed in parallel.

[0167] refer to Fig.10 In operation 1001, an electronic device according to an embodiment (eg, Figure 1 The electronic device 101 in the embodiment and the external display device (eg, Figure 2 The electronic device 101 may establish a communication channel with the external display device 201 before connecting (eg, pairing) with the external display device 201. The communication channel establishment may be used to exchange iris information for iris authentication. Operation 1001 may be performed with Fig. 9 The operation 901 in is the same or similar.

[0168] In operation 1003, the electronic device 101 may identify a data file stored in a memory (eg, Figure 1 The electronic device 101 may identify iris information corresponding to facial information for which facial authentication has succeeded when the user wears the external display device 201. Operation 1003 may be performed with Fig. 9 The operation 903 in is the same or similar.

[0169] In operation 1005, the external display device 201 may capture the iris of the user. Since the user wears the external display device 201, the external display device 201 may capture the iris of the user by using an eye tracking camera (e.g., Figure 3 The camera module 370 in the external display device 201 can capture the user's iris to identify the user's gaze direction. The external display device 201 can capture the user's iris and obtain the user's iris image. Operation 1005 can be performed with Fig. 9 The operation 905 in is the same or similar.

[0170] In the drawings, operation 1003 is described as being performed before operation 1005, but operations 1003 and 1005 may be performed in parallel, or operation 1003 may be performed after first performing operation 1005. This is merely an implementation issue, and the description does not limit the present disclosure.

[0171] In operation 1007, the external display device 201 can communicate with the display device 201 through the communication module (eg, Figure 3 The external display device 201 can send the acquired iris image to the electronic device 101 through the communication channel established in the electronic device 101.

[0172] In operation 1009, the electronic device 101 may authenticate the iris. For example, the electronic device 101 may authenticate the iris through a communication module (e.g., Figure 1 The communication module 190 in the memory 130 receives an iris image from the external display device 201. The electronic device 101 can analyze the acquired iris image to extract iris feature information. The electronic device 101 can compare the iris feature information extracted from the iris image with the iris information stored in the secure area of ​​the memory 130 to authenticate the user's iris.

[0173] In operation 1011, when the authentication of the user's iris has been completed, the electronic device 101 and the external display device 201 can be connected to each other. The electronic device 101 can send an iris authentication result (e.g., authentication success) to the external display device 201, and when the iris authentication result has been received, the external display device 201 can allow the connection with the electronic device 101.

[0174] Fig.11 1100 is a flowchart illustrating an iris authentication method of an electronic device according to an embodiment. Fig.11 Can be Figure 7 Executed before operation 713 in, or included in Figure 7 Operation 713 in is executed.

[0175] In the following embodiments, each operation may be performed sequentially, but it is not necessary to perform sequentially. For example, the sequential position of each operation may be changed, or at least two operations may be performed in parallel.

[0176] According to an embodiment, it can be understood that operations 1101 to 1109 are performed by an electronic device (e.g., Figure 1 A processor (eg, Figure 1 Executed by processor 120 in.

[0177] refer to Fig.11 In operation 1101, an electronic device according to an embodiment (eg, Figure 1A processor (eg, Figure 1 The processor 120 in the embodiment may be connected to an external display device (eg, Figure 2 The processor 120 may establish a communication channel with the external display device 201 through a communication module (eg, Figure 1 The communication module 190 in the embodiment of the present invention establishes a communication channel with the external display device 201. The communication channel establishment can be used to exchange iris information for iris authentication.

[0178] At operation 1103, the processor 120 may identify iris information corresponding to the facial information. The processor 120 may identify iris information stored in a memory (e.g., Figure 1 The processor 120 may identify iris information corresponding to facial information for which facial authentication has succeeded in a state where the user wears the external display device 201. The processor 120 may request an iris image acquired by capturing the user's iris from the external display device 201.

[0179] In operation 1105, the processor 120 may receive an iris image from the external display device 201 through the communication module 190. The processor 120 may receive an iris image acquired by capturing an iris of a user according to a request.

[0180] In operation 1107, the processor 120 may authenticate the iris. The processor 120 may analyze the received iris image to extract iris feature information. The processor 120 may compare the iris feature information extracted from the iris image with the iris information stored in the secure area of ​​the memory 130 to authenticate the user's iris. In the event that the iris feature information substantially matches the stored iris information, the processor 120 may determine that the authentication of the user's iris has succeeded. In the event that the iris feature information does not substantially match the stored iris information, the processor 120 may determine that the authentication of the user's iris has failed.

[0181] The processor 120 may perform operation 1109 if the iris authentication has succeeded, and return to operation 1105 if the iris authentication has failed. The processor 120 may return to operation 1105 to request an iris image acquired by recapturing the user's iris from the external display device 201, and receive an iris image acquired by capturing the user's iris according to the request.

[0182] In the case where iris authentication has succeeded, in operation 1109, the processor 120 may complete the connection with the external display device 201. Figure 7If both operation 705 in the example and iris authentication (eg, operation 1107) have been successful, the processor 120 may complete the connection with the external display device 201.

[0183] Fig.12 1200 is a flowchart illustrating an iris authentication method of an external display device according to an embodiment. Fig.12 Can be Figure 8 Executed before operation 811 in, or included in Figure 8 and execute in operation 811.

[0184] In the following embodiments, each operation may be performed sequentially, but it is not necessary to perform sequentially. For example, the sequential position of each operation may be changed, or at least two operations may be performed in parallel.

[0185] According to an embodiment, it can be understood that operations 1201 to 1209 are performed by an electronic device (e.g., Figure 2 A processor (eg, Figure 3 Processor 390 in executes.

[0186] refer to Fig.12 In operation 1201, an external display device (eg, Figure 2 A processor (eg, Figure 3 The processor 390 in the embodiment may be connected to an electronic device (e.g., Figure 1 The processor 390 can establish a communication channel through a communication module (for example, Figure 3 The communication module 340 in the embodiment of the present invention (hereinafter referred to as the communication module 340) uses the SSID of the electronic device 101 to request a connection with the electronic device 101 (for example, to establish a communication channel). The processor 390 may establish a communication channel with the electronic device 101 in response to the request before connecting (for example, pairing) with the electronic device 101. The communication channel establishment may be used to exchange iris information for iris authentication.

[0187] In operation 1203, the processor 390 may capture the user's iris. The processor 390 may capture the user's iris by using an eye tracking camera (e.g., Figure 3 The camera module 370 in the image processing unit 370 can be used to capture the user's iris to identify the user's gaze direction. The processor 390 can capture the user's iris and obtain the user's iris image.

[0188] At operation 1205, the processor 390 may communicate with the communication module (eg, Figure 3 The communication module 370 in the electronic device 101 receives iris information from the electronic device 101. The iris information may be stored in a secure area of ​​the electronic device 101 and may be recognized in response to the facial information.

[0189] In operation 1207, the processor 390 may authenticate the iris. The processor 390 may analyze the acquired iris image to extract iris feature information. The processor 390 may compare the iris feature information extracted from the iris image with the received iris information to authenticate the user's iris. In the event that the iris feature information substantially matches the received iris information, the processor 390 may determine that the authentication of the user's iris has succeeded. In the event that the iris feature information does not substantially match the received iris information, the processor 390 may determine that the authentication of the user's iris has failed.

[0190] The processor 390 may perform operation 1209 if the iris authentication has succeeded, and return to operation 1203 if the iris authentication has failed. The processor 390 may return to operation 1203, recapture the user's iris, and obtain a captured iris image.

[0191] In the event that iris authentication has been successful, at operation 1209, the processor 390 may complete the connection with the electronic device 101. The processor 390 may complete the connection with the electronic device 101 at operation 1209. Figure 7 Operation 705 in), authentication code authentication (for example, Figure 8 When both operation 805) and iris authentication (eg, operation 1207) have succeeded, the connection with the electronic device 101 is completed.

[0192] According to an embodiment of the present disclosure, an operating method of an electronic device may include: an operation of acquiring a facial image of a user through a camera module of the electronic device while the external display device is worn, based on a request for connection with an external display device; an operation of performing facial authentication by comparing the acquired facial image with facial information stored in a memory of the electronic device; an operation of displaying an authentication code associated with the external display device associated with the stored facial information on a display of the electronic device when facial authentication has been completed; an operation of receiving a request for communication connection with the electronic device from the external display device through a communication module of the electronic device based on an authentication result of the authentication code in the external display device; an operation of comparing iris information stored in a secure area of ​​the memory in response to the authenticated facial information to perform iris authentication; and an operation of establishing a connection with the external display device when iris authentication is completed.

[0193] In the secure area of ​​the memory, the user's facial information and the user's iris information may be stored in association with the device information of the external display device.

[0194] The authentication code may correspond to the SSID of the external display device and the SSID of the electronic device that are encrypted using a public key of the external display device corresponding to the stored facial information.

[0195] The electronic device according to various embodiments may be one of various types of electronic devices. The electronic device may include, for example, a portable communication device (e.g., a smart phone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a household appliance. According to an embodiment of the present disclosure, the electronic device is not limited to those electronic devices described above.

[0196] It should be understood that the various embodiments of the present disclosure and the terms used therein are not intended to limit the technical features set forth herein to specific embodiments, but include various changes, equivalent forms or alternative forms for corresponding embodiments. For the description of the accompanying drawings, similar reference numerals may be used to refer to similar or related elements. It will be understood that the nouns in the singular form corresponding to the term may include one or more things unless the relevant context clearly indicates otherwise. As used herein, each of the phrases such as "A or B", "at least one of A and B", "at least one of A or B", "A, B or C", "at least one of A, B and C" and "at least one of A, B or C" may include any one or all possible combinations of the items listed together with the corresponding one of the multiple phrases. As used herein, terms such as "1st" and "2nd" or "first" and "second" may be used to simply distinguish the corresponding component from another component, and do not limit the component in other aspects (e.g., importance or order). It will be understood that if an element (e.g., a first element) is referred to as being “combined with another element (e.g., the second element)”, “combined to another element (e.g., the second element)”, “connected with another element (e.g., the second element)”, or “connected to another element (e.g., the second element)” when the terms “operably” or “communicatively” are used or when the terms “operably” or “communicatively” are not used, it means that the element may be directly (e.g., wired) connected to the other element, wirelessly connected to the other element, or connected to the other element via a third element.

[0197] As used in connection with various embodiments of the present disclosure, the term "module" may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with other terms (e.g., "logic," "logic block," "portion," or "circuit"). A module may be a single integrated component adapted to perform one or more functions or a minimum unit or portion of the single integrated component. For example, according to an embodiment, a module may be implemented in the form of an application specific integrated circuit (ASIC).

[0198] The various embodiments described herein may be implemented as software (e.g., program 140) including one or more instructions stored in a storage medium (e.g., internal memory 136 or external memory 138) that can be read by a machine (e.g., electronic device 101). For example, under the control of a processor, a processor (e.g., processor 120) of the machine (e.g., electronic device 101) may call at least one of the one or more instructions stored in the storage medium and execute the at least one instruction with or without the use of one or more other components. This enables the machine to operate to perform at least one function according to the at least one instruction called. The one or more instructions may include code generated by a compiler or code that can be run by an interpreter. A machine-readable storage medium may be provided in the form of a non-transitory storage medium. Among them, the term "non-transitory" only means that the storage medium is a tangible device and does not include a signal (e.g., an electromagnetic wave), but the term does not distinguish between data being semi-permanently stored in a storage medium and data being temporarily stored in a storage medium.

[0199] According to an embodiment, the method according to various embodiments of the present disclosure may be included and provided in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be published in the form of a machine-readable storage medium (e.g., a compact disk read-only memory (CD-ROM)), or may be published online (e.g., downloaded or uploaded) via an application store (e.g., Play StoreTM), or may be distributed (e.g., downloaded or uploaded) directly between two user devices (e.g., smart phones). If published online, at least part of the computer program product may be temporarily generated, or at least part of the computer program product may be at least temporarily stored in a machine-readable storage medium (such as a manufacturer's server, an application store's server, or a memory of a forwarding server).

[0200] According to various embodiments, each component (e.g., module or program) in the above-mentioned components may include a single entity or multiple entities, and some entities in the multiple entities may be separately arranged in different components. According to various embodiments, one or more components in the above-mentioned components may be omitted, or one or more other components may be added. Alternatively or additionally, multiple components (e.g., module or program) may be integrated into a single component. In this case, according to various embodiments, the integrated component may still perform the one or more functions of each component in the multiple components in the same or similar manner as the corresponding one component in the multiple components before integration. According to various embodiments, the operations performed by a module, program or another component may be performed sequentially, in parallel, repeatedly or in a heuristic manner, or one or more operations in the operations may be run or omitted in different orders, or one or more other operations may be added.

[0201] The various embodiments of the present disclosure in this specification and the accompanying drawings are only specific examples presented to easily explain the technical ideas of the present disclosure and help understand the present disclosure, and are not intended to limit the scope of the present disclosure. Therefore, in addition to the embodiments disclosed herein, the scope of the present disclosure should be interpreted as including all changes or modifications derived from the technical ideas of the present disclosure.

Claims

1. An electronic device, comprising: Camera module; Communication module; monitor; Memory; and a processor operably connected to at least one of the camera module, the communication module, the display or the memory, Wherein, the processor is configured to: Based on a request for connection with an external display device, acquiring a facial image of a user through the camera module while the external display device is worn; performing facial authentication by comparing the acquired facial image with facial information stored in the memory; In the event that the facial authentication is completed, displaying on the display an authentication code associated with the external display device associated with the stored facial information; Based on the authentication result of the authentication code in the external display device, receiving a request for communication connection with the electronic device from the external display device through the communication module; and In response to the request, a connection with the external display device is established.

2. The electronic device according to claim 1, wherein: In the secure area of ​​the memory, the user's facial information and the user's iris information are stored in association with device information of the external display device.

3. The electronic device according to claim 2, wherein: The processor is configured to: analyzing the acquired facial image to extract facial feature information; and A determination is made as to whether the extracted facial feature information substantially matches facial information stored in the secure area.

4. The electronic device according to claim 3, wherein: The processor is configured to: determining that the facial authentication has succeeded if the extracted facial feature information substantially matches the facial information stored in the secure area; and In the event that the extracted facial feature information does not substantially match the facial information stored in the secure area, it is determined that the facial authentication has failed.

5. The electronic device according to claim 1, wherein: The processor is configured to generate the authentication code based on device information of the external display device stored in a secure area of ​​the memory in association with the stored facial information.

6. The electronic device according to claim 1, wherein: The authentication code is acquired by encrypting a service set identifier SSID of the external display device corresponding to the stored facial information and the SSID of the electronic device using a public key of the external display device.

7. The electronic device according to claim 1, wherein: The processor is configured to receive a request for communication connection with the electronic device from the external display device through the SSID of the electronic device included in the authentication code, the external display device making the request by authenticating the displayed authentication code.

8. The electronic device according to claim 1, wherein: The processor is configured to: identifying iris information stored in a secure area of ​​the memory in association with the stored facial information; receiving an iris image from the external display device via a communication channel established with the external display device; and The recognized iris information is compared with the received iris image to perform iris authentication.

9. The electronic device according to claim 8, wherein: The processor is configured to: In the case where the iris feature information extracted from the iris image substantially matches the identified iris information, determining that the iris authentication of the user has succeeded; and In case the iris authentication has succeeded, the connection with the external display device is completed.

10. The electronic device according to claim 8, wherein: The processor is configured to: determining that iris authentication of the user has failed if the iris feature information extracted from the iris image does not substantially match the identified iris information; requesting, in a case where the iris authentication has failed, an iris image acquired by re-capturing the user's iris from the external display device; and An iris image acquired by re-capturing the user's iris according to the request is received, and iris authentication is performed again.

11. An electronic device comprising: Camera module; Communication module; monitor; Memory; and a processor operably connected to at least one of the camera module, the communication module, the display or the memory, Wherein, the processor is configured to: Detecting whether the electronic device is worn by a user; capturing, by the camera module, an authentication code displayed on a display of an external electronic device in a state where the electronic device is worn on the user; authenticating the authentication code to extract device information of the external electronic device from the authentication code; Based on the extracted device information, sending a request for connection with the external electronic device through the communication module; and In response to the request, a connection with the external electronic device is established.

12. The electronic device according to claim 11, wherein: The processor is configured to: capture the authentication code if the external electronic device has successfully authenticated the user's face while the user is wearing the electronic device, and The authentication code is obtained by encrypting the service set identifier SSID of the electronic device and the SSID of the external electronic device corresponding to the facial information of the authenticated user using the public key of the electronic device.

13. The electronic device according to claim 12, wherein: The processor is configured to: decrypting the authentication code using a private key of the electronic device to extract the SSID of the electronic device and the SSID of the external electronic device from the authentication code; and In a case where the SSID of the electronic device extracted from the authentication code matches the SSID stored in the memory, a communication connection with the external electronic device is requested through the SSID of the external electronic device extracted from the authentication code.

14. The electronic device according to claim 11, wherein: The processor is configured to: capturing the user's iris through the camera module to obtain an iris image; receiving, from the external electronic device, iris information corresponding to the authenticated facial information of the user; comparing the acquired iris image with the received iris information to perform iris authentication; In the case where the iris feature information extracted from the iris image substantially matches the received iris information, determining that the iris authentication of the user has succeeded; and In case the iris authentication has been successful, the connection with the external electronic device is completed.

15. A method for operating an electronic device, the method comprising: Based on a request for connection with an external display device, acquiring a facial image of a user through a camera module of the electronic device while the external display device is worn; performing facial authentication by comparing the acquired facial image with facial information stored in a memory of the electronic device; displaying, on a display of the electronic device, an authentication code associated with the external display device associated with the stored facial information, in the event that the facial authentication has been completed; Based on the authentication result of the authentication code in the external display device, receiving a request for communication connection with the electronic device from the external display device through the communication module of the electronic device; comparing iris information stored in a secure area of ​​the memory in association with the authenticated facial information to perform iris authentication; and When the iris authentication is completed, a connection with the external display device is established.

Images