Wearable device and method for controlling light emission from light source and non-transitory computer-readable storage medium
The wearable device's advanced light source control and image capture system addresses the limitations of current AR devices by enabling precise eye tracking and iris recognition, enhancing user experience and authentication accuracy.
Patent Information
- Application Number
- PCT/KR2024/005424
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-08-30
- Filing Date
- 2024-04-22
- Publication Date
- 2025-01-09
AI Technical Summary
Current wearable devices for augmented reality lack efficient control over light sources for eye tracking and iris recognition, which limits their ability to provide enhanced user experiences and accurate biometric authentication.
A wearable device equipped with a processor, memory, and multiple light sources that can be controlled to emit light towards the user's eyes for eye tracking and iris recognition, using a camera to capture images and adjust the light sources' emission based on captured images for precise tracking and recognition.
Enables improved eye tracking and iris recognition capabilities, enhancing user experience and authentication accuracy by dynamically controlling light sources based on captured images.
Smart Images

Figure KR2024005424_09012025_PF_FP_ABST
Abstract
Description
Wearable device, method, and non-transitory computer-readable storage medium for controlling the emission of a light source
[0001] The disclosure relates to a wearable device, a method, and a non-transitory computer readable storage medium for controlling the emission of a light source.
[0002] To provide an enhanced user experience, electronic devices are being developed that provide augmented reality (AR) services, which display computer-generated information in conjunction with external objects in the real world. These electronic devices may be wearable devices worn by the user. For example, these electronic devices may be AR glasses and / or head-mounted devices (HMDs).
[0003] The above information is provided solely as background information to aid in understanding the disclosure. No claim or determination is made as to whether any of the above constitutes prior art with respect to the disclosure.
[0004] The disclosed aspects are intended to address at least the problems and / or drawbacks mentioned above and provide at least the advantages described below. Accordingly, the disclosed aspects are intended to provide a wearable device, method, and non-transitory computer-readable storage medium for controlling the emission of a light source.
[0005] Additional aspects will be partly described in the following description, partly will be apparent from the description, or may be learned by practice of the embodiments presented.
[0006] A wearable device is provided. The wearable device may include a plurality of light sources configured to emit light toward the eyes of a user wearing the wearable device. The wearable device may include a camera configured to capture images of at least a portion of the eye of the user wearing the wearable device. The wearable device may include at least one processor including processing circuitry. The wearable device may include a memory storing instructions, the memory including one or more storage media. The instructions, when individually or collectively executed by the processor, may cause the wearable device to perform eye tracking based on the images captured by the camera while the plurality of light sources are controlled to emit light toward the eyes of the user. The instructions, when individually or collectively executed by the processor, may cause the wearable device to perform iris recognition based on at least one image captured by the camera while some of the plurality of light sources are controlled to shine toward the eye of the user and while other some of the plurality of light sources are deactivated.
[0007] A method is provided, performed by a wearable device, comprising a plurality of light sources configured to emit light toward an eye of a user wearing the wearable device, and a camera configured to capture images of at least a portion of the eye of the user wearing the wearable device. The method may include performing eye tracking based on the images captured by the camera while the plurality of light sources are controlled to emit light toward the eye of the user. The method may include performing iris recognition based on at least one image captured by the camera while some of the plurality of light sources are controlled to emit light toward the eye of the user and other some of the plurality of light sources are deactivated.
[0008] A non-transitory computer-readable storage medium is provided. The non-transitory computer-readable storage medium can store one or more programs, including instructions. The instructions, when executed by a processor of a wearable device, the wearable device including a plurality of light sources configured to emit light toward an eye of a user wearing the wearable device, and a camera configured to capture images of at least a portion of the eye of the user wearing the wearable device, can cause the wearable device to perform eye tracking based on the images captured by the camera while the plurality of light sources are controlled to emit light toward the eye of the user. The instructions, when executed by the processor, can cause the wearable device to perform iris recognition based on at least one image captured by the camera while some of the plurality of light sources are controlled to emit light toward the eye of the user and other some of the plurality of light sources are deactivated.
[0009] Other aspects, advantages, and salient features of the disclosure will become apparent to those skilled in the art from the following detailed description, which sets forth various embodiments of the disclosure together with the accompanying drawings.
[0010] The above and other aspects, features, and advantages of specific embodiments of the disclosure will become more apparent from the following description taken in conjunction with the accompanying drawings.
[0011] FIG. 1 is a block diagram of an electronic device within a network environment according to one embodiment of the disclosure.
[0012] FIG. 2A illustrates an example of an appearance of a first side of a wearable device according to one embodiment of the disclosure.
[0013] FIG. 2b illustrates an example of an appearance of a second side of a wearable device according to one embodiment of the disclosure.
[0014] FIG. 2c schematically illustrates an example of a structure of a wearable device according to an embodiment of the disclosure.
[0015] FIG. 2d schematically illustrates an example of a structure for eye tracking (ET) of a wearable device according to an embodiment of the disclosure.
[0016] FIG. 2e schematically illustrates an example of a layout structure of light sources of a wearable device according to an embodiment of the disclosure.
[0017] FIG. 2f schematically illustrates an example of a circuit structure of light sources of a wearable device according to an embodiment of the disclosure.
[0018] FIG. 3A is a simplified block diagram of a wearable device according to one embodiment of the disclosure.
[0019] FIG. 3b is a simplified circuit diagram of a wearable device according to one embodiment of the disclosure.
[0020] FIG. 4A schematically illustrates an example of a circuit structure of light sources of a wearable device according to an embodiment of the disclosure.
[0021] FIG. 4b schematically illustrates an example of a circuit structure of light sources of a wearable device according to an embodiment of the disclosure.
[0022] FIG. 5A schematically illustrates an example of a layout structure of light sources of a wearable device according to an embodiment of the disclosure.
[0023] FIG. 5b schematically illustrates an example of a layout structure of light sources of a wearable device according to an embodiment of the disclosure.
[0024] FIG. 5c schematically illustrates an example of a layout structure of light sources of a wearable device according to an embodiment of the disclosure.
[0025] FIG. 5d schematically illustrates an example of a layout structure of light sources of a wearable device according to an embodiment of the disclosure.
[0026] FIG. 6 schematically illustrates an example of angles of light sources of a wearable device according to an embodiment of the disclosure.
[0027] FIG. 7 is a flowchart of a method executed on a wearable device according to an embodiment of the disclosure.
[0028] FIG. 8 is a flowchart of a method executed on a wearable device according to an embodiment of the disclosure.
[0029] FIG. 9 is a flowchart of a method executed on a wearable device according to an embodiment of the disclosure.
[0030] FIG. 10A illustrates an example of an appearance of a first side of a wearable device according to one embodiment of the disclosure.
[0031] FIG. 10b illustrates an example of an appearance of a second side of a wearable device according to an embodiment of the disclosure.
[0032] It should be noted that similar reference numbers are used throughout the drawings to describe identical or similar elements, features and structures.
[0033] The following description, with reference to the attached drawings, is provided to facilitate a comprehensive understanding of various embodiments of the disclosure defined by the claims and their equivalents. While numerous specific details are included to facilitate this understanding, they are to be considered merely exemplary. Accordingly, those skilled in the art will appreciate that various modifications and variations can be made to the various embodiments described herein without departing from the scope and spirit of the disclosure. Furthermore, descriptions of well-known functions and configurations may be omitted for clarity and brevity.
[0034] The terms and words used in the following description and claims are not limited to their bibliographic meanings and are merely used by the inventors to facilitate a clear and consistent understanding of the disclosure. Therefore, it will be apparent to those skilled in the art that the following description of various embodiments of the disclosure is provided for illustrative purposes only and is not intended to limit the disclosure as defined by the appended claims and their equivalents.
[0035] The singular forms "a," "an," and "the" should be understood to include the plural unless the context clearly indicates otherwise. Thus, for example, reference to "a surface of a part" includes reference to one or more of those surfaces.
[0036] It should be understood that the blocks and combinations of flowcharts in each flowchart can be executed by one or more computer programs containing computer-executable instructions. The one or more computer programs may be stored entirely in a single memory device, or the one or more computer programs may be divided into different portions and stored in different memory devices.
[0037] Any of the functions or operations described herein may be processed by a single processor or a combination of processors. A single processor or a combination of processors is a circuit that performs processing, and may be an application processor (AP, e.g., a central processing unit (CPU)), a communication processor (CP, e.g., a modem), a graphical processing unit (GPU), a neural processing unit (NPU) (e.g., an artificial intelligence (AI) chip), a wireless-fidelity chip (Wi-Fi), or a Bluetooth chip. TM Chips, Global Positioning System (GPS) chips, near field communication (NFC) chips, connectivity chips, sensor controllers, touch controllers, fingerprint sensor controllers, display driver integrated circuits (ICs), audio codec chips, universal serial bus (USB) controllers, camera controllers, image processing ICs, microprocessor units (MPUs), system on a chip (SoCs), or ICs.
[0038] FIG. 1 is a block diagram of an electronic device within a network environment according to one embodiment of the disclosure.
[0039] Referring to FIG. 1, in a network environment (100), an electronic device (101) may communicate with an electronic device (102) via a first network (198) (e.g., a short-range wireless communication network), or may communicate with at least one of an electronic device (104) or a server (108) via a second network (199) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (101) may communicate with the electronic device (104) via the server (108). According to one embodiment, the electronic device (101) may include a processor (120), a memory (130), an input module (150), an audio output module (155), a display module (160), an audio module (170), a sensor module (176), an interface (177), a connection terminal (178), a haptic module (179), a camera module (180), a power management module (188), a battery (189), a communication module (190), a subscriber identification module (196), or an antenna module (197). In some embodiments, the electronic device (101) may omit at least one of these components (e.g., the connection terminal (178)), or may have one or more other components added. In some embodiments, some of these components (e.g., the sensor module (176), the camera module (180), or the antenna module (197)) may be integrated into one component (e.g., the display module (160)).
[0040] The processor (120) may, for example, execute software (e.g., a program (140)) to control at least one other component (e.g., a hardware or software component) of the electronic device (101) connected to the processor (120) and perform various data processing or operations. According to one embodiment, as at least a part of the data processing or operations, the processor (120) may store commands or data received from other components (e.g., a sensor module (176) or a communication module (190)) in a volatile memory (132), process the commands or data stored in the volatile memory (132), and store result data in a non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., a central processing unit or an application processor) or an auxiliary processor (123) (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor) that can operate independently or together with the main processor (121). For example, when the electronic device (101) includes the main processor (121) and the auxiliary processor (123), the auxiliary processor (123) may be configured to use less power than the main processor (121) or to be specialized for a given function. The auxiliary processor (123) may be implemented separately from the main processor (121) or as a part thereof.
[0041] The auxiliary processor (123) may control at least a portion of functions or states associated with at least one component (e.g., a display module (160), a sensor module (176), or a communication module (190)) of the electronic device (101), for example, on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. In one embodiment, the auxiliary processor (123) (e.g., an image signal processor or a communication processor) may be implemented as a part of another functionally related component (e.g., a camera module (180) or a communication module (190)). In one embodiment, the auxiliary processor (123) (e.g., a neural network processing unit) may include a hardware structure specialized for processing artificial intelligence models. The artificial intelligence models may be generated through machine learning. This learning can be performed, for example, on the electronic device (101) itself where the artificial intelligence model is executed, or can be performed through a separate server (e.g., server (108)). The learning algorithm can include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model can include multiple artificial neural network layers.The artificial neural network may be one of a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to, or alternatively to, a hardware structure, an artificial intelligence model may include a software structure.
[0042] The memory (130) can store various data used by at least one component (e.g., processor (120) or sensor module (176)) of the electronic device (101). The data can include, for example, software (e.g., program (140)) and input data or output data for commands related thereto. The memory (130) can include volatile memory (132) or non-volatile memory (134).
[0043] The program (140) may be stored as software in the memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).
[0044] The input module (150) can receive commands or data to be used in a component of the electronic device (101) (e.g., a processor (120)) from an external source (e.g., a user) of the electronic device (101). The input module (150) can include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).
[0045] The audio output module (155) can output audio signals to the outside of the electronic device (101). The audio output module (155) can include, for example, a speaker or a receiver. The speaker can be used for general purposes, such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. In one embodiment, the receiver can be implemented separately from the speaker or as part of the speaker.
[0046] The display module (160) can visually provide information to an external party (e.g., a user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling the device. According to one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of a force generated by the touch.
[0047] The audio module (170) can convert sound into an electrical signal, or vice versa, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150), output sound through the sound output module (155), or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphone) directly or wirelessly connected to the electronic device (101).
[0048] The sensor module (176) can detect the operating status (e.g., power or temperature) of the electronic device (101) or the external environmental status (e.g., user status) and generate an electrical signal or data value corresponding to the detected status. According to one embodiment, the sensor module (176) can include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.
[0049] The interface (177) may support one or more designated protocols that may be used to directly or wirelessly connect the electronic device (101) to an external electronic device (e.g., the electronic device (102)). In one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.
[0050] The connection terminal (178) may include a connector through which the electronic device (101) may be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).
[0051] The haptic module (179) can convert electrical signals into mechanical stimuli (e.g., vibration or movement) or electrical stimuli that a user can perceive through tactile or kinesthetic sensations. According to one embodiment, the haptic module (179) can include, for example, a motor, a piezoelectric element, or an electrical stimulation device.
[0052] The camera module (180) can capture still images and moving images. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.
[0053] The power management module (188) can manage power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented as, for example, at least a part of a power management integrated circuit (PMIC).
[0054] A battery (189) may power at least one component of the electronic device (101). In one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.
[0055] The communication module (190) may support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may operate independently from the processor (120) (e.g., application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (194) (e.g., a local area network (LAN) communication module, or a power line communication module). Among these communication modules, the corresponding communication module can communicate with an external electronic device (104) via a first network (198) (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network (199) (e.g., a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules can be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can verify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) by using subscriber information (e.g., an international mobile subscriber identity (IMSI)) stored in the subscriber identification module (196).
[0056] The wireless communication module (192) can support 5G networks and next-generation communication technologies following the 4G network, such as NR access technology (new radio access technology). The NR access technology can support high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimization of terminal power and connection of multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low-latency communications)). The wireless communication module (192) can support, for example, a high-frequency band (e.g., millimeter wave (mmWave) band) to achieve a high data transmission rate. The wireless communication module (192) can support various technologies for securing performance in a high-frequency band, such as beamforming, massive multiple-input and multiple-output (MIMO), full dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), an external electronic device (e.g., the electronic device (104)), or a network system (e.g., the second network (199)). According to one embodiment, the wireless communication module (192) may be configured to achieve a peak data rate (e.g., 20 Gbps (gigabits per second) or higher) for eMBB implementation, a loss coverage (e.g., 664 decibels (dB) or lower) for mMTC implementation, or a U-plane latency (e.g., 0 for downlink (DL) and uplink (UL) respectively) for URLLC implementation.It can support latency of 5 milliseconds (ms) or less, or round trip of 6ms or less.
[0057] The antenna module (197) can transmit or receive signals or power to or from an external device (e.g., an external electronic device). In one embodiment, the antenna module (197) may include an antenna including a radiator formed of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). In one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as the first network (198) or the second network (199), may be selected from the plurality of antennas, for example, by the communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device via the at least one selected antenna. In some embodiments, in addition to the radiator, another component (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as a part of the antenna module (197).
[0058] According to various embodiments, the antenna module (197) may form a mmWave antenna module. In one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent a first side (e.g., a bottom side) of the printed circuit board and capable of supporting a designated high-frequency band (e.g., a mmWave band), and a plurality of antennas (e.g., an array antenna) disposed on or adjacent a second side (e.g., a top side or a side side) of the printed circuit board and capable of transmitting or receiving signals in the designated high-frequency band.
[0059] At least some of the above components can be interconnected and exchange signals (e.g., commands or data) with each other via a communication method between peripheral devices (e.g., a bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)).
[0060] According to one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) via a server (108) connected to a second network (199). Each of the external electronic devices (102 or 104) may be the same or a different type of device as the electronic device (101). According to one embodiment, all or part of the operations executed in the electronic device (101) may be executed in one or more of the external electronic devices (e.g., the electronic devices (102 and 104), or the server (108)). For example, when the electronic device (101) is to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device (101) may, instead of or in addition to executing the function or service itself, request one or more external electronic devices to perform the function or at least a part of the service. One or more external electronic devices that receive the request may execute at least a portion of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may process the result as is or additionally and provide it as at least a portion of a response to the request. For this purpose, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used, for example. The electronic device (101) may provide an ultra-low latency service by using distributed computing or mobile edge computing, for example. In another embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server utilizing machine learning and / or a neural network.According to one embodiment, an external electronic device (104) or server (108) may be included in the second network (199). The electronic device (101) may be applied to intelligent services (e.g., smart homes, smart cities, smart cars, or healthcare) based on 5G communication technology and IoT-related technology.
[0061] FIG. 2A illustrates an example of the appearance of a first side of a wearable device according to an embodiment of the disclosure. FIG. 2B illustrates an example of the appearance of a second side of a wearable device according to an embodiment of the disclosure.
[0062] The wearable device (200) of FIGS. 2A and 2B may be the electronic device (101) described in FIG. 1. An example of the appearance of a first side (210) of a housing of the wearable device (200) may be illustrated in FIG. 2A, and an example of the appearance of a second side (220) opposite to the first side (210) may be illustrated in FIG. 2B.
[0063] Referring to FIG. 2A, according to one embodiment, the first side (210) of the wearable device (200) may have a form attachable to a body part of a user (e.g., the face of the user). Although not shown, the wearable device (200) may further include a strap for being fixed to a body part of the user, and / or one or more temples.
[0064] A first display (250-1) for outputting an image to the left eye among the user's two eyes, and a second display (250-2) for outputting an image to the right eye among the two eyes may be arranged on the first surface (210). The wearable device (200) may be formed on the first surface (210) and may further include a rubber or silicone packing to prevent interference by light (e.g., ambient light) different from light radiated from the first display (250-1) and the second display (250-2).
[0065] According to one embodiment, the wearable device (200) may include cameras (240-1, 240-2) for photographing and / or tracking both eyes of a user adjacent to the first display (250-1) and the second display (250-2), respectively. The cameras (240-1, 240-2) may be referred to as ET (eye tracking) cameras. According to one embodiment, the wearable device (200) may include cameras (240-3, 240-4, 240-11) for photographing and / or recognizing a user's face. The cameras (240-3, 240-4, 240-11) may be referred to as FT (face tracking) cameras.
[0066] Referring to FIG. 2B, a camera (e.g., cameras (240-5, 240-6, 240-7, 240-8, 240-9, 240-10)) and / or a sensor (e.g., depth sensor (230)) for obtaining information related to the external environment of the wearable device (200) may be disposed on a second surface (220) opposite to the first surface (210) of FIG. 2A. For example, the cameras (240-5, 240-6, 240-7, 240-8, 240-9, 240-10) may be disposed on the second surface (220) to recognize an external object different from the wearable device (200). For example, using cameras (240-9, 240-10), the wearable device (200) can obtain images and / or videos to be transmitted to each of the user's eyes. The camera (240-9) can be placed on the second face (220) of the wearable device (200) to obtain an image to be displayed through the second display (250-2) corresponding to the right eye among the two eyes. The camera (240-10) can be placed on the second face (220) of the wearable device (200) to obtain an image to be displayed through the first display (250-1) corresponding to the left eye among the two eyes.
[0067] According to one embodiment, the wearable device (200) may include a depth sensor (230) disposed on the second surface (220) to identify a distance between the wearable device (200) and an external object. Using the depth sensor (230), the wearable device (200) may obtain spatial information (e.g., a depth map) for at least a portion of a field of view (FoV) of a user wearing the wearable device (200).
[0068] Although not shown, a microphone may be placed on the second side (220) of the wearable device (200) to acquire sound output from an external object. The number of microphones may be one or more depending on the embodiment.
[0069] As described above, according to one embodiment, the wearable device (200) may have a form factor for being worn on a user's head. The wearable device (200), while worn on the head, may provide a user experience based on augmented reality, virtual reality, and / or mixed reality. Using cameras (240-5, 240-6, 240-7, 240-8, 240-9, 240-10) for recording video of an external space, the wearable device (200) and a server connected to the wearable device (200) (e.g., server (108) of FIG. 1) may provide an on-demand service that provides video of a location and / or place selected by the user, and / or a metaverse service.
[0070] According to one embodiment, the wearable device (200) may display frames acquired through cameras (240-9, 240-10) on the first display (250-1) and the second display (250-2), respectively. The wearable device (200) may combine virtual objects within frames displayed through the first display (250-1) and the second display (250-2), which include real objects, to provide a user with a mixed user experience of real objects and virtual objects (e.g., video see-through (VST)). The wearable device (200) may change the virtual objects based on information acquired by the cameras (240-1, 240-2, 240-3, 240-4, 240-5, 240-6, 240-7, 240-8) and / or the depth sensor (230). For example, if a visual object corresponding to a real object and a virtual object at least partially overlap within the frame, the wearable device (200) may stop displaying the virtual object based on detecting a motion for interacting with the real object. By stopping displaying the virtual object, the wearable device (200) may prevent the visibility of the real object from being reduced as the visual object corresponding to the real object is occluded by the virtual object.
[0071] FIG. 2C schematically illustrates an example of a structure of a wearable device according to an embodiment of the disclosure. FIG. 2D schematically illustrates an example of a structure for eye tracking (ET) of a wearable device according to an embodiment of the disclosure.
[0072] The wearable device (200) of FIGS. 2c and 2d may be the electronic device (101) described in FIG. 1.
[0073] Referring to FIG. 2C, the wearable device (200) may include a lens assembly (260) including lens assemblies (260-1, 260-2) adjacent to each of the first display (250-1) and the second display (250-2). The lens assemblies (260-1, 260-2) may include one or more lenses.
[0074] A wearable device (200) may include a structure (270) including structures (270-1, 270-2) adjacent to each of the first display (250-1) and the second display (250-2). Referring to FIG. 2D, each of the structures (270-1, 270-2) may include one or more light sources (280).
[0075] For example, each of the structures (270-1, 270-2) may have a circular structure. For example, one side of each of the structures (270-1, 270-2) may have a circular structure. For example, the side of each of the structures (270-1, 270-2) facing the user's two eyes (265-1, 265-2) may have a circular structure. However, the present invention is not limited thereto. For example, each of the structures (270-1, 270-2) may be polygonal, elliptical, or oval-shaped. For example, one side of each of the structures (270-1, 270-2) may be polygonal, elliptical, or oval-shaped. For example, the surface facing the user's two eyes (265-1, 265-2) of each of the structures (270-1, 270-2) may be polygonal, elliptical, or egg-shaped. Each of the structures (270-1, 270-2) may include a printed circuit board (PCB) or a flexible PCB (FPCB).
[0076] The wearable device (200) may include cameras (240-1, 240-2) for photographing and / or tracking the user's two eyes (265-1, 265-2) adjacent to the first display (250-1) and the second display (250-2), respectively. Each of the cameras (240-1, 240-2) may be arranged in each of the structures (270-3, 270-4) of the structure (270). Each of the structures (270-3, 270-4) may include a PCB or an FPCB. However, the present invention is not limited thereto. Each of the cameras (240-1, 240-2) may be arranged in each of the structures (270-1, 270-2) in which the light sources (280) are arranged.
[0077] The light sources (280) may be arranged to be spaced apart from each other in each of the structures (270-1, 270-2). The light sources (280) may be arranged to be spaced apart from each other at equal intervals in each of the structures (270-1, 270-2). The light sources (280) may be arranged to be spaced apart from each other at equal angles in each of the structures (270-1, 270-2). Here, the angle may be an angle formed by two neighboring light sources from the center points of the light sources (280) arranged in each of the structures (270-1, 270-2). However, the present invention is not limited thereto. For example, some of the light sources (280) may be arranged to be spaced apart from each other at different intervals in each of the structures (270-1, 270-2). Some of the light sources (280) may be arranged to be spaced at different angles from each of the structures (270-1, 270-2).
[0078] Each of the cameras (240-1, 240-2) can acquire an image while the light sources (280) arranged in each of the structures (270-1, 270-2) emit light. The acquired image may include reflected light of light emitted by at least some of the light sources (280). For example, the camera (240-1) can acquire an image including reflected light of light of the light sources (280) arranged in the structure (270-1) while the light sources (280) emit light. For example, the camera (240-2) can acquire an image including reflected light of light of the light sources (280) arranged in the structure (270-2) while the light sources (280) emit light. Here, the reflected light may be referred to as glint. An image that includes reflected light from light sources (280) may be referred to as a glint image.
[0079] The wearable device (200) can track the user's gaze based on the glint image. For example, the wearable device (200) can identify the position of the iris of each of the user's eyes (265-1, 265-2) based on the glint image. For example, the wearable device (200) can track the user's gaze based on the position of the iris. However, the present invention is not limited thereto.
[0080] FIG. 2e schematically illustrates an example of a layout structure of light sources of a wearable device according to an embodiment of the disclosure.
[0081] The wearable device (200) of FIG. 2e may be the electronic device (101) described in FIG. 1.
[0082] Referring to FIG. 2e, the surface of the structures (270-1, 270-2) facing the user's two eyes (265-1, 265-2) may have a circular structure. However, the present invention is not limited thereto.
[0083] A plurality of light sources (280) may be arranged in the structures (270-1, 270-2). For example, the structure (270-1) may include ten light sources (281-1, 281-2, 281-3, 281-4, 281-5, 282-1, 282-2, 282-3, 282-4, 282-5). The structure (270-2) may include ten light sources (283-1, 283-2, 283-3, 283-4, 283-5, 284-1, 284-2, 284-3, 284-4, 284-5). However, the present invention is not limited thereto. For example, each of the structures (270-1, 270-2) may have ten or more light sources (280) arranged therein. For example, each of the structures (270-1, 270-2) may have eight light sources (280) arranged therein.
[0084] The light sources (280) may be arranged to be spaced apart from each other in each of the structures (270-1, 270-2). The light sources (280) may be arranged to be spaced apart from each other at equal intervals in each of the structures (270-1, 270-2). The light sources (280) may be arranged to be spaced apart from each other at equal angles in each of the structures (270-1, 270-2). Here, the angle may be an angle formed by two neighboring light sources from the center point of the light sources (280) arranged in each of the structures (270-1, 270-2).
[0085] However, this is not limited thereto. For example, some of the light sources (280) may be arranged to be spaced apart from each of the structures (270-1, 270-2) at different intervals. Some of the light sources (280) may be arranged to be spaced apart from each of the structures (270-1, 270-2) at different angles.
[0086] The plurality of light sources (280) included in each of the structures (270-1, 270-2) may be divided into two or more sets. For example, the plurality of light sources (280) included in the structure (270-1) may be divided into a first set of light sources (281) and a second set of light sources (282). For example, the plurality of light sources (280) included in the structure (270-2) may be divided into a third set of light sources (283) and a fourth set of light sources (284). However, the present invention is not limited thereto.
[0087] Any set of light sources can be selected from non-adjacent light sources (280). For example, a first set of light sources (281) can be selected from non-adjacent light sources (280). For example, non-adjacent light sources (281-1, 281-2, 281-3, 281-4, 281-5) can be included in the first set of light sources (281). For example, a second set of light sources (282) can be selected from non-adjacent light sources (280). For example, non-adjacent light sources (282-1, 282-2, 282-3, 282-4, 282-5) can be included in the second set of light sources (282). For example, the third set of light sources (283) may be selected from among non-adjacent light sources (280). For example, non-adjacent light sources (283-1, 283-2, 283-3, 283-4, 283-5) may be included in the third set of light sources (283). For example, the fourth set of light sources (284) may be selected from among non-adjacent light sources (280). For example, non-adjacent light sources (284-1, 284-2, 284-3, 284-4, 284-5) may be included in the fourth set of light sources (284). However, the present invention is not limited thereto. Any set of light sources may be selected from among light sources (280) some of which are adjacent to each other. Any set of light sources can be selected from among light sources (280) arranged adjacent to each other.
[0088] The wearable device (200) may utilize a specific set among two or more sets of structures (270-1, 270-2) for iris recognition. For example, the wearable device (200) may utilize a first set of light sources (281) among a first set of light sources (281) and a second set of light sources (282) for iris recognition for the structure (270-1). For example, the wearable device (200) may utilize a third set of light sources (283) among a third set of light sources (283) and a fourth set of light sources (284) for iris recognition for the structure (270-2). Hereinafter, a light source that may be utilized for iris recognition may be referred to as an iris light source.
[0089] The wearable device (200) may use some of the light sources included in a specific set among two or more sets of each of the structures (270-1, 270-2) for iris recognition. For example, the wearable device (200) may use a specified number of light sources included in a specific set for iris recognition. Here, the specified number may be 1 or more. For example, the wearable device (200) may use the light source (281-5) among the light sources (281-1, 281-2, 281-3, 281-4, 281-5) included in the first set of light sources (281) for iris recognition. For example, the wearable device (200) can use the light source (283-5) among the light sources (283-1, 283-2, 283-3, 283-4, 283-5) included in the third set of light sources (283) for iris recognition.
[0090] The wearable device (200) may use a light source positioned at a specific location among the light sources included in a specific set among two or more sets of each of the structures (270-1, 270-2) for iris recognition. For example, the wearable device (200) may use a light source adjacent to the camera (240) among the light sources included in a specific set for iris recognition. For example, the wearable device (200) may use the light source (281-5) among the light sources (281-1, 281-2, 281-3, 281-4, 281-5) included in the first set of light sources (281) for iris recognition. For example, the wearable device (200) may use the light source (283-5) among the light sources (283-1, 283-2, 283-3, 283-4, 283-5) included in the third set of light sources (283) for iris recognition. However, the present invention is not limited thereto. For example, the wearable device (200) may use a light source that is not adjacent to the camera (240) among the light sources included in a specific set for iris recognition. For example, the wearable device (200) may use the light source (281-3) among the light sources (281-1, 281-2, 281-3, 281-4, 281-5) included in the first set of light sources (281) for iris recognition. For example, the wearable device (200) can use the light source (283-3) among the light sources (283-1, 283-2, 283-3, 283-4, 283-5) included in the third set of light sources (283) for iris recognition.
[0091] FIG. 2f schematically illustrates an example of a circuit structure of light sources of a wearable device according to an embodiment of the disclosure.
[0092] The wearable device (200) of FIG. 2f may be the electronic device (101) described in FIG. 1.
[0093] FIG. 2f illustrates the circuit structure of light sources (281-1, 281-2, 281-3, 281-4, 281-5, 282-1, 282-2, 282-3, 282-4, 282-5) arranged in a structure (270-1) among the light sources (280) of a wearable device (200). FIG. 2f illustrates the structure (270-1) for the left eye (265-1). The same circuit structure as illustrated in FIG. 2f can also be applied to the structure (270-2) for the right eye (265-2), which is not illustrated.
[0094] The plurality of light sources (280) included in each of the structures (270-1, 270-2) can be controlled by two or more channels. The plurality of light sources (280) included in each of the structures (270-1, 270-2) can be powered by two or more channels.
[0095] Referring to FIG. 2f, a plurality of light sources (280) included in a structure (270-1) can be controlled by a light source driver (290). The light source driver (290) can have a plurality of output terminals. Here, the output terminals can be referred to as channels.
[0096] The light source driver (290) can individually control multiple output channels. For example, the light source driver (290) can individually supply power to multiple output channels. The light source driver (290) can individually control multiple output channels under the control of the processor (120). The light source driver (290) can control the light sources (280) included in the multiple output channels to emit light based on a timing signal from a camera (e.g., the camera (310) of FIG. 3A).
[0097] A plurality of light sources (280) included in the structure (270-1) can be controlled by four channels (291-1, 291-2, 291-3, 291-4) of the light source driver (290). For example, a first channel (291-1) can include light sources (281-3, 281-4). A second channel (291-2) can include light sources (281-1, 281-2, 281-5). A third channel (291-3) can include light sources (282-1, 282-5). A fourth channel (291-4) can include light sources (282-2, 282-3, 282-4). The first and second channels may include a first set of light sources (281). The third and fourth channels may include a second set of light sources (282). Similarly, the plurality of light sources (280) included in the structure (270-2) may be controlled by four channels. However, this is not limited thereto. The number of channels may be less than or more than four.
[0098] Some of the four channels (291-1, 291-2, 291-3, 291-4) may include resistance elements (292-1, 292-2). For example, some of the channels may include resistance components (292-1, 292-2) so that the resistances of each of the four channels (291-1, 291-2, 291-3, 291-4) are equalized. For example, the channels (291-1, 291-3) may include resistance components (292-1, 292-2) so that the resistances of each of the channels (291-1, 291-3) correspond to the resistances of the other channels (291-2, 291-4). For example, the first channel (291-1) may include a resistance component (292-1) such that the resistance of the first channel (291-1) corresponds to the resistance of the second channel (291-2). Here, the resistance components (292-1, 292-2) may have resistances such that the resistances of each of the four channels (291-1, 291-2, 291-3, 291-4) are equal to each other. For example, the resistance component (292-1) may correspond to the difference between the number of light sources (281-3, 281-4) included in the first channel (291-1) and the number of light sources (281-1, 281-2, 281-5) included in the second channel (291-2). For example, the resistance component (292-1) may have a resistance equal to the resistance of one light source.
[0099] Each of the resistance components (292-1, 292-2) may be a resistor having the same resistance as the light source. However, the present invention is not limited thereto. For example, each of the resistance components (292-1, 292-2) may be a wiring pattern configured to have the same resistance as the light source. For example, each of the resistance components (292-1, 292-2) may have a different structure from the wiring on which the light sources (280) are arranged. For example, at least one of the wiring length, wiring width, or wiring shape of each of the resistance components (292-1, 292-2) may be different from the wiring on which the light sources (280) are arranged.
[0100] The wearable device (200) may perform eye tracking or iris recognition using some of the multiple channels of each of the structures (270-1, 270-2). For example, the wearable device (200) may control light sources included in some of the multiple channels of each of the structures (270-1, 270-2) to emit light. For example, the wearable device (200) may supply power to some of the multiple channels of each of the structures (270-1, 270-2).
[0101] FIG. 3A is a simplified block diagram of a wearable device according to an embodiment of the disclosure. FIG. 3B is a simplified circuit diagram of a wearable device according to an embodiment of the disclosure.
[0102] The description of FIGS. 3A and 3B may refer to components or structures of the electronic device (101) or wearable device (200) described through FIGS. 1, 2A, 2B, 2C, 2D, 2E, and 2F.
[0103] The wearable device (200) of FIGS. 3A and 3B may be the electronic device (101) described in FIG. 1.
[0104] Referring to FIG. 3A, the wearable device (200) may include a processor (120), a memory (130), light sources (280), a camera (310), and a sensor (320). For example, the light sources (280) may be the light sources (280) of FIGS. 2D, 2E, and 2F. For example, the camera (310) may be included in the camera module (180) of FIG. 1. For example, the camera (310) may correspond to each of the cameras (240-1, 240-2) of FIGS. 2A, 2C, 2D, and 2E. For example, the sensor (320) may be included in the sensor module (176) of FIG. 1.
[0105] The light sources (280) can emit light. The light sources (280) can emit light of a specified frequency. For example, the light sources (280) can emit infrared light.
[0106] The light sources (280) can emit light toward the eye (265) of a user wearing the wearable device (200). The light sources (280) can be arranged to face the eye (265) of the user wearing the wearable device (200). For example, among the light sources (280), the light sources (281-1, 281-2, 281-3, 281-4, 281-5, 282-1, 282-2, 282-3, 282-4, 282-5) can be arranged to face the left eye (265-1) of the user. For example, among the light sources (280), the light sources (283-1, 283-2, 283-3, 283-4, 283-5, 284-1, 284-2, 284-3, 284-4, 284-5) may be arranged to face the user's right eye (265-2).
[0107] The light sources (280) may include a plurality of light sources. The plurality of light sources may emit light under the control of the processor (120). The light sources (281-1, 281-2, 281-3, 281-4, 281-5, 282-1, 282-2, 282-3, 282-4, 282-5) may emit light under the control of the processor (120). The light sources (283-1, 283-2, 283-3, 283-4, 283-5, 284-1, 284-2, 284-3, 284-4, 284-5) may emit light under the control of the processor (120).
[0108] For example, some of the plurality of light sources may emit light for eye tracking under the control of the processor (120). For example, the processor (120) may control the light sources included in at least one set of the plurality of sets for the left eye (265-1) to emit light for eye tracking. For example, the processor (120) may control the light sources (281-1, 281-2, 281-3, 281-4, 281-5) included in the first set of the two sets for the left eye (265-1) to emit light for eye tracking. For example, the processor (120) may control the light sources included in at least one channel of the plurality of channels for the left eye (265-1) to emit light for eye tracking. For example, the processor (120) may control the light sources (281-1, 281-2, 281-3, 281-4, 281-5) included in the first channel and the second channel among the four channels for the left eye (265-1) to emit light for eye tracking. For example, the processor (120) may control the light sources included in at least one set among the plurality of sets for the right eye (265-2) to emit light for eye tracking. For example, the processor (120) may control the light sources included in at least one channel among the plurality of channels for the right eye (265-2) to emit light for eye tracking.
[0109] For example, the processor (120) can perform eye tracking based on the images captured by the camera (310) while the plurality of light sources (280) are controlled to emit light toward the user's eyes.
[0110] For example, some of the other light sources may emit light for iris recognition under the control of the processor (120). Some of the other light sources for iris recognition may emit light for iris recognition under the control of the processor (120). For example, the processor (120) may control some of the light sources included in at least one set of the plurality of sets for the left eye (265-1) to emit light for iris recognition. For example, the processor (120) may control some (e.g., light source (281-5)) of the light sources (281-1, 281-2, 281-3, 281-4, 281-5) included in the first set of the two sets for the left eye (265-1) to emit light for iris recognition. For example, the processor (120) may control light sources included in at least one channel among the plurality of channels for the left eye (265-1) to emit light for iris recognition. For example, the processor (120) may control light sources (281-3, 281-4) included in a first channel among the four channels for the left eye (265-1) to emit light for iris recognition. For example, the processor (120) may control some of the light sources included in at least one set among the plurality of sets for the right eye (265-2) to emit light for iris recognition. For example, the processor (120) may control light sources included in at least one channel among the plurality of channels for the right eye (265-2) to emit light for iris recognition. Here, the number of light sources included in at least one channel for iris recognition may be less than the number of light sources included in at least one channel for eye tracking.
[0111] For example, the processor (120) may perform iris recognition based on at least one image captured by the camera (310) while some of the plurality of light sources (280) are controlled to shine light toward the user's eyes and while other some of the plurality of light sources (280) are inactivated. For example, some of the plurality of light sources (280) used to perform the iris recognition may correspond to a subset of the same light sources used for the eye tracking. For example, the number of light sources used to perform the iris recognition may be less than the number of light sources used for the eye tracking. For example, two light sources (or two or more light sources) of the light sources used for the eye tracking may be selected to perform the iris recognition. For example, only one of the light sources used for the eye tracking may be selected to perform the iris recognition.
[0112] Referring to FIG. 3B, the light sources (280) may be electrically connected to the processor (120) and / or the camera (310) via the light source driver (290). For example, the light sources (280) may be electrically connected to the light source driver (290) via an output terminal of the light source driver (290). For example, the light sources (280) may be electrically connected to the light source driver (290) via each of a plurality of channels of the light source driver (290).
[0113] The light source driver (290) may be electrically connected to the processor (120) via a communication path (301) based on I2C (inter integrated circuit). However, the present invention is not limited thereto. For example, the light source driver (290) may be electrically connected to the processor (120) via another interface. For example, the light source driver (290) may be electrically connected to the processor (120) via a bus, GPIO, SPI, or MIPI.
[0114] The light source driver (290) may be electrically connected to the camera (310). For example, the light source driver (290) may be electrically connected to the camera (310) via a bus-based communication path (302). However, the present invention is not limited thereto. For example, the light source driver (290) may be electrically connected to the camera (310) via GPIO or SPI.
[0115] The light source driver (290) can obtain a designated signal from the camera (310). For example, the light source driver (290) can obtain a strobe signal of the camera (310). Here, the strobe signal can indicate the timing of light emission of the light sources (280). For example, the camera (310) can transmit the strobe signal to the light source driver (290) to synchronize the light sources (280) and the camera (310) so that the light sources (280) emit light at a time for acquiring an image.
[0116] The camera (310) can acquire an image including an eye (265) of a user wearing the wearable device (200). The camera (310) can capture images of at least a portion of the eye (265) of the user wearing the wearable device (200). The camera (310) can acquire a glint image of the user wearing the wearable device (200). The camera (310) can acquire a glint image for eye tracking. The camera (310) can acquire an image for iris recognition.
[0117] The camera (310) can acquire images under the control of the processor (120). For example, the camera (310) can acquire images based on a command for acquiring images from the processor (120). For example, the camera (310) can provide information related to the light emission timing of the light sources (280) to the light sources (280) based on the command. For example, the camera (310) can acquire images based on the light emission timing. For example, the camera (310) can acquire images while the light sources (280) emit light based on the light emission timing. However, the present invention is not limited thereto. For example, the processor (120) can provide information related to the light emission timing to the light sources (280).
[0118] The camera (310) may be electrically connected to the processor (120) via an I2C-based communication path (311). However, the present invention is not limited thereto. For example, the camera (310) may be electrically connected to the processor (120) via another interface. For example, the camera (310) may be electrically connected to the processor (120) via a bus, GPIO, SPI, or MIPI.
[0119] The camera (310) can obtain a designated signal from the processor (120). For example, the camera (310) can obtain an initialization signal (or reset signal) from the processor (120). For example, the camera (310) can obtain the initialization signal (or reset signal) from the processor (120) through a designated signal line (312). The camera (310) can be initialized based on the initialization signal (or reset signal) from the processor (120). Here, initialization of the camera (310) may mean that the camera (310) is rebooted.
[0120] The camera (310) can obtain a designated signal from the processor (120). For example, the camera (310) can obtain a synchronization signal (or clock (CLK) signal) from the processor (120). For example, the camera (310) can obtain a synchronization signal (or clock (CLK) signal) from the processor (120) through a designated signal line (313). The camera (310) can be synchronized in operation with the processor (120) based on the synchronization signal (or clock (CLK) signal) from the processor (120).
[0121] The camera (310) may be electrically connected to the processor (120) via a communication path (314) based on MIPI. However, the present invention is not limited thereto. For example, the camera (310) may be electrically connected to the processor (120) via a bus, GPIO, SPI, or I2C.
[0122] The camera (310) can transmit the acquired image to the processor (120) through a communication path (314) based on MIPI.
[0123] The camera (310) can obtain a designated signal from the processor (120). For example, the camera (310) can obtain a synchronization signal (or clock (CLK) signal) from the processor (120). For example, the camera (310) can obtain a synchronization signal (or clock (CLK) signal) from the processor (120) through a designated signal line (313). The camera (310) can be synchronized in operation with the processor (120) based on the synchronization signal (or clock (CLK) signal) from the processor (120).
[0124] The sensor (320) can provide information to the processor (120). For example, the sensor (320) can provide environmental information surrounding the wearable device (200) to the processor (120). For example, the sensor (320) can provide information for identifying a device coupled to the wearable device (200). For example, the sensor (320) can provide information for identifying another device worn by a user of the wearable device (200). Here, the device coupled to the wearable device (200) or the other device worn by the user may include a lens. However, the present invention is not limited thereto. Hereinafter, the device coupled to the wearable device (200) or the other device worn by the user may be referred to as an auxiliary device.
[0125] The sensor (320) can provide information related to the auxiliary device to the processor (120). For example, the sensor (320) can obtain readable information from the auxiliary device. The sensor (320) can provide the information read from the auxiliary device to the processor (120). Here, the readable information can be recorded in the auxiliary device through a specified method. For example, the readable information can be recorded through a barcode or a QR (quick response) code. However, the present invention is not limited thereto. The sensor (320) can obtain information related to the auxiliary device based on near field communication (NFC). The readable information can be based on the physical characteristics of the lens. For example, the readable information can be identified based on the refractive index of the lens or the power of the lens.
[0126] The processor (120) can obtain input. The processor (120) can obtain input for eye tracking. For example, the processor (120) can trigger eye tracking by the program (140) while the program (140) is running. For example, the processor (120) can obtain an eye tracking request generated by the program (140) while the program (140) is running. For example, the processor (120) can obtain that a specified program (140) is running as input for eye tracking. For example, the processor (120) can obtain that a specified function is running while the specified program (140) is running as input for eye tracking. However, the present invention is not limited thereto. For example, the processor (120) can obtain a user input requesting eye tracking. For example, the processor (120) can obtain that the user is wearing the wearable device (200) as input for eye tracking.
[0127] The processor (120) can control the light sources (280) to emit light. The processor (120) can control the light sources (280) to emit light in response to an input for eye tracking.
[0128] The processor (120) can control two or more of the light sources (280) to emit light. The processor (120) can control the two or more light sources to emit light in response to an input for eye tracking. For example, the processor (120) can control the two or more light sources to emit light through at least one channel (e.g., channels (291-1, 291-2)) among the plurality of channels (291-1, 291-2, 291-3, 291-4) in response to an input for eye tracking.
[0129] The processor (120) can control two or more light sources among the light sources (281-1, 281-2, 281-3, 281-4, 281-5, 282-1, 282-2, 282-3, 282-4, 282-5) corresponding to the left eye (265-1) to emit light in response to an input for eye tracking. The processor (120) can control two or more light sources among the light sources (283-1, 283-2, 283-3, 283-4, 283-5, 284-1, 284-2, 284-3, 284-4, 284-5) corresponding to the right eye (265-2) to emit light in response to an input for eye tracking.
[0130] The processor (120) can acquire an image for eye tracking. The processor (120) can acquire an image for eye tracking in response to an input for eye tracking. The processor (120) can acquire an image for eye tracking while the light sources (280) are emitting light. The processor (120) can acquire an image for eye tracking while the two or more light sources are emitting light.
[0131] The processor (120) may acquire an image for one of the user's eyes (265-1, 265-2) in response to an input for eye tracking. The processor (120) may acquire an image for each of the user's eyes (265-1, 265-2) in response to an input for eye tracking. The processor (120) may sequentially acquire images for each of the user's eyes (265-1, 265-2) in response to an input for eye tracking. For example, the processor (120) may acquire an image corresponding to the left eye (265-1) and then acquire an image corresponding to the right eye (265-2) in response to an input for eye tracking. However, the present invention is not limited thereto.
[0132] The processor (120) can identify the user's gaze. The processor (120) can track the user's gaze. For example, the processor (120) can identify the location of the iris of each of the user's eyes (265-1, 265-2) based on an image for eye tracking. For example, the processor (120) can track the user's gaze based on the location of the iris. However, the present invention is not limited thereto. For example, the processor (120) can track the user's gaze based on the location of the iris identified based on an image corresponding to one of the user's eyes (265-1, 265-2).
[0133] The processor (120) can obtain input. The processor (120) can obtain input for iris recognition. For example, the processor (120) can obtain an iris recognition request generated by the program (140) while the program (140) is running. For example, the processor (120) can obtain a user authentication request by the program (140) as an iris recognition request. However, the present invention is not limited thereto. For example, the processor (120) can obtain a user input requesting iris recognition.
[0134] The processor (120) can control the light sources (280) to emit light. The processor (120) can control the light sources (280) to emit light in response to an input for iris recognition.
[0135] The processor (120) can control at least one of the light sources (280) to emit light. The processor (120) can control at least one of the light sources to emit light in response to an input for iris recognition. For example, the processor (120) can control at least one of the light sources to emit light through at least one channel (e.g., the first channel (291-1)) among the plurality of channels (291-1, 291-2, 291-3, 291-4) for the left eye (265-1) in response to an input for iris recognition. For example, the processor (120) can control at least one of the plurality of channels (not shown) for the right eye (265-2) in response to an input for iris recognition to emit light.
[0136] The processor (120) may select at least one light source among the light sources (280) in response to an input for iris recognition. For example, the processor (120) may select at least one light source based on user information. For example, the processor (120) may select at least one light source used in a previous iris recognition of the user. Here, the at least one light source used in a previous iris recognition of the user may be at least one light source used when the iris recognition was successful. For example, the processor (120) may select at least one light source designated in advance when the user first attempts iris recognition. For example, the processor (120) may select at least one light source based on situational information. For example, the processor (120) may select at least one light source designated by the program (140) that requested iris recognition. The processor (120) may control the at least one selected light source to emit light.
[0137] The processor (120) can control at least one of the light sources (281-1, 281-2, 281-3, 281-4, 281-5, 282-1, 282-2, 282-3, 282-4, 282-5) corresponding to the left eye (265-1) to emit light in response to an input for iris recognition. The processor (120) can control at least one of the light sources (283-1, 283-2, 283-3, 283-4, 283-5, 284-1, 284-2, 284-3, 284-4, 284-5) corresponding to the right eye (265-2) to emit light in response to an input for iris recognition.
[0138] The processor (120) can acquire an image for iris recognition. The processor (120) can acquire an image for iris recognition in response to an input for iris recognition. The processor (120) can acquire an image for iris recognition while the light sources (280) are emitting light. The processor (120) can acquire an image for iris recognition while at least one light source is emitting light.
[0139] The processor (120) may acquire an image of one of the user's eyes (265-1, 265-2) in response to an input for iris recognition. The processor (120) may acquire an image of each of the user's eyes (265-1, 265-2) in response to an input for iris recognition. The processor (120) may sequentially acquire images of each of the user's eyes (265-1, 265-2) in response to an input for iris recognition. For example, the processor (120) may acquire an image corresponding to the left eye (265-1) and then acquire an image corresponding to the right eye (265-2) in response to an input for iris recognition. However, the present invention is not limited thereto.
[0140] The processor (120) can perform biometric authentication (or biometric identification). The processor (120) can perform iris recognition based on an image for iris recognition. For example, the processor (120) can perform iris recognition based on an iris image of the user included in the image for iris recognition. However, the present invention is not limited thereto.
[0141] The processor (120) can obtain input. The processor (120) can obtain input for positional information of light sources (280). For example, the positional information can indicate which light source each of the plurality of glints included in the glint image corresponds to.
[0142] For example, the processor (120) may obtain a location information request generated by the program (140) while the program (140) is being executed. For example, the processor (120) may obtain that a specified program (140) is being executed as an input for location information. However, the present invention is not limited thereto. For example, the processor (120) may obtain that a new user is wearing the wearable device (200) as an input for location information. Here, the new user may refer to a user who uses the wearable device (200) for which location information has not been registered.
[0143] The processor (120) can control the light sources (280) to emit light at at least two points in time. The processor (120) can control the light sources (280) to emit light at at least two points in time in response to an input for position information. For example, the processor (120) can control some of the light sources (280) to emit light at a first point in time in response to an input for position information. For example, the processor (120) can control other some of the light sources (280) to emit light at a second point in time in response to an input for position information. Here, the light sources emitting light at the first point in time and the light sources emitting light at the second point in time may not overlap each other.
[0144] The processor (120) may control the plurality of channels (291-1, 291-2, 291-3, 291-4) to emit light at at least two points in time in response to an input for location information. For example, the processor (120) may control some of the plurality of channels (291-1, 291-2, 291-3, 291-4) to emit light at a first point in time in response to an input for location information. For example, the processor (120) may control some of the other of the plurality of channels (291-1, 291-2, 291-3, 291-4) to emit light at a second point in time in response to an input for location information. Here, the channels of the light sources emitting light at the first point in time and the channels of the light sources emitting light at the second point in time may not overlap with each other.
[0145] The processor (120) can obtain an image for location information at each of at least two points in time. In response to an input for location information, the processor (120) can obtain an image for location information at each of at least two points in time.
[0146] The processor (120) can identify location information. For example, the processor (120) can identify the location relationship of each light source (280) based on an image for location information.
[0147] Below, the operations performed by the wearable device (200) for eye tracking and iris recognition are described.
[0148] Below, the operation of the wearable device (200) performing eye tracking and iris recognition based on multiple cameras is described.
[0149] The camera (310) may include a plurality of cameras. The camera (310) may include a plurality of cameras for each of the structures (270-1, 270-2). The camera (310) may include a plurality of cameras for acquiring images based on light sources (280) arranged in each of the structures (270-1, 270-2). At least two of the plurality of cameras may be arranged in structures symmetrical with respect to the center point of each of the structures (270-1, 270-2) in which the plurality of light sources (280) are arranged. At least two of the plurality of cameras may be arranged in structures positioned at specific portions of each of the structures (270-1, 270-2) in which the plurality of light sources (280) are arranged. For example, at least two of the plurality of cameras may be arranged in a structure positioned below the iris of the user's eye (265). However, the present invention is not limited thereto.
[0150] Each of the plurality of cameras can acquire images under the control of the processor (120). Each of the plurality of cameras can acquire images including the eyes (265) of a user wearing the wearable device (200). Each of the plurality of cameras can be used for eye tracking. Each of the plurality of cameras can be used for iris recognition. Each of the plurality of cameras can be used for either eye tracking or iris recognition. However, the present invention is not limited thereto.
[0151] The processor (120) can obtain an image for eye tracking based on a first camera among a plurality of cameras. The processor (120) can obtain an image for eye tracking based on a first camera among a plurality of cameras arranged for the left eye (265-1). The processor (120) can obtain an image for eye tracking based on a first camera among a plurality of cameras arranged for the right eye (265-2). The processor (120) can obtain an image for eye tracking based on the first camera arranged for the left eye (265-1) and the first camera arranged for the right eye (265-2). The processor (120) can obtain an image for eye tracking based on the first camera among the plurality of cameras, based on an input for eye tracking.
[0152] The processor (120) can obtain an image for iris recognition based on a second camera among the plurality of cameras. The processor (120) can obtain an image for iris recognition based on a second camera among the plurality of cameras arranged for the left eye (265-1). The processor (120) can obtain an image for iris recognition based on a second camera among the plurality of cameras arranged for the right eye (265-2). The processor (120) can obtain an image for iris recognition based on the second camera arranged for the left eye (265-1) and the second camera arranged for the right eye (265-2). The processor (120) can obtain an image for iris recognition based on an input for iris recognition based on the second camera among the plurality of cameras.
[0153] Below, the operation of the wearable device (200) selecting light sources for iris recognition is described.
[0154] The processor (120) may select some of the light sources (280). In response to the input for iris recognition, the processor (120) may select some of the light sources (280). For example, the processor (120) may select at least one light source based on user information. For example, the processor (120) may select some of the light sources used in the user's previous iris recognition. Here, some of the light sources used in the user's previous iris recognition may be the light sources used when the iris recognition was successful. For example, the processor (120) may select a pre-designated light source when the user first attempts iris recognition. For example, the processor (120) may select a light source based on contextual information. For example, the processor (120) may select a light source designated by the program (140) that requested iris recognition.
[0155] The processor (120) can control some light sources to emit light. The processor (120) can control some light sources to emit light in response to the input for iris recognition. The processor (120) can control some selected light sources to emit light in response to the input for iris recognition.
[0156] The processor (120) can acquire an image for iris recognition through the camera (310) while some light sources are emitting light. The processor (120) can perform iris recognition based on the image acquired while some light sources are emitting light. The processor (120) can determine whether to re-acquire the image for iris recognition in response to the iris recognition result based on the image acquired while the first light source is emitting light.
[0157] The processor (120) may select other portions of the light sources (280) in response to the iris recognition result based on the image acquired while some of the light sources are emitting light. The processor (120) may select other portions of the light sources (280) in response to the iris recognition based on the image acquired while some of the light sources are emitting light failing. Here, some of the other light sources may be selected based on user information. For example, some of the other light sources may be other light sources used when the previous iris recognition was successful. For example, some of the other light sources may be other light sources designated by the program (140). Some of the other light sources may be other light sources than some of the light sources, but are not limited thereto. Some of the other light sources may include some of the light sources. The number of some of the light sources may be different from the number of some of the other light sources. Some of the light sources may be arranged closer to the camera (310) than some of the other light sources. But are not limited thereto. Some of the other light sources may be arranged closer to the camera (310) than some of the light sources.
[0158] The processor (120) can acquire an image for iris recognition through the camera (310) while some other light sources are emitting light. The processor (120) can perform iris recognition based on the image acquired while some other light sources are emitting light. The processor (120) can determine whether to re-acquire the image for iris recognition in response to the iris recognition result based on the image acquired while some other light sources are emitting light.
[0159] Below, the operation of the wearable device (200) selecting light sources for iris recognition based on the surrounding environment is described.
[0160] The processor (120) can determine, based on information acquired through the sensor (320), whether a device other than the wearable device (200) is identified between the wearable device (200) and the user's eye. The processor (120) can determine, based on information acquired through the sensor (320), whether an additional device is identified between the wearable device (200) and the user's eye. Here, the additional device may include a lens. Hereinafter, a lens included in the additional device may be referred to as an additional lens. Here, information acquired through the sensor (320) may be recorded in the additional device through a designated method. For example, information acquired through the sensor (320) may be recorded through a barcode or a QR (quick response) code. Information acquired through the sensor (320) may indicate physical characteristics of the lens. For example, information acquired through the sensor (320) may indicate a refractive index of the lens or a power of the lens.
[0161] The processor (120) may control some of the light sources (280) to emit light in response to the input for iris recognition. The processor (120) may control some of the light sources (280) to emit light in response to the input for iris recognition, depending on whether an additional lens of the additional device is identified.
[0162] The processor (120) may control at least one of the light sources (280) to emit light in response to the input for iris recognition of the eye, based on the identification of the additional lens. The processor (120) may control at least one of the light sources (280) to emit light in response to the input for iris recognition of the eye, when the additional lens is identified. The processor (120) may control at least one of the light sources (280) to emit light in response to the input for iris recognition of the eye, while the additional lens is being identified.
[0163] The processor (120) may select at least one light source among the light sources (280) based on the information acquired through the sensor (320) based on the identification of the additional lens. The processor (120) may control the selected at least one light source to emit light. Here, the selected at least one light source may be a light source indicated by the information acquired through the sensor (320). For example, the processor (120) may select at least one light source corresponding to a physical characteristic indicated by the additional lens based on the information acquired through the sensor (320) among the light sources (280). For example, the processor (120) may select at least one light source corresponding to a refractive index of the lens and / or a lens power indicated by the additional lens based on the information acquired through the sensor (320) among the light sources (280). Here, the fact that at least one light source corresponds to the refractive index of the lens and / or the power of the lens represented by the additional lens may mean that at least one light source is pre-designated to acquire an image for iris recognition.
[0164] The processor (120) may obtain an image for iris recognition through the camera (310) while the at least one light source is emitting light. The processor (120) may obtain an image for iris recognition through the camera (310) while the at least one light source is emitting light based on the identification of the additional lens. When the additional lens is identified, the processor (120) may obtain an image for iris recognition through the camera (310) while the at least one light source is emitting light. The processor (120) may obtain an image for iris recognition through the camera (310) while the at least one light source is emitting light while the additional lens is identified.
[0165] The processor (120) may perform iris recognition based on the acquired image in response to the input for iris recognition of the eye based on the identification of the additional lens. The processor (120) may stop eye tracking based on the acquired image in response to the input for iris recognition of the eye based on the identification of the additional lens. Here, stopping eye tracking may mean that the processor (120) performs iris recognition based on the acquired image in response to the input for iris recognition of the eye based on the identification of the additional lens, and does not perform eye tracking. Stopping eye tracking may include re-performing eye tracking after the processor (120) successfully performs iris recognition based on the acquired image based on the identification of the additional lens.
[0166] The processor (120) may control two or more of the light sources (280) to emit light in response to the input for iris recognition of the eye based on the fact that the additional lens is not identified. The processor (120) may control two or more of the light sources (280) to emit light in response to the input for iris recognition of the eye when the additional lens is not identified. The processor (120) may control two or more of the light sources (280) to emit light in response to the input for iris recognition of the eye while the additional lens is not identified.
[0167] The processor (120) can obtain an image for iris recognition through the camera (310) while the two or more light sources are emitting light. The processor (120) can obtain an image for iris recognition through the camera (310) while the two or more light sources are emitting light based on the fact that the additional lens is not identified. The processor (120) can obtain an image for iris recognition through the camera (310) while the two or more light sources are emitting light when the additional lens is not identified. The processor (120) can obtain an image for iris recognition through the camera (310) while the two or more light sources are emitting light while the additional lens is not identified.
[0168] The processor (120) may perform iris recognition based on the acquired image based on the fact that the additional lens is not identified. The processor (120) may perform iris recognition based on the acquired image in response to the input for iris recognition of the eye based on the fact that the additional lens is not identified. The processor (120) may perform eye tracking based on the acquired image in response to the input for iris recognition of the eye based on the fact that the additional lens is not identified. Here, performing eye tracking may mean that the processor (120) performs iris recognition and eye tracking based on the acquired image in response to the input for iris recognition of the eye based on the fact that the additional lens is not identified. Performing eye tracking may include stopping the iris recognition after the processor (120) successfully performs iris recognition based on the acquired image based on the fact that the additional lens is identified.
[0169] Below, the operation of the wearable device (200) performing iris recognition during eye tracking is described.
[0170] The processor (120) can control the light sources (280) to emit light in response to an input for eye tracking. The processor (120) can control two or more light sources (280) to emit light in response to an input for eye tracking.
[0171] The processor (120) can acquire an image for eye tracking. The processor (120) can acquire an image for eye tracking in response to an input for eye tracking. The processor (120) can acquire an image for eye tracking while the light sources (280) are emitting light. The processor (120) can acquire an image for eye tracking while the two or more light sources are emitting light.
[0172] The processor (120) can identify the user's gaze. The processor (120) can track the user's gaze. For example, the processor (120) can identify the location of the irises of each of the user's eyes (265-1, 265-2) based on an image for eye tracking. For example, the processor (120) can track the user's gaze based on the location of the irises. However, the present invention is not limited thereto.
[0173] The processor (120) may acquire input for iris recognition. The processor (120) may acquire the input for iris recognition during eye tracking. Here, during eye tracking, the processor (120) may include a time for identifying the user's gaze based on images acquired in response to the input for eye tracking. During eye tracking, the processor (120) may include a time for sequentially identifying the user's gaze based on images acquired sequentially in response to the input for eye tracking.
[0174] The processor (120) can perform iris recognition based on an image for eye tracking acquired while two or more light sources are emitting light.
[0175] The processor (120) can determine whether to reacquire the image for iris recognition during eye tracking based on the result of iris recognition based on the image for eye tracking acquired while two or more light sources are emitting light.
[0176] In response to determining not to interrupt the eye tracking, the processor (120) may perform iris recognition using at least one additional light source, different from the two or more light sources. Here, the at least one additional light source may not be included in the two or more light sources. However, this is not a limitation. For example, the at least one additional light source may be part of the two or more light sources.
[0177] The processor (120) may perform iris recognition using at least one light source other than the two or more light sources based on the results of iris recognition based on an image for eye tracking acquired while the two or more light sources are emitting light. The processor (120) may perform iris recognition using a light source other than the two or more light sources in response to a failure of iris recognition based on an image for eye tracking acquired while the two or more light sources are emitting light. However, the present invention is not limited thereto. The processor (120) may perform iris recognition using some of the two or more light sources in response to a failure of iris recognition based on an image for eye tracking acquired while the two or more light sources are emitting light.
[0178] For example, in response to determining to perform iris recognition during the eye tracking, the processor (120) may turn off some of the two or more light sources. In response to determining to perform iris recognition during the eye tracking, the processor (120) may acquire an image for eye tracking with some of the two or more light sources turned off. Here, some of the light sources to be turned off may be selected based on user information. For example, some of the light sources to be turned off may be light sources that were not used in a previous successful iris recognition. For example, some of the light sources to be turned off may be light sources that were not used in a previous successful iris recognition. For example, some of the light sources to be turned off may be other light sources designated by the program (140).
[0179] In response to a decision to stop the eye tracking, the processor (120) may perform iris recognition using at least one light source. Here, the at least one light source may be a light source used in a previous successful iris recognition, but is not limited thereto.
[0180] For example, in response to determining not to perform iris recognition during the eye tracking, the processor (120) may perform iris recognition using at least one light source. In response to determining not to perform iris recognition during the eye tracking, the processor (120) may acquire an image for iris recognition using at least one light source.
[0181] FIG. 4A schematically illustrates an example of a circuit structure of light sources of a wearable device according to an embodiment of the disclosure.
[0182] The description of FIG. 4a may refer to components or structures of the electronic device (101) or wearable device (200) described through FIGS. 1, 2a, 2b, 2c, 2d, 2e, 2f, 3a, and 3b.
[0183] FIG. 4a illustrates a circuit structure of light sources arranged in structures according to one embodiment of the disclosure.
[0184] Referring to FIG. 4a, a plurality of light sources (421-1, 421-2, 421-3, 423-1, 423-2, 423-3, 425-1, 425-2, 425-3, 427-1, 427-2, 427-3, 429-1, 429-2, 429-3) included in each of the structures (270-1, 270-2) can be arranged in equal numbers in the output channels (411, 413, 415, 417, 419). The same number of light sources may be included in the multiple output channels (411, 413, 415, 417, 419) of FIG. 4a compared to the output channels (291-1, 291-2, 291-3, 291-4) of FIG. 2f.
[0185] Compared to the output channels (291-1, 291-2, 291-3, 291-4) of FIG. 2F, the plurality of output channels (411, 413, 415, 417, 419) of FIG. 4A may not include a resistance component. Since the plurality of output channels (411, 413, 415, 417, 419) include an equal number of light sources, the resistance component may not be included.
[0186] The light source driver (290) can control output channels (411, 413, 415, 417, 419). The light source driver (290) can individually control a plurality of output channels (411, 413, 415, 417, 419). For example, the light source driver (290) can individually supply power to a plurality of output channels (411, 413, 415, 417, 419).
[0187] The light source driver (290) can individually control a plurality of output channels (411, 413, 415, 417, 419) under the control of the processor (120). The light source driver (290) can individually control the power supply to each of the plurality of output channels (411, 413, 415, 417, 419) under the control of the processor (120). The light source driver (290) can control light sources (421-1, 421-2, 421-3, 423-1, 423-2, 423-3, 425-1, 425-2, 425-3, 427-1, 427-2, 427-3, 429-1, 429-2, 429-3) included in a plurality of output channels (411, 413, 415, 417, 419) to emit light based on a timing signal of the camera (310).
[0188] The wearable device (200) can perform eye tracking or iris recognition using some of the plurality of output channels (411, 413, 415, 417, 419). For example, the wearable device (200) can perform eye tracking using a number of output channels greater than the number of output channels for iris recognition. For example, the wearable device (200) can perform eye tracking using output channels including an output channel for iris recognition. For example, the wearable device (200) can perform eye tracking using a different output channel that is distinct from the output channel for iris recognition.
[0189] The wearable device (200) can perform iris recognition using an output channel selected from among a plurality of output channels (411, 413, 415, 417, 419). For example, the wearable device (200) can perform iris recognition using an output channel selected based on the iris recognition result. For example, the wearable device (200) can perform iris recognition using an output channel selected based on user information. For example, the wearable device (200) can perform iris recognition using an output channel selected based on the surrounding environment.
[0190] FIG. 4b schematically illustrates an example of a circuit structure of light sources of a wearable device according to an embodiment of the disclosure.
[0191] The description of FIG. 4b may refer to components or structures of the electronic device (101) or wearable device (200) described through FIGS. 1, 2a, 2b, 2c, 2d, 2e, 2f, 3a, and 3b.
[0192] The circuit structures (401, 402) of FIG. 4b illustrate the circuit structures of light sources arranged in each of the structures (270-1, 270-2).
[0193] Referring to the circuit structure (401) of FIG. 4B, an equal number of light sources may be arranged in each of the multiple channels of the light source driver (290). For example, one light source may be arranged in each of the multiple channels of the light source driver (290).
[0194] The processor (120) may perform iris recognition of the user based on a light source (435) arranged in one channel (431) among ten channels of the circuit structure (401). In response to the iris recognition result based on the light source (435) arranged in one channel (431) among ten channels of the circuit structure (401), the processor (120) may perform iris recognition of the user based on a light source arranged in another channel. For example, in response to the iris recognition based on the light source (435) arranged in one channel (431) among ten channels of the circuit structure (401) failing, the processor (120) may select another channel among ten channels. The processor (120) may perform iris recognition of the user based on a light source arranged in the selected other channel. However, the present invention is not limited thereto. For example, the processor (120) may perform iris recognition of a user based on light sources included in a specified number of channels among 10 channels of the circuit structure (401). In response to the iris recognition result based on the light sources included in the specified number of channels of the circuit structure (401), the processor (120) may perform iris recognition of the user based on light sources in a specified number of other channels. Here, the specified number of other channels may be distinguished from the specified number of channels. However, the present invention is not limited thereto. Some of the specified number of other channels may be included in the specified number of channels.
[0195] Referring to the circuit structure (402) of FIG. 4B, an unequal number of light sources may be arranged in each of the multiple channels of the light source driver (290). For example, one or more light sources may be arranged in each of the multiple channels of the light source driver (290).
[0196] The processor (120) can perform iris recognition of a user based on light sources (442, 443, 446, 447) arranged in two channels (441, 445) among eight channels of the circuit structure (402). In response to the iris recognition result based on the light sources (442, 443) arranged in one channel (441) among the eight channels, the processor (120) can perform iris recognition of the user based on light sources (446, 447) arranged in another channel (445). For example, the processor (120) can sequentially perform iris recognition based on the channels (441, 445) for iris recognition. For example, in response to a failure of the iris recognition based on the light sources (442, 443) arranged in one of the eight channels (441) of the circuit structure (402), the processor (120) may select another channel (445) of the eight channels. The processor (120) may perform iris recognition of the user based on the light sources (446, 447) arranged in the selected other channel (445). However, the present invention is not limited thereto.
[0197] Each of the six channels, excluding two channels (441, 445) among the eight channels of the circuit structure (402), may include a resistance component.
[0198] FIG. 5A schematically illustrates an example of a layout structure of light sources of a wearable device according to an embodiment of the disclosure.
[0199] The description of FIG. 5a may refer to components or structures of the electronic device (101) or wearable device (200) described through FIGS. 1, 2a, 2b, 2c, 2d, 2e, 2f, 3a, and 3b.
[0200] Referring to FIG. 5A, a structure (270-1) for the left eye (265-1) is illustrated. The same circuit structure as illustrated in FIG. 5A may also be applied to the structure (270-2) for the right eye (265-2), which is not illustrated. Here, the application of the same structure to the structure (270-2) may mean that the structures (270-2) and (270-1) are mirror images of each other. The application of the same structure to the structure (270-2) may mean that the structures (270-2) and (270-1) are symmetrical to each other.
[0201] Referring to the arrangement structures (501, 502) of FIG. 5A, the structure (270-1) for the left eye (265-1) may be circular. Referring to the arrangement structures (501, 502), light sources (280) may be evenly arranged in the structure (270-1) for the left eye (265-1).
[0202] Referring to the arrangement structure (501), the light sources (280) of the structure (270-1) for the left eye (265-1) may be arranged at equal angles (511) with respect to each other. Here, the angle (511) may be an angle formed by two neighboring light sources from the center point of the light sources (280) arranged in the structure (270-1).
[0203] Referring to the arrangement structure (501), the light sources (280) of the structure (270-1) for the left eye (265-1) can be arranged at equal intervals (515) from each other. Here, the interval (515) can be the distance between two adjacent light sources (280) arranged in the structure (270-1).
[0204] Referring to the arrangement structure (502), a smaller number of light sources (280) may be arranged in the structure (270-1) compared to the arrangement structure (501). In the arrangement structure (502), the light sources (280) of the structure (270-1) for the left eye (265-1) may be arranged at equal angles (521) with respect to each other, or may be arranged at equal intervals (525) with respect to each other.
[0205] FIG. 5b schematically illustrates an example of a layout structure of light sources of a wearable device according to an embodiment of the disclosure.
[0206] The description of FIG. 5b may refer to components or structures of the electronic device (101) or wearable device (200) described through FIGS. 1, 2a, 2b, 2c, 2d, 2e, 2f, 3a, and 3b.
[0207] Referring to FIG. 5B, a structure (270-1) for the left eye (265-1) is illustrated. The same circuit structure as illustrated in FIG. 5B may also be applied to the structure (270-2) for the right eye (265-2), which is not illustrated. Here, the application of the same structure to the structure (270-2) may mean that the structures (270-2) and (270-1) are mirror images of each other. The application of the same structure to the structure (270-2) may mean that the structures (270-2) and (270-1) are symmetrical to each other.
[0208] Referring to the arrangement structures (503, 504) of FIG. 5B, the structure (270-1) for the left eye (265-1) may be elliptical. Referring to the arrangement structures (503, 504), light sources (280) may be evenly arranged in the structure (270-1) for the left eye (265-1).
[0209] Referring to the arrangement structure (503), the light sources (280) of the structure (270-1) for the left eye (265-1) may be arranged at equal angles (531) with respect to each other. Referring to the arrangement structure (503), the light sources (280) of the structure (270-1) for the left eye (265-1) may not be arranged at equal intervals with respect to each other. The separation distance between two adjacent light sources among the light sources (280) of the structure (270-1) may not be equal. Here, the angle (531) may be the angle formed by two adjacent light sources from the center point of the light sources (280) arranged in the structure (270-1).
[0210] Referring to the arrangement structure (504), the light sources (280) of the structure (270-1) for the left eye (265-1) may be arranged at equal intervals (541) with respect to each other. Referring to the arrangement structure (504), the light sources (280) of the structure (270-1) for the left eye (265-1) may not be arranged at equal angles with respect to each other. The angle formed by two neighboring light sources from the center point of the light sources (280) of the structure (270-1) may not be equal. Here, the interval (541) may be the distance between two neighboring light sources among the light sources (280) arranged in the structure (270-1).
[0211] FIG. 5c schematically illustrates an example of a layout structure of light sources of a wearable device according to an embodiment of the disclosure.
[0212] The description of FIG. 5c may refer to components or structures of the electronic device (101) or wearable device (200) described through FIGS. 1, 2a, 2b, 2c, 2d, 2e, 2f, 3a, and 3b.
[0213] Referring to FIG. 5C, a structure (270-1) for the left eye (265-1) is illustrated. The same circuit structure as illustrated in FIG. 5C may also be applied to the structure (270-2) for the right eye (265-2), which is not illustrated. Here, the application of the same structure to the structure (270-2) may mean that the structures (270-2) and (270-1) are mirror images of each other. The application of the same structure to the structure (270-2) may mean that the structures (270-2) and (270-1) are symmetrical to each other.
[0214] Referring to the arrangement structures (505, 506) of FIG. 5C, the structure (270-1) for the left eye (265-1) may be elliptical. Referring to the arrangement structures (505, 506), light sources (280) may be arranged in the structure (270-1) for the left eye (265-1). Referring to the arrangement structures (505, 506), the light sources (280) may be arranged more intensively in at least some areas of the left eye (265-1) than in other areas. Here, the arrangement of the light sources (280) in an intensive manner in some areas may mean that the angle formed by the light sources (280) arranged in some areas is narrower than the angle formed by the light sources (280) arranged in other areas. The fact that light sources (280) are concentrated in some areas may mean that the interval between light sources (280) arranged in some areas is shorter than the interval between light sources (280) arranged in other areas.
[0215] Referring to the arrangement structure (505), the light sources (280) of the structure (270-1) for the left eye (265-1) may be arranged in a concentrated manner in the area (551). For example, the area (551) may be positioned relatively upward with respect to the structure (270-1). Here, the area (551) being positioned relatively upward with respect to the structure (270-1) may mean that the area (551) is arranged in the direction of the upper eyelid with respect to the pupil of the user's left eye (265-1) when the user wears the wearable device (200). The area (551) being positioned relatively upward with respect to the structure (270-1) may mean that the area (551) is arranged in the direction of the upper eyelid with respect to the pupil of the user's left eye (265-1) when the user wearing the wearable device (200) looks straight ahead. However, the present invention is not limited thereto. The region (551) may be positioned relatively lower, right, or left with respect to the structure (270-1). Here, the region (551) being positioned relatively lower with respect to the structure (270-1) may mean that when the user wears the wearable device (200), the region (551) is arranged in the direction of the lower eyelid with respect to the pupil of the user's left eye (265-1). The region (551) being positioned relatively right with respect to the structure (270-1) may mean that when the user wears the wearable device (200), the region (551) is arranged in the direction of the medial canthus with respect to the pupil of the user's left eye (265-1). The fact that the region (551) is positioned relatively to the left with respect to the structure (270-1) may mean that when the user wears the wearable device (200), it is arranged in the direction of the lateral canthus with respect to the pupil of the user's left eye (265-1). The fact that the region (551) is positioned relatively to the left with respect to the structure (270-2) may mean that when the user wears the wearable device (200), it is arranged in the direction of the medial canthus with respect to the pupil of the user's right eye (265-2).Positioning relatively to the right with respect to the structure (270-2) may mean that when the user wears the wearable device (200), it is arranged in the direction of the outer corner with respect to the pupil of the user's right eye (265-2).
[0216] Referring to the arrangement structure (506), the light sources (280) of the structure (270-1) for the left eye (265-1) may be concentratedly arranged in two or more regions (561, 562). For example, the two or more regions (561, 562) may be symmetrical with respect to the structure (270-1). Here, the two or more regions (561, 562) being symmetrical with respect to the structure (270-1) may mean that at least two regions among the two or more regions (561, 562) are positioned to be opposite to each other with respect to the center point of the structure (270-1). However, the present invention is not limited thereto. For example, the regions (561, 562) may not be symmetrical with respect to the structure (270-1). For example, region (561) may be positioned relatively above the center point of the structure (270-1), and region (562) may be positioned relatively to the right of the center point of the structure (270-1).
[0217] FIG. 5d schematically illustrates an example of a layout structure of light sources of a wearable device according to an embodiment of the disclosure.
[0218] The description of FIG. 5d may refer to components or structures of the electronic device (101) or wearable device (200) described through FIGS. 1, 2a, 2b, 2c, 2d, 2e, 2f, 3a, and 3b.
[0219] Referring to FIG. 5D, a structure (270-1) for the left eye (265-1) is illustrated. The same circuit structure as illustrated in FIG. 5D may also be applied to the structure (270-2) for the right eye (265-2), which is not illustrated. Here, the application of the same structure to the structure (270-2) may mean that the structures (270-2) and (270-1) are mirror images of each other. The application of the same structure to the structure (270-2) may mean that the structures (270-2) and (270-1) are symmetrical to each other.
[0220] Referring to the layout structure (507) of FIG. 5d, the structure (270-1) for the left eye (265-1) may be rectangular. The layout structure (507) may be exemplified as having a corner (or vertex) of the rectangular structure (270-1) having a rounded shape. However, the present invention is not limited thereto.
[0221] Referring to the arrangement structure (507), the light sources (280) of the structure (270-1) for the left eye (265-1) may be arranged in regions (571, 572, 573, 574). For example, the region (571) may be positioned relatively above the structure (270-1). For example, the region (572) may be positioned relatively below the structure (270-1). For example, the region (573) may be positioned relatively to the right of the structure (270-1). For example, the region (574) may be positioned relatively to the left of the structure (270-1).
[0222] Different numbers of light sources (280) may be arranged in regions (571, 572, 573, 574). For example, the number of light sources (280) arranged in regions (571, 572) may be different from the number of light sources (280) arranged in regions (573, 574). However, this is not limited thereto. The number of light sources (280) arranged in region (571) may be different from the number of light sources (280) arranged in region (572). The number of light sources (280) arranged in region (573) may be different from the number of light sources (280) arranged in region (574).
[0223] FIG. 6 schematically illustrates an example of angles of light sources of a wearable device according to an embodiment of the disclosure.
[0224] The description of FIG. 6 may refer to components or structures of the electronic device (101) or wearable device (200) described through FIGS. 1, 2a, 2b, 2c, 2d, 2e, 2f, 3a, and 3b.
[0225] Referring to FIG. 6, a structure (270-1) for the left eye (265-1) is illustrated. The same structure as the circuit structure illustrated in FIG. 6 may also be applied to the structure (270-2) for the right eye (265-2), which is not illustrated. Here, the application of the same structure to the structure (270-2) may mean that the structures (270-2) and (270-1) are mirror images of each other. The application of the same structure to the structure (270-2) may mean that the structures (270-2) and (270-1) are symmetrical to each other.
[0226] The cut drawing (602) may illustrate a cross-section of the arrangement structure (601) cut by a line including AB.
[0227] Referring to the arrangement structure (601) of FIG. 6, the structure (270-1) for the left eye (265-1) may be circular. Referring to the arrangement structure (601), light sources (280) may be evenly arranged in the structure (270-1) for the left eye (265-1). However, the present invention is not limited thereto. The structure (270-1) may be polygonal, elliptical, or oval. At least some of the light sources (280) may be arranged unevenly in the structure (270-1).
[0228] Some of the light sources (280) may be arranged to have a predetermined angle with respect to an axis (640) perpendicular to a plane perpendicular to the XY plane from the center point (630) of the structure (270-1). Here, the angle that the light source has with respect to the axis (640) may mean the angle between the optical axis of the light source and the axis (640). For example, the angle between the optical axes (615, 625) of the light sources (610, 620) and the axis (640) may be the angle that the light sources (610, 620) have with respect to the axis (640). Here, the center point (630) may be the point where the pupil of the left eye (265-1) is located when a user wearing the wearable device (200) looks straight ahead. However, the present invention is not limited thereto.
[0229] For example, some of the light sources (280) may be arranged parallel to the axis (640). For example, a light source located at the top of the light sources (280) (e.g., light source (610)) may be arranged parallel to the axis (640). For example, the optical axis (615) of the light source (610) may be arranged parallel to the axis (640). However, the present invention is not limited thereto.
[0230] For example, some of the light sources (280) may be arranged to form an acute angle with the axis (640). For example, some of the light sources (280) may be arranged such that their optical axes form an acute angle with the axis (640). For example, a light source located at the bottom of the light sources (280) may be arranged to form an acute angle with the axis (640). For example, the light source (620) may be arranged to form an acute angle with the axis (640). For example, the optical axis (625) of the light source (620) may form an acute angle with the axis (640). For example, the light sources (620) may be arranged such that the optical axis (625) of the light source (620) forms a 30 degree angle with the axis (640). Here, when the light source forms an acute angle with the axis (640), the optical axis of the light source may be directed toward the pupil of the left eye (265-1). However, this is not limited thereto.
[0231] FIG. 7 is a flowchart of a method executed on a wearable device according to an embodiment of the disclosure.
[0232] The description of FIG. 7 may refer to components or structures of the electronic device (101) or wearable device (200) described through FIGS. 1, 2a, 2b, 2c, 2d, 2e, 2f, 3a, and 3b.
[0233] Referring to FIG. 7, in operation 710, the processor (120) may receive an input for emitting light from a light source. Here, the input for emitting light from a light source may include an input for eye tracking and / or an input for iris recognition.
[0234] The processor (120) can obtain input for eye tracking. For example, the processor (120) can obtain an eye tracking request generated by the program (140) while the program (140) is being executed. For example, the processor (120) can obtain that a specified program (140) is being executed as input for eye tracking. However, the present invention is not limited thereto. For example, the processor (120) can obtain a user input requesting eye tracking. For example, the processor (120) can obtain that a user is wearing a wearable device (200) as input for eye tracking.
[0235] The processor (120) may obtain input for iris recognition. For example, the processor (120) may obtain an iris recognition request generated by the program (140) while the program (140) is running. For example, the processor (120) may obtain a user authentication request by the program (140) as an iris recognition request. However, the present invention is not limited thereto. For example, the processor (120) may obtain a user input requesting iris recognition.
[0236] In operation 720, the processor (120) can emit light corresponding to the input.
[0237] The processor (120) may control two or more light sources to emit light in response to an input for eye tracking. For example, the processor (120) may control two or more light sources to emit light through at least one channel (e.g., channels (291-1, 291-2)) among the plurality of channels (291-1, 291-2, 291-3, 291-4) in response to an input for eye tracking.
[0238] The processor (120) can control two or more light sources among the light sources (281-1, 281-2, 281-3, 281-4, 281-5, 282-1, 282-2, 282-3, 282-4, 282-5) corresponding to the left eye (265-1) to emit light in response to an input for eye tracking. The processor (120) can control two or more light sources among the light sources (283-1, 283-2, 283-3, 283-4, 283-5, 284-1, 284-2, 284-3, 284-4, 284-5) corresponding to the right eye (265-2) to emit light in response to an input for eye tracking.
[0239] The processor (120) may control at least one light source to emit light in response to an input for iris recognition. For example, the processor (120) may control at least one light source to emit light through at least one channel (e.g., channel (291-1)) among a plurality of channels (291-1, 291-2, 291-3, 291-4) for the left eye (265-1) in response to an input for iris recognition. For example, the processor (120) may control at least one light source to emit light through at least one channel (not shown) among a plurality of channels for the right eye (265-2) in response to an input for iris recognition.
[0240] The processor (120) may select at least one light source among the light sources (280) in response to an input for iris recognition. For example, the processor (120) may select at least one light source based on user information. For example, the processor (120) may select at least one light source used in a previous iris recognition of the user. Here, the at least one light source used in a previous iris recognition of the user may be at least one light source used when the iris recognition was successful. For example, the processor (120) may select at least one light source designated in advance when the user first attempts iris recognition. For example, the processor (120) may select at least one light source based on situational information. For example, the processor (120) may select at least one light source designated by the program (140) that requested iris recognition. The processor (120) may control the at least one selected light source to emit light.
[0241] The processor (120) can control at least one of the light sources (281-1, 281-2, 281-3, 281-4, 281-5, 282-1, 282-2, 282-3, 282-4, 282-5) corresponding to the left eye (265-1) to emit light in response to an input for iris recognition. The processor (120) can control at least one of the light sources (283-1, 283-2, 283-3, 283-4, 283-5, 284-1, 284-2, 284-3, 284-4, 284-5) corresponding to the right eye (265-2) to emit light in response to an input for iris recognition.
[0242] FIG. 8 is a flowchart of a method executed on a wearable device according to an embodiment of the disclosure.
[0243] The description of FIG. 8 may refer to components or structures of the electronic device (101) or wearable device (200) described through FIGS. 1, 2a, 2b, 2c, 2d, 2e, 2f, 3a, and 3b. The operations of FIG. 8 may be included in operation 720 of FIG. 7.
[0244] Referring to FIG. 8, in operation 810, the processor (120) may cause the selected light source to emit light.
[0245] The processor (120) can control the selected light source to emit light. The processor (120) can control the selected light source to emit light in response to an input for eye tracking. The processor (120) can control the selected light source to emit light in response to the input for iris recognition.
[0246] The processor (120) can obtain an image for iris recognition through the camera (310) while the selected light source is emitting light. In response to the input for iris recognition, the processor (120) can obtain an image for iris recognition through the camera (310) while the selected light source is emitting light. During eye tracking, the processor (120) can obtain an image for iris recognition and eye tracking through the camera (310) while the selected light source is emitting light. The processor (120) can obtain an image for eye tracking through the camera (310) while the selected light source is emitting light. The processor (120) can obtain an image for eye tracking through the camera (310) while the selected light source is emitting light, in response to the input for eye tracking.
[0247] In operation 820, the processor (120) may recognize the iris based on the acquired image. The processor (120) may identify the location of the iris of each of the user's eyes (265-1, 265-2) based on the image for eye tracking. The processor (120) may perform iris recognition based on the image for iris recognition. The processor (120) may perform iris recognition and / or eye tracking based on an image acquired through the camera (310) in response to the input for iris recognition and / or the input for eye tracking. The processor (120) may perform iris recognition and eye tracking based on an image acquired in response to the input for eye tracking. The processor (120) may perform iris recognition based on an image acquired in response to the input for iris recognition. According to an embodiment, the processor (120) may identify the location of the iris of each of the user's eyes (265-1, 265-2) based on an image for eye tracking, and perform eye tracking based on the location of the identified iris.
[0248] In operation 830, the processor (120) can determine whether iris recognition is successful.
[0249] The processor (120) can determine, in response to the input for iris recognition, whether the iris recognition based on the image for iris recognition is successful. The processor (120) can determine, in response to the input for eye tracking, whether the iris recognition based on the image for eye tracking is successful. During eye tracking, the processor (120) can determine, in response to the input for iris recognition, whether the iris recognition based on the image for eye tracking is successful. The processor (120) can determine, in response to the input for eye tracking, whether the eye tracking based on the image for eye tracking is successful. The processor (120) can determine, in response to the input for eye tracking, whether the user's gaze based on the image for eye tracking is identified.
[0250] In operation 830, if the iris recognition based on the image for iris recognition is successful, the processor (120) may end the operation of FIG. 8. In operation 830, if the iris recognition based on the image for iris recognition fails, the processor (120) may perform operation 840.
[0251] In operation 830, if the iris recognition based on the image for eye tracking is successful, the processor (120) may terminate the operation of FIG. 8. In operation 830, if the iris recognition based on the image for eye tracking is unsuccessful, the processor (120) may perform operation 840.
[0252] In operation 830, if the iris recognition and eye tracking based on the image for eye tracking are successful, the processor (120) may terminate the operation of FIG. 8. In operation 830, if at least one of the iris recognition or eye tracking based on the image for eye tracking fails, the processor (120) may perform operation 840.
[0253] In operation 830, if iris recognition is successful, the processor (120) may return to a previous state. For example, if iris recognition in response to an input for iris recognition is successful, the processor (120) may return to a previous state. Here, the previous state may be a state prior to acquiring the input for iris recognition. For example, if the previous state was performing eye tracking prior to the input for iris recognition, the processor (120) may perform eye tracking again if iris recognition is successful.
[0254] In operation 840, the processor (120) may select a light source. The processor (120) may select a light source that is at least partially different from the light source selected for the previous operation. The processor (120) may select a light source that is at least partially different from the light source selected for the previous operations 810 and 820.
[0255] The processor (120) may select a different light source than the light source previously selected for iris recognition. The processor (120) may select a different light source than the light source previously selected for eye tracking. Here, the different light source may be selected based on user information. For example, the different light source may be a light source used when a previous operation was successful. For example, the different light source may be a light source designated by the program (140). However, the present invention is not limited thereto. The different light source may include the previously selected light source. When performing operation 810 after operation 840, the processor (120) may change at least one of the intensity, the emission time, or the frequency of the emission light of the selected light source. For example, when the different light source includes the previously selected light source, the processor (120) may change at least one of the intensity, the emission time, or the frequency of the emission light of the selected light source when performing operation 810 after operation 840.
[0256] FIG. 9 is a flowchart of a method executed on a wearable device according to an embodiment of the disclosure.
[0257] The description of FIG. 9 may refer to components or structures of the electronic device (101) or wearable device (200) described through FIGS. 1, 2a, 2b, 2c, 2d, 2e, 2f, 3a, and 3b.
[0258] Operations 810, 820, and 830 included in the flowchart of FIG. 9 may be identical to operations 810, 820, and 830 described with reference to FIG. 8.
[0259] Referring to FIG. 9, in operation 901, the processor (120) may perform eye tracking. In response to an input for eye tracking, the processor (120) may control a light source for eye tracking to emit light. The processor (120) may acquire an image for eye tracking based on a first camera for eye tracking among a plurality of cameras. The processor (120) may acquire an image for eye tracking based on the first camera for eye tracking among a plurality of cameras, based on an input for eye tracking. The processor (120) may identify the user's gaze. The processor (120) may track the user's gaze. For example, the processor (120) may identify the location of the iris of each of the user's eyes based on the image for eye tracking. For example, the processor (120) may track the user's gaze based on the location of the iris. However, the present invention is not limited thereto. For example, the processor (120) can track the user's gaze based on the location of the iris identified based on an image corresponding to one of the user's eyes.
[0260] At operation 902, the processor (120) may receive an input for iris recognition. For example, the processor (120) may obtain an iris recognition request generated by the program (140) while the program (140) is running. For example, the processor (120) may obtain a user authentication request by the program (140) as an iris recognition request. However, the present invention is not limited thereto. For example, the processor (120) may obtain a user input requesting iris recognition.
[0261] In operation 903, the processor (120) may select a light source corresponding to the input. In response to the input for iris recognition, the processor (120) may select at least one light source among the light sources. For example, the processor (120) may select at least one light source based on user information. For example, the processor (120) may select at least one light source used in a previous iris recognition of the user. Here, the at least one light source used in a previous iris recognition of the user may be at least one light source used when the iris recognition was successful. For example, the processor (120) may select at least one light source designated in advance when the user first attempts iris recognition. For example, the processor (120) may select at least one light source based on contextual information. For example, the processor (120) may select at least one light source designated by the program (140) that requested iris recognition. The processor (120) may control the at least one selected light source to emit light.
[0262] In operation 810, the processor (120) can cause the selected light source to emit light.
[0263] In operation 820, the processor (120) can recognize the iris based on the acquired image.
[0264] In operation 830, the processor (120) can determine whether iris recognition is successful.
[0265] In operation 830, if the iris recognition based on the image for iris recognition is successful, the processor (120) may end the operation of FIG. 8. In operation 830, if the iris recognition based on the image for iris recognition is unsuccessful, the processor (120) may perform operation 910.
[0266] In operation 830, if iris recognition is successful, the processor (120) may return to a previous state. For example, if iris recognition in response to an input for iris recognition is successful, the processor (120) may return to a previous state. Here, the previous state may be a state prior to acquiring the input for iris recognition. For example, if the previous state was performing eye tracking prior to the input for iris recognition, the processor (120) may perform eye tracking again if iris recognition is successful.
[0267] At operation 910, the processor (120) may determine whether to stop eye tracking. The processor (120) may determine whether to reacquire an image for iris recognition during the eye tracking.
[0268] For example, if the processor (120) determines that simultaneous execution of eye tracking and iris recognition is possible, the processor (120) may determine not to stop eye tracking. For example, if the processor (120) determines that simultaneous execution of eye tracking and iris recognition is difficult, the processor (120) may determine to stop eye tracking. Here, simultaneous execution of eye tracking and iris recognition may mean performing iris recognition on an image for eye tracking. Simultaneous execution of eye tracking and iris recognition may mean performing eye tracking and iris recognition on an image for eye tracking. Simultaneous execution of eye tracking and iris recognition may mean acquiring an image for eye tracking with a smaller number of light sources than the number of light sources used when only eye tracking is performed. Simultaneous execution of eye tracking and iris recognition may mean that at least one combination of light sources for eye tracking and iris recognition exists among a plurality of combinations of light sources.
[0269] In operation 910, in response to a determination to continue the eye tracking, the processor (120) may perform operation 920. In operation 910, in response to a determination to stop the eye tracking, the processor (120) may perform operation 930.
[0270] At step 920, the processor (120) may select a first set of light sources. In response to determining not to stop the eye tracking, the processor (120) may perform iris recognition using a first set of light sources different from the previously selected light sources. Here, the first set of light sources may not be included in the previously selected light sources. However, this is not limited thereto. For example, the first set of light sources may be a part of the previously selected light sources. Here, the first set may include at least one combination of light sources for eye tracking and iris recognition.
[0271] At step 930, the processor (120) may select a second set of light sources. In response to determining to stop the eye tracking, the processor (120) may perform iris recognition using a second set of light sources different from the previously selected light sources. Here, the second set of light sources may be light sources used in a previous successful iris recognition, but is not limited thereto. Here, the second set may include at least one combination of light sources for iris recognition.
[0272] FIG. 10A illustrates an example of the appearance of a first side of a wearable device according to an embodiment of the disclosure. FIG. 10B illustrates an example of the appearance of a second side of a wearable device according to an embodiment of the disclosure.
[0273] Referring to FIGS. 10A and 10B, a wearable device (1000) may be included in the electronic device (101) of FIG. 1.
[0274] FIG. 10A illustrates an example of a perspective view of a wearable device according to one embodiment. FIG. 10B illustrates an example of one or more hardware elements positioned within the wearable device according to one embodiment.
[0275] Referring to FIGS. 10A and 10B , a wearable device (1000) may include the first terminal (120-1) of FIGS. 1 and 2A to 2F . As shown in FIG. 10A , according to one embodiment, the wearable device (1000) may include at least one display (1050) and a frame supporting the at least one display (1050).
[0276] According to one embodiment, a wearable device (1000) may be worn on a part of a user's body. The wearable device (1000) may provide augmented reality (AR), virtual reality (VR), or mixed reality (MR) that combines augmented reality and virtual reality to a user wearing the wearable device (1000). For example, the wearable device (1000) may output a virtual reality image to the user through at least one display (1050) in response to a user's designated gesture acquired through the motion recognition cameras (1040-2, 1040-5) of FIG. 10B.
[0277] According to one embodiment, at least one display (1050) within a wearable device (1000) may provide visual information to a user. For example, at least one display (1050) may include a transparent or translucent lens. At least one display (1050) may include a first display (1050-1) and / or a second display (1050-2) spaced apart from the first display (1050-1). For example, the first display (1050-1) and the second display (1050-2) may be positioned at positions corresponding to the user's left and right eyes, respectively.
[0278] Referring to FIG. 10B, at least one display (1050) may form a display area on a lens to provide a user wearing the wearable device (1000) with visual information contained in external light passing through the lens, along with other visual information distinct from the visual information. The lens may be formed based on at least one of a Fresnel lens, a pancake lens, or a multi-channel lens. The display area formed by at least one display (1050) may be formed on the second surface (1032) among the first surface (1031) and the second surface (1032) of the lens. When the user wears the wearable device (1000), external light may be incident on the first surface (1031) and transmitted through the second surface (1032) to the user. As another example, at least one display (1050) may display a virtual reality image to be combined with a real screen transmitted via external light. The virtual reality image output from the at least one display (1050) may be transmitted to the user's eyes via one or more hardware included in the wearable device (1000) (e.g., optical devices (1082, 1084) and / or at least one waveguide (1033, 1034)).
[0279] According to one embodiment, a wearable device (1000) may include waveguides (1033, 1034) that diffract light transmitted from at least one display (1050) and relayed by optical devices (1082, 1084) and transmit the diffracted light to a user. The waveguides (1033, 1034) may be formed based on at least one of glass, plastic, or polymer. Nanopatterns may be formed on at least a portion of the exterior or interior of the waveguides (1033, 1034). The nanopatterns may be formed based on a grating structure having a polygonal and / or curved shape. Light incident on one end of the waveguides (1033, 1034) may be propagated to the other end of the waveguides (1033, 1034) by the nanopatterns. The waveguides (1033, 1034) may include at least one diffractive element (e.g., a diffractive optical element (DOE), a holographic optical element (HOE)) and at least one reflective element (e.g., a reflective mirror). For example, the waveguides (1033, 1034) may be arranged within the wearable device (1000) to guide a screen displayed by at least one display (1050) to the user's eyes. For example, the screen may be transmitted to the user's eyes based on total internal reflection (TIR) occurring within the waveguides (1033, 1034).
[0280] According to one embodiment, the wearable device (1000) may analyze an object included in a real image collected through a capturing camera (1040-3), combine a virtual object corresponding to an object to be provided with augmented reality among the analyzed objects, and display the virtual object on at least one display (1050). The virtual object may include at least one of text and an image regarding various information related to the object included in the real image. The wearable device (1000) may analyze the object based on a multi-camera, such as a stereo camera. For the object analysis, the wearable device (1000) may execute time-of-flight (ToF) and / or simultaneous localization and mapping (SLAM) supported by the multi-camera. A user wearing the wearable device (1000) may view an image displayed on at least one display (1050).
[0281] According to one embodiment, the frame may be formed as a physical structure that allows the wearable device (1000) to be worn on the user's body. According to one embodiment, the frame may be configured so that, when the user wears the wearable device (1000), the first display (1050-1) and the second display (1050-2) can be positioned corresponding to the user's left and right eyes. The frame may support at least one display (1050). For example, the frame may support the first display (1050-1) and the second display (1050-2) to be positioned corresponding to the user's left and right eyes.
[0282] Referring to FIG. 10A, the frame may include a region (1020) that at least partially contacts a portion of the user's body when the user wears the wearable device (1000). For example, the region (1020) of the frame that contacts a portion of the user's body may include a region that contacts a portion of the user's nose, a portion of the user's ear, and a portion of the side of the user's face that the wearable device (1000) makes contact with. According to one embodiment, the frame may include a nose pad (1010) that contacts a portion of the user's body. When the wearable device (1000) is worn by the user, the nose pad (1010) may contact a portion of the user's nose. The frame may include a first temple (1004) and a second temple (1005) that contact a portion of the user's body that is distinct from the portion of the user's body.
[0283] For example, the frame may include a first rim (1001) that surrounds at least a portion of the first display (1050-1), a second rim (1002) that surrounds at least a portion of the second display (1050-2), a bridge (1003) that is disposed between the first rim (1001) and the second rim (1002), a first pad (1011) that is disposed along a portion of an edge of the first rim (1001) from one end of the bridge (1003), a second pad (1012) that is disposed along a portion of an edge of the second rim (1002) from the other end of the bridge (1003), a first temple (1004) that extends from the first rim (1001) and is secured to a portion of an ear of the wearer, and a second temple (1005) that extends from the second rim (1002) and is secured to a portion of an ear opposite the ear. The first pad (1011) and the second pad (1012) may be in contact with a portion of the user's nose, and the first temple (1004) and the second temple (1005) may be in contact with a portion of the user's face and a portion of the user's ear. The temples (1004, 1005) may be rotatably connected to the rim through the hinge units (1006, 1007) of FIG. 10B. The first temple (1004) may be rotatably connected to the first rim (1001) through the first hinge unit (1006) disposed between the first rim (1001) and the first temple (1004). The second temple (1005) may be rotatably connected to the second rim (1002) via a second hinge unit (1007) disposed between the second rim (1002) and the second temple (1005). In one embodiment, the wearable device (1000) may use a touch sensor, a grip sensor, and / or a proximity sensor formed on at least a portion of a surface of the frame to identify an external object (e.g., a user's fingertip) touching the frame and / or a gesture performed by the external object.
[0284] According to one embodiment, the wearable device (1000) may include hardware that performs various functions. For example, the hardware may include at least one display (1050), one or more cameras (1040-1, 1040-2, 1040-3, 1040-4, 1040-5, 1040-6), a battery module (1070), an antenna module (1075), optical devices (1082, 1084), speakers (1092-1, 1092-2), microphones (1094-1, 1094-2, 1094-3), a light-emitting module (not shown), and / or a printed circuit board (1090). The various hardware may be arranged within a frame. At least one display (1050) of FIG. 10B may correspond to the display module (160) of FIG. 1. At least one display (1050) of FIG. 10b may correspond to the displays (250-1, 250-2) of FIG. 2a. One or more cameras (1040-1, 1040-2, 1040-3, 1040-4, 1040-5, 1040-6) of FIG. 10b may correspond to the camera module (180) of FIG. 1. One or more cameras (1040-1, 1040-2, 1040-3, 1040-4, 1040-5, 1040-6) of FIG. 10b may correspond to cameras (240-1, 240-2, 240-3, 240-4, 240-5, 240-6, 240-7, 240-8, 240-9, 240-10, 240-11) of FIG. 2a or 2b. The battery module (1070) of FIG. 10b may correspond to the battery (189) of FIG. 1. The antenna module (1075) of FIG. 10b may correspond to the antenna module (197) of FIG. 1. The speakers (1092-1, 1092-2) of FIG. 10b may correspond to the audio output module (155) of FIG. 1. The microphones (1094-1, 1094-2, 1094-3) of FIG. 10b may correspond to the input module (150) of FIG. 1. The light-emitting module (not shown) of FIG. 10b may correspond to one or more light sources (280) of FIG. 2d.The printed circuit board (1090) of FIG. 10b may correspond to the structures (270-1, 270-2, 270-3, 270-4) of FIG. 2d.
[0285] According to one embodiment, microphones (1094-1, 1094-2, 1094-3) of the wearable device (1000) may be arranged on at least a portion of the frame to acquire sound signals. A first microphone (1094-1) arranged on the nose pad (1010), a second microphone (1094-2) arranged on the second rim (1002), and a third microphone (1094-3) arranged on the first rim (1001) are illustrated in FIG. 10B , but the number and arrangement of the microphones (1094) are not limited to the embodiment of FIG. 10B . When the number of microphones (1094) included in the wearable device (1000) is two or more, the wearable device (1000) may identify the direction of the sound signal by using a plurality of microphones arranged on different portions of the frame.
[0286] According to one embodiment, the optical devices (1082, 1084) can transmit virtual objects transmitted from at least one display (1050) to the wave guides (1033, 1034). For example, the optical devices (1082, 1084) can be projectors. The optical devices (1082, 1084) can be positioned adjacent to at least one display (1050) or can be included within at least one display (1050) as a part of the at least one display (1050). The first optical device (1082) can correspond to the first display (1050-1), and the second optical device (1084) can correspond to the second display (1050-2). The first optical device (1082) can transmit light output from the first display (1050-1) to the first waveguide (1033), and the second optical device (1084) can transmit light output from the second display (1050-2) to the second waveguide (1034).
[0287] In one embodiment, the camera may include an eye tracking camera (ET CAM) (1040-1, 1040-4), a motion recognition camera (1040-2, 1040-5), and / or a recording camera (1040-3, 1040-6). The recording camera, the eye tracking camera (1040-1, 1040-4), and the motion recognition camera (1040-2, 1040-5) may be positioned at different locations on the frame and may perform different functions. The eye tracking camera (1040-1) may output data representing the gaze of a user wearing the wearable device (1000). For example, the wearable device (1000) may detect the gaze from an image including the user's pupils obtained through the eye tracking camera (1040-1). An example in which the gaze tracking cameras (1040-1, 1040-4) are positioned toward both eyes of the user is illustrated in FIG. 10b, but the embodiment is not limited thereto, and the gaze tracking cameras (1040-1, 1040-4) may be positioned singly toward the user's left eye, or may be positioned toward the left eye.
[0288] In one embodiment, the capturing cameras (1040-3, 1040-6) can capture an actual image or background to be aligned with a virtual image to implement augmented reality or mixed reality content. The capturing cameras can capture an image of a specific object existing at a location viewed by the user and provide the image to at least one display (1050). The at least one display (1050) can display a single image in which information about the actual image or background including the image of the specific object acquired using the capturing cameras is superimposed on a virtual image provided through optical devices (1082, 1084). In one embodiment, the capturing cameras can be placed on a bridge (1003) disposed between the first rim (1001) and the second rim (1002).
[0289] In one embodiment, the gaze tracking cameras (1040-1, 1040-4) can track the gaze of a user wearing the wearable device (1000) to match the user's gaze with visual information provided to at least one display (1050) to implement more realistic augmented reality. For example, the wearable device (1000) can naturally display environmental information related to the user's front at a location where the user is located on at least one display (1050) when the user looks straight ahead. The gaze tracking cameras (1040-1, 1040-4) can be configured to capture an image of the user's pupil to determine the user's gaze. For example, the gaze tracking cameras (1040-1, 1040-4) can receive gaze detection light reflected from the user's pupil and track the user's gaze based on the position and movement of the received gaze detection light. In one embodiment, the gaze tracking cameras (1040-1, 1040-4) may be positioned at positions corresponding to the user's left and right eyes. For example, the gaze tracking cameras (1040-1, 1040-4) may be positioned within the first rim (1001) and / or the second rim (1002) so as to face the direction in which the user wearing the wearable device (1000) is positioned.
[0290] In one embodiment, the gesture recognition camera (1040-2, 1040-5) can provide a specific event on a screen provided on at least one display (1050) by recognizing the movement of the user's whole or part of the body, such as the user's torso, hand, or face. The gesture recognition camera (1040-2, 1040-5) can recognize the user's gesture (gesture recognition), obtain a signal corresponding to the gesture, and provide an indication corresponding to the signal on at least one display (1050). The processor can identify the signal corresponding to the gesture and perform a designated function based on the identification. In one embodiment, the gesture recognition camera (1040-2, 1040-5) can be disposed on the first limb (1001) and / or the second limb (1002).
[0291] In one embodiment, the camera (1040) included in the wearable device (1000) is not limited to the gaze tracking cameras (1040-1, 1040-4) and motion recognition cameras (1040-2, 1040-5) described above. For example, the wearable device (1000) can identify an external object included in the FoV using the photographing cameras (1040-3, 1040-6) positioned toward the user's FoV. The wearable device (1000) can identify an external object based on a sensor for identifying the distance between the wearable device (1000) and the external object, such as a depth sensor and / or a time of flight (ToF) sensor. The camera positioned toward the FoV can support an autofocus function and / or an optical image stabilization (OIS) function. For example, the wearable device (1000) may include a camera (e.g., a face tracking (FT) camera) positioned toward the face to obtain an image including the face of a user wearing the wearable device (1000).
[0292] Although not shown, in one embodiment, the wearable device (1000) may further include a light source (e.g., an LED) that emits light toward a subject being photographed using the camera (e.g., a user's eyes, face, and / or an external object within the FoV). The light source may include an infrared wavelength LED. The light source may be disposed on at least one of the frame and hinge units (1006, 1007).
[0293] According to one embodiment, the battery module (1070) may supply power to electronic components of the wearable device (1000). In one embodiment, the battery module (1070) may be disposed within the first temple (1004) and / or the second temple (1005). For example, the battery module (1070) may be a plurality of battery modules (1070). The plurality of battery modules (1070) may be disposed within each of the first temple (1004) and the second temple (1005). In one embodiment, the battery module (1070) may be disposed at an end of the first temple (1004) and / or the second temple (1005).
[0294] In one embodiment, the antenna module (1075) may transmit signals or power to the outside of the wearable device (1000), or receive signals or power from the outside. The antenna module (1075) may be electrically and / or operatively connected to the communication circuitry of the wearable device (1000). In one embodiment, the antenna module (1075) may be disposed within the first temple (1004) and / or the second temple (1005). For example, the antenna module (1075) may be disposed close to one surface of the first temple (1004) and / or the second temple (1005).
[0295] According to one embodiment, speakers (1092-1, 1092-2) may output audio signals to the outside of the wearable device (1000). The audio output module may be referred to as a speaker. In one embodiment, speakers (1092-1, 1092-2) may be positioned within a first temple (1004) and / or a second temple (1005) to be positioned adjacent to the ears of a user wearing the wearable device (1000). For example, the wearable device (1000) may include a second speaker (1092-2) positioned within the first temple (1004) and thus adjacent to the user's left ear, and a first speaker (1092-1) positioned within the second temple (1005) and thus adjacent to the user's right ear.
[0296] In one embodiment, the light-emitting module (not shown) may include at least one light-emitting element. The light-emitting module may emit light of a color corresponding to a specific state or emit light with an action corresponding to a specific state to visually provide a user with information regarding a specific state of the wearable device (1000). For example, when the wearable device (1000) requires charging, the wearable device (1000) may repeatedly emit red light at a designated time. In one embodiment, the light-emitting module may be disposed on the first rim (1001) and / or the second rim (1002).
[0297] Referring to FIG. 10B, according to one embodiment, a wearable device (1000) may include a printed circuit board (PCB) (1090). The PCB (1090) may be included in at least one of the first temple (1004) and the second temple (1005). The PCB (1090) may include an interposer positioned between at least two sub-PCBs. One or more hardware components included in the wearable device (1000) may be positioned on the PCB (1090). The wearable device (1000) may include a flexible PCB (FPCB) for interconnecting the hardware components.
[0298] According to one embodiment, a wearable device (1000) may include at least one of a gyro sensor, a gravity sensor, and / or an acceleration sensor for detecting a posture of the wearable device (1000) and / or a posture of a body part (e.g., a head) of a user wearing the wearable device (1000). Each of the gravity sensor and the acceleration sensor may measure gravitational acceleration and / or acceleration based on mutually perpendicular designated three-dimensional axes (e.g., an x-axis, a y-axis, and a z-axis). The gyro sensor may measure an angular velocity of each of the designated three-dimensional axes (e.g., an x-axis, a y-axis, and a z-axis). At least one of the gravity sensor, the acceleration sensor, and the gyro sensor may be referred to as an inertial measurement unit (IMU). According to one embodiment, the wearable device (1000) may identify a user's motion and / or gesture performed to execute or stop a specific function of the wearable device (1000) based on the IMU.
[0299] The wearable device (1000) can perform eye tracking and / or iris recognition using an eye tracking camera (1040-1) and a light source (e.g., an LED) that emits light toward the user's eyes. The method by which the wearable device (1000) performs eye tracking and / or iris recognition may be the same as the method by which the wearable device (200) performs eye tracking and / or iris recognition.
[0300] The wearable device (200, 1000) described above may include a plurality of light sources (280) configured to emit light toward the eyes of a user wearing the wearable device (200, 1000). According to one embodiment, the wearable device (200, 1000) may include a camera (310) configured to capture images of at least a portion of the eyes of the user wearing the wearable device (200, 1000). According to one embodiment, the wearable device (200, 1000) may include at least one processor (e.g., processor (120)) including a processing circuit. According to one embodiment, the wearable device (200, 1000) may include a memory (130) that stores instructions and includes one or more storage media. In one embodiment, the instructions, when individually or collectively executed by the at least one processor (120), may cause the wearable device (200, 1000) to perform eye tracking based on the images captured by the camera (310) while the plurality of light sources (280) are controlled to shine light toward the eye of the user. In one embodiment, the instructions, when individually or collectively executed by the at least one processor (120), may cause the wearable device (200, 1000) to perform iris recognition based on at least one image captured by the camera (310) while some of the plurality of light sources (280) are controlled to shine light toward the eye of the user and other some of the plurality of light sources (280) are deactivated.
[0301] In one embodiment, some of the plurality of light sources (280) used to perform the iris recognition may correspond to a subset of the same light sources used to perform the eye tracking.
[0302] In one embodiment, the number of light sources used to perform the iris recognition may be less than the number of light sources used to perform the eye tracking.
[0303] In one embodiment, two of the light sources used to perform the eye tracking may be selected to perform the iris recognition.
[0304] In one embodiment, only one of the light sources used to perform the eye tracking may be selected to perform the iris recognition.
[0305] In one embodiment, the instructions, when individually or collectively executed by the at least one processor (120), may cause the wearable device (200, 1000) to select another portion of the plurality of light sources (280) in response to a failure of the iris recognition based on the at least one image. In one embodiment, the instructions, when individually or collectively executed by the at least one processor (120), may cause the wearable device (200, 1000) to perform the iris recognition based on at least one other image captured by the camera (310) while the other portion of the plurality of light sources (280) is controlled to shine light toward the eye of the user and while the other portion of the plurality of light sources (280) is deactivated.
[0306] In one embodiment, the number of some of the plurality of light sources (280) may be different from the number of other some of the plurality of light sources (280).
[0307] In one embodiment, some of the plurality of light sources (280) may be configured to be closer to the camera (310) than other some of the plurality of light sources (280).
[0308] In one embodiment, said other part of said plurality of light sources (280) may be configured to include said part of said plurality of light sources (280).
[0309] In one embodiment, the wearable device (200, 1000) may further include a sensor. In one embodiment, the instructions, when individually or collectively executed by the at least one processor (120), may cause the wearable device (200, 1000) to determine, based on information obtained through the sensor, whether a lens is identified between the wearable device (200, 1000) and the eye of the user. In one embodiment, the instructions, when individually or collectively executed by the at least one processor (120), may cause the wearable device (200, 1000) to perform the iris recognition based on the at least one image captured by the camera (310) while some of the plurality of light sources (280) are controlled to shine toward the eye of the user based on the lens being identified, and while other some of the plurality of light sources (280) are deactivated. In one embodiment, the instructions, when individually or collectively executed by the at least one processor (120), may cause the wearable device (200, 1000) to perform the iris recognition based on the at least one image captured by the camera (310) while the same light sources used to perform the eye tracking are controlled to shine toward the eye of the user based on the lens not being identified.
[0310] In one embodiment, the instructions, when individually or collectively executed by the at least one processor (120), may cause the wearable device (200, 1000) to select, based on the lens being identified, some of the plurality of light sources (280) that correspond to the refractive index of the lens indicated by the information.
[0311] In one embodiment, the instructions, when individually or collectively executed by the at least one processor (120), may cause the wearable device (200, 1000) to perform iris recognition based on the images captured by the camera (310) for the eye tracking while the plurality of light sources (280) are controlled to shine light toward the eye of the user. In one embodiment, the instructions, when individually or collectively executed by the at least one processor (120), may cause the wearable device (200, 1000) to perform iris recognition based on the at least one image captured by the camera (310) while controlling some of the plurality of light sources (280) to illuminate toward the eye of the user in response to a failure of the iris recognition based on the images, and while the remaining some of the plurality of light sources (280) are deactivated.
[0312] In one embodiment, the camera (310) may include a first camera and a second camera. In one embodiment, the instructions, when individually or collectively executed by the at least one processor (120), may cause the wearable device (200, 1000) to perform eye tracking based on the images captured by the first camera while the plurality of light sources (280) are controlled to shine light toward the eye of the user. The instructions, when individually or collectively executed by the at least one processor (120), may cause the wearable device (200, 1000) to perform iris recognition based on the at least one image captured by the second camera while some of the plurality of light sources (280) are controlled to shine light toward the eye of the user and the remaining some of the plurality of light sources (280) are deactivated.
[0313] In one embodiment, the instructions, when individually or collectively executed by the at least one processor (120), may cause the wearable device (200, 1000) to deactivate the remaining portions of the plurality of light sources in response to determining to perform iris recognition while performing eye tracking.
[0314] In one embodiment, the instructions, when individually or collectively executed by the at least one processor (120), may cause the wearable device (200, 1000) to select two light sources based on contextual information.
[0315] In one embodiment, the instructions, when individually or collectively executed by the at least one processor (120), may cause the wearable device (200, 1000) to decide to continue performing eye tracking if simultaneous execution of eye tracking and iris recognition is identified as possible. In one embodiment, the instructions, when individually or collectively executed by the at least one processor (120), may cause the wearable device (200, 1000) to decide to stop performing eye tracking if simultaneous execution of eye tracking and iris recognition is identified as difficult.
[0316] A method as described above may be performed in a wearable device (200, 1000) including a plurality of light sources (280) configured to emit light toward an eye of a user wearing the wearable device (200, 1000), and a camera (310) configured to capture images of at least a portion of an eye of the user wearing the wearable device (200, 1000). In one embodiment, the method may include performing eye tracking based on the images captured by the camera (310) while the plurality of light sources (280) are controlled to emit light toward the eye of the user. In one embodiment, the method may include performing iris recognition based on at least one image captured by the camera (310) while some of the plurality of light sources (280) are controlled to emit light toward the user's eye and while other some of the plurality of light sources (280) are deactivated.
[0317] In one embodiment, the method may include, in response to a failure of the iris recognition based on the at least one image, selecting another portion of the plurality of light sources (280). In one embodiment, the method may include, while the other portion of the plurality of light sources (280) is controlled to emit light toward the eye of the user, and while the other portion of the plurality of light sources (280) is deactivated, performing the iris recognition based on at least one other image captured by the camera (310).
[0318] A non-transitory computer readable storage medium as described above may store one or more programs including instructions. In one embodiment, the instructions, when executed by a processor (120) of a wearable device (200, 1000), including a plurality of light sources (280) configured to emit light toward an eye of a user wearing the wearable device (200, 1000), and a camera (310) configured to capture images of at least a portion of an eye of the user wearing the wearable device (200, 1000), may cause the wearable device (200, 1000) to perform eye tracking based on the images captured by the camera (310) while the plurality of light sources (280) are controlled to emit light toward the eye of the user. In one embodiment, the instructions, when executed by the processor (120), may cause the wearable device (200, 1000) to perform iris recognition based on at least one image captured by the camera (310) while some of the plurality of light sources (280) are controlled to emit light toward the user's eyes and while other some of the plurality of light sources (280) are deactivated.
[0319] In one embodiment, the instructions, when executed by the processor (120), may cause the wearable device (200, 1000) to select another portion of the plurality of light sources in response to a failure in iris recognition based on at least one image. In one embodiment, the instructions, when executed by the processor (120), may cause the wearable device (200, 1000) to perform iris recognition based on at least one other image captured by the camera while another portion of the plurality of light sources is controlled to shine light toward the user's eye and while the remaining portions of the plurality of light sources are deactivated.
[0320] Electronic devices according to the various embodiments disclosed in this document may take various forms. Electronic devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, wearable devices, or home appliances. Electronic devices according to the embodiments of this document are not limited to the aforementioned devices.
[0321] The various embodiments of the document and the terminology used therein are not intended to limit the technical features described in the document to specific embodiments, but should be understood to include various modifications, equivalents, or substitutes of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the items, unless the context clearly indicates otherwise. In this document, phrases such as "A or B," "at least one of A and B," "at least one of A or B," "A, B, or C," "at least one of A, B, and C," and "at least one of A, B, or C" can each include any one of the items listed together in the corresponding phrase, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used simply to distinguish the corresponding component from other corresponding components, and do not limit the corresponding components in any other respect (e.g., importance or order). When a component (e.g., a first component) is referred to as being "coupled" or "connected" to another component (e.g., a second component), with or without the terms "functionally" or "communicatively," it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.
[0322] The term "module" used in various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integral component, or a minimum unit or part of such a component that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).
[0323] Various embodiments of the present document may be implemented as software (e.g., a program (140)) including one or more instructions stored in a storage medium (e.g., an internal memory (136) or an external memory (138)) readable by a machine (e.g., an electronic device (101)). For example, a processor (e.g., a processor (120)) of the machine (e.g., an electronic device (101)) may call at least one instruction among the one or more instructions stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' simply means that the storage medium is a tangible device and does not contain signals (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently or temporarily on the storage medium.
[0324] According to one embodiment, the method according to various embodiments disclosed in this document may be provided as included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., a compact disc read-only memory (CD-ROM)), or may be distributed online (e.g., by download or upload) through an application store (e.g., Play Store™) or directly between two user devices (e.g., smart phones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.
[0325] According to various embodiments, each component (e.g., a module or a program) of the above-described components may include one or more entities, and some of the entities may be separated and placed in other components. According to various embodiments, one or more components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., a module or a program) may be integrated into a single component. In such a case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. According to various embodiments, the operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.
[0326] While the disclosure has been illustrated and described with reference to various embodiments, it will be understood by those skilled in the art that various changes in form and detail may be made therein without departing from the spirit and scope of the disclosure as defined by the appended claims and their equivalents.
Claims
1. In a wearable device (200, 1000), A plurality of light sources (280) configured to emit light toward the eyes of a user wearing the wearable device (200, 1000); A camera (310) configured to capture images of at least a portion of the eye of the user wearing the wearable device (200, 1000); At least one processor (120) comprising a processing circuit, and A wearable device (200, 1000) comprises a memory (130) storing instructions and including one or more storage media, wherein the instructions, when individually or collectively executed by the at least one processor (120), While the plurality of light sources (280) are controlled to emit light toward the user's eyes, eye tracking is performed based on the images captured by the camera (310). Some of the plurality of light sources (280) are controlled to emit light toward the user's eyes, and while the remaining some of the plurality of light sources (280) are deactivated, iris recognition is performed based on at least one image captured by the camera (310). Wearable devices.
2. In claim 1, Among the plurality of light sources (280) used to perform the iris recognition, some correspond to a subset of the same light sources used to perform the eye tracking. Wearable devices.
3. In claim 1, The number of multiple light sources used to perform the above iris recognition is less than the number of light sources used to perform the above eye tracking. Wearable devices.
4. In claim 1, Among the plurality of light sources used to perform the eye tracking, two light sources are selected to perform the iris recognition. Wearable devices.
5. In claim 1, wherein only one of the plurality of light sources used to perform the eye tracking is selected to perform the iris recognition; Wearable devices.
6. In any one of claims 1 to 5, The above instructions, when individually or collectively executed by the at least one processor (120), cause the wearable device (200, 1000) to: In response to a failure of said iris recognition based on said at least one image, another portion of said plurality of light sources (280) is selected, The other part of the plurality of light sources (280) is controlled to emit light toward the user's eyes, and while the other part of the plurality of light sources (280) is deactivated, the iris recognition is performed based on at least one other image captured by the camera (310). Wearable devices.
7. In any one of claims 1 to 6, The number of some of the above plurality of light sources (280) is different from the number of other some of the above plurality of light sources (280). Wearable devices.
8. In any one of claims 1 to 7, Some of the above plurality of light sources (280) are configured to be closer to the camera (310) than other some of the above plurality of light sources (280). Wearable devices.
9. In any one of claims 1 to 8, Among the plurality of light sources (280), the other part is configured to include the part of the plurality of light sources (280). Wearable devices.
10. In any one of claims 1 to 9, Contains a sensor, The above instructions, when individually or collectively executed by the at least one processor (120), cause the wearable device (200, 1000) to: Based on the information acquired through the above sensor, it is determined whether a lens is identified between the wearable device (200, 1000) and the eye of the user, Based on the identification of the lens, some of the plurality of light sources (280) are controlled to emit light toward the user's eye, and while the remaining some of the plurality of light sources (280) are deactivated, the iris recognition is performed based on the at least one image captured by the camera (310). Based on the fact that the lens is not identified, the same light sources used to perform the eye tracking are controlled to shine toward the eye of the user, while causing the iris recognition to be performed based on at least one image captured by the camera (310). Wearable devices.
11. In any one of claims 1 to 10, The above instructions, when individually or collectively executed by the at least one processor (120), cause the wearable device (200, 1000) to: Based on the identification of the above lens, causing the selection of some of the plurality of light sources (280) corresponding to the refractive index of the lens indicated by the information. Wearable devices.
12. In any one of claims 1 to 11, The above instructions, when individually or collectively executed by the at least one processor (120), cause the wearable device (200, 1000) to: While the plurality of light sources (280) are controlled to shine light toward the user's eyes, the iris recognition is performed based on the images captured by the camera (310) to perform the eye tracking, In response to a failure of said iris recognition based on said images, said some of said plurality of light sources (280) are controlled to shine toward said eye of said user, and while said remaining some of said plurality of light sources (280) are deactivated, causing said iris recognition to be performed based on said at least one image captured by said camera (310). Wearable devices.
13. In any one of claims 1 to 12, The above camera (310) includes a first camera and a second camera, The above instructions, when individually or collectively executed by the at least one processor (120), cause the wearable device (200, 1000) to: Based on the images captured by the first camera, the eye tracking is performed, Causing said iris recognition to be performed based on said at least one image captured by said second camera. Wearable devices.
14. A method performed by a wearable device (200, 1000), comprising a plurality of light sources (280) configured to emit light toward the eyes of a user wearing the wearable device (200, 1000), and a camera (310) configured to capture images of at least a portion of the eyes of the user wearing the wearable device (200, 1000). An operation of performing eye tracking based on the images captured by the camera (310) while the plurality of light sources (280) are controlled to emit light toward the eyes of the user by the wearable device (200, 1000), and An operation of performing iris recognition based on at least one image captured by the camera (310) while some of the plurality of light sources (280) are controlled to emit light toward the user's eyes by the wearable device (200, 1000) and while the remaining some of the plurality of light sources (280) are deactivated. method.
15. In claim 14, Among the plurality of light sources (280) used to perform the iris recognition, some correspond to a subset of the same light sources used to perform the eye tracking. method.
Citation Information
Patent Citations
Iris recognition device and method for dynamic human eye tracking
CN106778713A
Wearable Band with Pulse Measuring and Preventing Accident
KR102176743B1
Camera that uses light from plural light sources disposed on a device
US20160283790A1
Eye tracker illumination through a waveguide
US20220300073A1
Wearable electronic device adjusting transmittance of visor and brightness of display
US20230122744A1
