Electronic device including infrared camera

Abstract

The wearable electronic device according to an embodiment may include: at least one infrared light source; a lens module including at least one lens; a display disposed to face a first surface of the lens module and configured to output visual information toward the lens module; an infrared camera for obtaining user information by infrared light output from the at least one infrared light source; and an infrared pass filter allowing infrared light of a designated band, incident through a second surface of the lens module, to pass therethrough, the infrared pass filter being disposed on a third surface of the lens module.

Description

Electronic device containing an infrared camera

[0001] Various embodiments of the present disclosure relate to an infrared camera and an electronic device including the same.

[0002] With the advancement of electronics, information, and communication technologies, a variety of functions are being integrated into a single electronic device. For example, electronic devices (e.g., smartphones) incorporate functions such as audio playback, imaging, and electronic notebooks in addition to communication functions. Furthermore, smartphones can be equipped with even more diverse functions through the installation of additional applications. In addition to executing pre-installed applications or stored functions, electronic devices can also access servers or other electronic devices via wired or wireless means to receive a variety of information in real time.

[0003] As the use of electronic devices becomes more commonplace, user demands for portability and usability may increase. In response to this demand, electronic devices that can be worn on the body (hereinafter referred to as "wearable electronic devices"), similar to wristwatches or glasses, have become commercially available. Examples of wearable electronic devices include head-mounted wearable devices (HMDs), smart glasses, smart watches (or bands), contact lens-type devices, ring-type devices, and clothing / shoe / glove-type devices. These body-worn electronic devices are easy to carry and can improve user accessibility. A "head-mounted wearable device" is a device worn on the user's head or face that projects images onto the user's retina, allowing the user to view virtual images in three-dimensional space. For example, head-mounted wearable devices can be divided into see-through types that provide augmented reality (AR) and see-closed types that provide virtual reality (VR). A see-through type head-mounted wearable device can be implemented in the form of glasses, for example, and can provide information such as buildings and objects in the user's field of vision in the form of images or text to the user. A see-closed type head-mounted wearable device can output independent images to both eyes of the user, and can provide an excellent sense of immersion by outputting content (such as games, movies, streaming, and broadcasting) provided from a mobile communication terminal or an external input in the form of images or audio to the user or individual wearing the device.Additionally, head-mounted wearable devices may be used to provide mixed reality (MR) or extended reality (XR), which are a combination of augmented reality (AR) and virtual reality (VR).

[0004] The above information may be provided as background art to aid in understanding the present disclosure. No claim or determination is made as to whether any of the above is applicable as prior art in connection with the present disclosure.

[0005] A wearable electronic device according to one embodiment may include: at least one infrared light source; a lens module including at least one lens; a display arranged to face a first surface of the lens module and configured to output visual information toward the lens module; an infrared camera for acquiring user information by infrared light output from the at least one infrared light source; and an infrared pass filter arranged on a third surface of the lens module, the infrared light passing through a designated band of infrared light incident through a second surface of the lens module.

[0006] According to one embodiment, a wearable electronic device may include: at least one infrared light source; a first lens module and a second lens module each including at least one lens; a first display arranged to face a first surface of the first lens module and configured to output visual information toward the first lens module; a second display arranged to face a first surface of the second lens module and configured to output visual information toward the first lens module; a single infrared camera for acquiring user information by infrared light output from the at least one infrared light source; and a first infrared pass filter arranged on a third surface of the first lens module, which passes infrared light of a specified band incident through the second surface of the first lens module, and a second infrared pass filter arranged on a third surface of the second lens module, which passes infrared light of a specified band incident through the second surface of the second lens module. The single infrared camera may be positioned between the third face of the first lens module and the third face of the second lens module.

[0007] The above-described aspects or other aspects, configurations and / or advantages of one embodiment of the present disclosure may be further clarified by the following detailed description taken in conjunction with the accompanying drawings.

[0008] FIG. 1 is a block diagram illustrating an electronic device within a network environment according to one embodiment of the present disclosure.

[0009] FIG. 2 is a diagram illustrating a wearable electronic device according to one embodiment of the present disclosure.

[0010] FIG. 3 is a perspective view showing the front of a wearable electronic device according to one embodiment of the present disclosure.

[0011] FIG. 4 is a perspective view showing the rear side of a wearable electronic device according to one embodiment of the present disclosure.

[0012] FIG. 5 is a front view showing the rear side of a wearable electronic device according to one embodiment of the present disclosure.

[0013] Figure 6a is a drawing showing the structure of the human eye.

[0014] Figure 6b shows a simplified drawing of the structure of the human eye of Figure 6a.

[0015] FIG. 7A is a diagram illustrating a method for tracking a person's gaze using a simplified eye model according to one embodiment of the present disclosure.

[0016] Figure 7b is a drawing showing information about the location of the glint on a 2D drawing.

[0017] FIG. 8 is a perspective view of a wearable electronic device according to one embodiment of the present disclosure.

[0018] FIG. 9 is a diagram illustrating internal components of a wearable electronic device according to one embodiment of the present disclosure.

[0019] FIG. 10 is a drawing of an infrared pass filter according to one embodiment of the present disclosure.

[0020] FIG. 11 is a graph showing optical characteristics according to a frequency band of an infrared band pass film according to an embodiment of the present disclosure.

[0021] FIG. 12 is a diagram illustrating the principle of wave destructive interference of an anti-reflection film according to one embodiment of the present disclosure.

[0022] FIG. 13A is a drawing of a lens module according to one embodiment of the present disclosure.

[0023] FIG. 13b is a drawing of a lens module according to one embodiment of the present disclosure.

[0024] FIG. 13c is a drawing of a lens module according to one embodiment of the present disclosure.

[0025] FIG. 14 is a diagram illustrating internal components of a wearable electronic device according to one embodiment of the present disclosure.

[0026] FIG. 15 is a diagram illustrating internal components of a wearable electronic device according to one embodiment of the present disclosure.

[0027] FIG. 16 is a diagram illustrating internal components of a wearable electronic device according to one embodiment of the present disclosure.

[0028] FIG. 17 is a drawing showing a display according to one embodiment of the present disclosure.

[0029] FIG. 18 is a drawing showing a display according to one embodiment of the present disclosure.

[0030] FIG. 19 is a schematic diagram illustrating a gaze tracking method using a wearable electronic device according to one embodiment of the present disclosure.

[0031] FIG. 20 is a schematic diagram illustrating a gaze tracking method using a wearable electronic device according to one embodiment of the present disclosure.

[0032] Throughout the attached drawings, similar reference numbers may be assigned to similar parts, components and / or structures.

[0033] In a wearable electronic device, two infrared cameras can be used to capture images of the left and right eyes, respectively, as a camera for tracking gaze or recognizing irises. However, if two infrared cameras are installed, the component mounting space inside the wearable electronic device is reduced, which may limit the external design. In addition, if two infrared cameras are used in the past, the images are captured from the sides of the user's left and right eyes, which may lower the accuracy of gaze tracking and iris recognition, that is, lower performance of gaze tracking and iris recognition. In addition, since two infrared cameras must be operated and the images acquired from them must be individually processed, there may be a problem of increased power consumption in acquiring images.

[0034] The present disclosure can solve the above-described problem by providing a wearable electronic device having an infrared camera applied as a camera for tracking gaze or a camera for recognizing irises.

[0035] The following description of the accompanying drawings may provide an understanding of various exemplary implementations of the present disclosure, including the claims and their equivalents. While the exemplary embodiments disclosed in the following description include numerous specific details to aid understanding, they are to be considered as one example of various exemplary embodiments. Accordingly, those skilled in the art will appreciate that various modifications and variations of the various implementations described herein may be made without departing from the scope and spirit of the disclosure. Furthermore, descriptions of well-known functions and configurations may be omitted for clarity and conciseness.

[0036] The terms and words used in the following description and claims are not limited to their reference meanings and can be used to clearly and consistently describe one embodiment of the present disclosure. Therefore, it will be apparent to those skilled in the art that the following description of various implementations of the disclosure is provided for illustrative purposes, not for the purpose of limiting the scope of the disclosure and its equivalents.

[0037] Unless the context clearly dictates otherwise, the singular forms of "a," "an," and "the" should be understood to include plural meanings. Thus, for example, "a component surface" could be understood to include one or more of the surfaces of the component.

[0038] FIG. 1 is a block diagram of an electronic device (101) within a network environment (100) according to one embodiment of the present disclosure. Referring to FIG. 1 , in the network environment (100), the electronic device (101) may communicate with the electronic device (102) via a first network (198) (e.g., a short-range wireless communication network), or may communicate with at least one of the electronic device (104) or the server (108) via a second network (199) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (101) may communicate with the electronic device (104) via the server (108). According to one embodiment, the electronic device (101) may include a processor (120), a memory (130), an input module (150), an audio output module (155), a display module (160), an audio module (170), a sensor module (176), an interface (177), a connection terminal (178), a haptic module (179), a camera module (180), a power management module (188), a battery (189), a communication module (190), a subscriber identification module (196), or an antenna module (197). In one embodiment, the electronic device (101) may have at least one of these components (e.g., the connection terminal (178)) omitted, or one or more other components added. In one embodiment, some of these components (e.g., the sensor module (176), the camera module (180), or the antenna module (197)) may be integrated into one component (e.g., the display module (160)).

[0039] The processor (120) may, for example, execute software (e.g., a program (140)) to control at least one other component (e.g., a hardware or software component) of the electronic device (101) connected to the processor (120) and perform various data processing or operations. According to one embodiment, as at least a part of the data processing or operations, the processor (120) may store commands or data received from other components (e.g., a sensor module (176) or a communication module (190)) in a volatile memory (132), process the commands or data stored in the volatile memory (132), and store result data in a non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., a central processing unit or an application processor), or an auxiliary processor (123) (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor) that can operate independently or together with the main processor (121). For example, when the electronic device (101) includes the main processor (121) and the auxiliary processor (123), the auxiliary processor (123) may be configured to use less power than the main processor (121) or to be specialized for a given function. The auxiliary processor (123) may be implemented separately from the main processor (121) or as a part thereof.

[0040] The auxiliary processor (123) may control at least a portion of functions or states associated with at least one component (e.g., a display module (160), a sensor module (176), or a communication module (190)) of the electronic device (101), for example, on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. In one embodiment, the auxiliary processor (123) (e.g., an image signal processor or a communication processor) may be implemented as a part of another functionally related component (e.g., a camera module (180) or a communication module (190)). In one embodiment, the auxiliary processor (123) (e.g., a neural network processing unit) may include a hardware structure specialized for processing artificial intelligence models. The artificial intelligence models may be generated through machine learning. This learning can be performed, for example, on the electronic device (101) itself where the artificial intelligence model is executed, or can be performed through a separate server (e.g., server (108)). The learning algorithm can include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model can include multiple artificial neural network layers.The artificial neural network may be one of a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to, or alternatively to, a hardware structure, an artificial intelligence model may include a software structure.

[0041] The memory (130) can store various data used by at least one component (e.g., processor (120) or sensor module (176)) of the electronic device (101). The data can include, for example, software (e.g., program (140)) and input data or output data for commands related thereto. The memory (130) can include volatile memory (132) or non-volatile memory (134).

[0042] The program (140) may be stored as software in the memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0043] The input module (150) can receive commands or data to be used in a component of the electronic device (101) (e.g., a processor (120)) from an external source (e.g., a user) of the electronic device (101). The input module (150) can include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0044] The audio output module (155) can output audio signals to the outside of the electronic device (101). The audio output module (155) can include, for example, a speaker or a receiver. The speaker can be used for general purposes, such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. In one embodiment, the receiver can be implemented separately from the speaker or as part of the speaker.

[0045] The display module (160) can visually provide information to an external party (e.g., a user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling the device. According to one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of a force generated by the touch.

[0046] The audio module (170) can convert sound into an electrical signal, or vice versa, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150), output sound through the sound output module (155), or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphone) directly or wirelessly connected to the electronic device (101).

[0047] The sensor module (176) can detect the operating status (e.g., power or temperature) of the electronic device (101) or the external environmental status (e.g., user status) and generate an electrical signal or data value corresponding to the detected status. According to one embodiment, the sensor module (176) can include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0048] The interface (177) may support one or more designated protocols that may be used to directly or wirelessly connect the electronic device (101) to an external electronic device (e.g., the electronic device (102)). In one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0049] The connection terminal (178) may include a connector through which the electronic device (101) may be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0050] The haptic module (179) can convert electrical signals into mechanical stimuli (e.g., vibration or movement) or electrical stimuli that a user can perceive through tactile or kinesthetic sensations. According to one embodiment, the haptic module (179) can include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

[0051] The camera module (180) can capture still images and videos. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0052] The power management module (188) can manage power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented as, for example, at least a part of a power management integrated circuit (PMIC).

[0053] A battery (189) may power at least one component of the electronic device (101). In one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0054] The communication module (190) may support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may operate independently from the processor (120) (e.g., application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (194) (e.g., a local area network (LAN) communication module, or a power line communication module). Among these communication modules, the corresponding communication module can communicate with an external electronic device via a first network (198) (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network (199) (e.g., a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules can be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can verify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) by using subscriber information (e.g., an international mobile subscriber identity (IMSI)) stored in the subscriber identification module (196).

[0055] The wireless communication module (192) can support 5G networks and next-generation communication technologies following the 4G network, such as NR access technology (new radio access technology). The NR access technology can support high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimization of terminal power and connection of multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low-latency communications)). The wireless communication module (192) can support, for example, a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate. The wireless communication module (192) can support various technologies for securing performance in a high-frequency band, such as beamforming, massive multiple-input and multiple-output (MIMO), full dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), an external electronic device (e.g., the electronic device (104)), or a network system (e.g., the second network (199)). According to one embodiment, the wireless communication module (192) can support a peak data rate (e.g., 20 Gbps or more) for eMBB realization, a loss coverage (e.g., 164 dB or less) for mMTC realization, or a U-plane latency (e.g., 0.5 ms or less for downlink (DL) and uplink (UL), or 1 ms or less for round trip) for URLLC realization.

[0056] The antenna module (197) can transmit or receive signals or power to or from an external device (e.g., an external electronic device). In one embodiment, the antenna module may include an antenna including a radiator formed of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). In one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as the first network (198) or the second network (199), may be selected from the plurality of antennas, for example, by the communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device via the at least one selected antenna. In one embodiment, in addition to the radiator, another component (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as a part of the antenna module (197).

[0057] In one embodiment, the antenna module (197) may form a mmWave antenna module. In one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent a first side (e.g., a bottom side) of the printed circuit board and capable of supporting a designated high-frequency band (e.g., a mmWave band), and a plurality of antennas (e.g., an array antenna) disposed on or adjacent a second side (e.g., a top side or a side) of the printed circuit board and capable of transmitting or receiving signals in the designated high-frequency band.

[0058] At least some of the above components can be interconnected and exchange signals (e.g., commands or data) with each other via a communication method between peripheral devices (e.g., a bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)).

[0059] According to one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) via a server (108) connected to a second network (199). Each of the external electronic devices (102 or 104) may be the same or a different type of device as the electronic device (101). According to one embodiment, all or part of the operations executed in the electronic device (101) may be executed in one or more of the external electronic devices (102, 104, or 108). For example, when the electronic device (101) is to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device (101) may, instead of or in addition to executing the function or service itself, request one or more external electronic devices to perform the function or at least part of the service. One or more external electronic devices that receive the request may execute at least a portion of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may process the result as is or additionally and provide it as at least a portion of a response to the request. For this purpose, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used, for example. The electronic device (101) may provide an ultra-low latency service by using distributed computing or mobile edge computing, for example. In one embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server utilizing machine learning and / or a neural network. According to one embodiment, the external electronic device (104) or the server (108) may be included in the second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0060] FIG. 2 is a diagram illustrating a wearable electronic device according to one embodiment of the present disclosure.

[0061] In describing one embodiment of the present disclosure, some numerical values ​​and the like may be presented, but it should be noted that such numerical values ​​do not limit one embodiment of the present disclosure unless stated in the claims.

[0062] Referring to FIG. 2, a wearable electronic device (200) (e.g., electronic device (101) of FIG. 1) is an electronic device that can be worn on a user's head or face, and the user can visually recognize surrounding objects or environments even while wearing the wearable electronic device (200). The wearable electronic device (200) uses a camera module to acquire and / or recognize visual images of objects or environments viewed by the user or in the direction in which the wearable electronic device (200) is directed, and can receive information about the objects or environments from an external electronic device via a network. The wearable electronic device (200) can provide the user with information about the objects or environments received in an acoustic or visual form. For example, the wearable electronic device (200) can provide the user with information about the objects or environments received in a visual form by using a display member such as a display module. By visualizing information about objects or the environment and combining it with actual images (or videos) of the user's surroundings, the wearable electronic device (200) can implement augmented reality (AR), virtual reality (VR), mixed reality (MR), and / or extended reality (XR). The display member can provide the user with information about objects or the environment around him / her by outputting a screen in which an augmented reality object is added to an actual image (or video) of the user's surroundings.

[0063] According to one embodiment, all or part of the operations executed by the electronic device (101) or the wearable electronic device (200) may be executed by one or more of the external electronic devices (102, 104, or 108). For example, when the electronic device (101) or the wearable electronic device (200) needs to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device (101) or the wearable electronic device (200) may, instead of executing the function or service by itself or in addition, request one or more external electronic devices to execute the function or at least a part of the service. The one or more external electronic devices that receive the request may execute at least a part of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device (101) or the wearable electronic device (200). The electronic device (101) or the wearable electronic device (200) may provide the result as is or additionally processed as at least a part of the response to the request. For example, the external electronic device (102) may render content data executed in an application and then transmit it to the electronic device (101) or the wearable electronic device (200), and the electronic device (101) or the wearable electronic device (200) that has received the data may output the content data to a display module. When the electronic device (101) or the wearable electronic device (200) detects user movement through a sensor(s) such as an inertial measurement unit sensor, the processor (e.g., the processor (120) of FIG. 1) of the electronic device (101) or the wearable electronic device (200) may correct the rendering data received from the external electronic device (102) based on the movement information and output the corrected data to the display module.Or, when a user movement is detected through a sensor(s), a processor (e.g., processor (120) of FIG. 1) of the electronic device (101) or wearable electronic device (200) may transmit the movement information to an external electronic device (102) and request rendering so that screen data is updated accordingly. According to various embodiments, the external electronic device (102) may be a device of various forms, such as a case device capable of storing and charging the electronic device (101).

[0064] It should be noted that the detailed description below may refer to various things such as "a state or position in which an electronic device or a designated component of an electronic device faces the user's face," and this is based on the assumption that the user is wearing the wearable electronic device (200).

[0065] According to one embodiment, a wearable electronic device (200) may include at least one display member and a wearing member (e.g., a temple). Depending on the structure of the display member, the wearable electronic device (200) may further include a structure (e.g., a lens frame) for mounting or supporting the display member. The display members may be provided as a pair including a first display member and a second display member, and may be arranged to correspond to the user's right eye and left eye, respectively, when the wearable electronic device (200) is worn on the user's body. In one embodiment, the wearable electronic device (200) may also include a housing form (e.g., a goggle form) including one display member corresponding to the right eye and the left eye.

[0066] According to one embodiment, the display member is a configuration provided to provide visual information to a user, and may include, for example, a display (D), a plurality of lenses (L1, L2, L3, L4) (e.g., a lens assembly) and / or at least one sensor. Here, the lens assembly and the display (D) may each be formed transparently or translucently. However, the display member is not limited thereto. In one embodiment, the display member may include a window member, and the window member may be a translucent glass or a member whose light transmittance can be adjusted as a tinting density is adjusted. In one embodiment, the display member may include a lens including a waveguide or a reflective lens, and an image output from an optical output device (e.g., a projector or the display (D)) is formed on each lens to provide visual information to the user. For example, the display member may include a waveguide (e.g., a light waveguide) in at least a portion of each lens, and may mean a display that transmits an image (or light) output from an optical output device, such as a display (D), to the user's eyes through the waveguide included in the display member, and at the same time transmits the real world to the user's eyes through that area in a see-through manner. In one embodiment, the waveguide may be understood as a part of a lens assembly. The lens assembly may be a configuration including a plurality of lenses (e.g., L1, L2, L3, L4), and may be arranged in a state aligned with an optical axis (e.g., axis I of FIG. 2) in a space within the wearable electronic device (200).

[0067] FIG. 3 is a perspective view illustrating a front side of a wearable electronic device according to an embodiment of the present disclosure. FIG. 4 is a perspective view illustrating a rear side of a wearable electronic device according to an embodiment of the present disclosure.

[0068] Referring to FIGS. 3 and 4, in one embodiment, camera modules (211, 212, 213, 214, 215, 216) and / or depth sensors (217) may be arranged on a first surface (210) of an electronic device (200) (e.g., a housing) to obtain information related to the surrounding environment of the wearable electronic device (200).

[0069] In one embodiment, the camera modules (211, 212) can acquire images related to the environment surrounding the wearable electronic device.

[0070] In one embodiment, the camera modules (213, 214, 215, 216) can acquire images while the wearable electronic device is worn by a user. The camera modules (213, 214, 215, 216) can be used for hand detection, tracking, and recognition of user gestures (e.g., hand movements). The camera modules (213, 214, 215, 216) can be used for 3DoF (degrees of freedom), 6DoF head tracking, position (spatial, environmental) recognition, and / or movement recognition. In one embodiment, the camera modules (211, 212) can also be used for hand detection and tracking or recognition or detection of user gestures.

[0071] In one embodiment, the depth sensor (217) may be configured to transmit a signal and receive a signal reflected from a subject, and may be used for purposes such as time of flight (TOF) to determine the distance to an object. Instead of or in addition to the depth sensor (217), camera modules (213, 214, 215, 216) may determine the distance to an object.

[0072] According to one embodiment, a camera module (225, 226) and / or a lens module (221, 222) may also be disposed on the second side (220) of the housing. The camera module (225, 226) disposed on the second side (220) of the housing may be used to recognize the user's face, or may recognize (e.g., iris recognition) and / or track (e.g., gaze tracking) both eyes of the user.

[0073] In one embodiment, the lens modules (221, 222) may be disposed on the second side (220) of the wearable electronic device (200). In one embodiment, the lens modules (221, 222) may be at least partially similar to, or substantially identical to, the display (D) and / or the lenses (L1, L2, L3, L4) of FIG. 2. In one embodiment, the wearable electronic device (200) may not include the camera modules (215, 216) among the plurality of camera modules (213, 214, 215, 216). Although not illustrated in FIGS. 3 and 4, the wearable electronic device (200) may further include at least one of the configurations illustrated in FIGS. 1 and / or 2.

[0074] As described above, according to one embodiment, the wearable electronic device (200) may have a form factor for being worn on a user's head. The wearable electronic device (200) may further include a strap and / or a wearing member for being secured to a body part of the user. The wearable electronic device (200) may provide a user experience based on augmented reality, virtual reality, and / or mixed reality while being worn on the user's head.

[0075] FIG. 5 is a front view showing the rear side of a wearable electronic device according to one embodiment of the present disclosure.

[0076] Referring to FIG. 5, the wearable electronic device (200) may include a first lens module (221) and a second lens module (222) corresponding to the user's two eyes. The first lens module (221) and the second lens module (222) may have rims formed thereon. The rim may refer to, for example, a ring-shaped portion formed at an end (or periphery) of the first lens module (221) and the second lens module (222). According to one embodiment, when a user wears the wearable electronic device (200), the rims may surround the user's eyes.

[0077] The wearable electronic device (200) may include at least one infrared light source (227, 228). The at least one infrared light source may be disposed around the first lens module (221) and / or the second lens module (222). The type of infrared light emitted from the at least one infrared light source (e.g., point light, linear light, area light) may vary. According to one embodiment, the infrared light sources (227, 228) may be disposed on the rims of the wearable electronic device (200). For example, the infrared light sources may be disposed on the first rim (223) and the second rim (224) of the wearable electronic device (200), respectively. In another example, a plurality of infrared light sources may be radially disposed on the first rim (223) and the second rim (224), respectively. As illustrated in FIG. 5, the wearable electronic device (200) may include a plurality of infrared light sources (227, 228) arranged in a first rim (223) and a second rim (224) surrounding the user's eyes to provide light toward the user's eyes. For example, the plurality of infrared light sources may be configured to provide light toward the eyes at predetermined time intervals. In addition, for example, the plurality of infrared light sources may project light toward the eyes in a predetermined pattern.

[0078] The camera modules (225, 226) can track the user's gaze by identifying the cornea area and the pupil area of ​​the user's eye based on the change in the amount of light reflected from the user's eye area, or recognize the user's iris by identifying the iris area of ​​the user's eye. According to the embodiment illustrated in FIG. 5, the camera modules (225, 226) are arranged in a position adjacent to the rims (223, 224), but are mainly arranged on the outer side of the rims (223, 224), and can capture images from the outside to the inside based on a virtual center point between the user's left and right eyes. At this time, the data for tracking the user's gaze and the data for recognizing the user's iris can be based on acquired data that is reflected into the user's eyes and is incident on the camera modules (225, 226) located outside the rims (223, 224) based on the virtual center point between the user's left and right eyes.

[0079] Fig. 6a is a drawing showing the structure of the human eye. Fig. 6b is a simplified drawing showing the structure of the human eye of Fig. 6a.

[0080] Referring to Fig. 6a, in the structure of the eye, the fovea is the part where light is focused, passing vertically through the center of the cornea and the lens, and the direction of gaze can mean the visual axis passing from the fovea to the center of the lens. When the line passing from the center of rotation of the eye to the pupil is called the optical axis, the angle between the visual axis, which is the direction of gaze, and the optical axis is called the kappa angle, and the visual axis can be tilted about 5 degrees inward in the horizontal direction and about 1 degree downward in the vertical direction from the optical axis in the direction of the center of the pupil. Since it is difficult to obtain actual data on each element that constitutes the eye through image analysis, gaze tracking data can be obtained by using an eye model or through user-specific calibration.

[0081] As shown in Fig. 6b, a simplified representation of a human eye can be formed by two overlapping spheres having different curvatures. A portion of a second sphere with a smaller curvature protrudes from a first sphere corresponding to the eyeball (310). The protruding portion of the second sphere can be represented as the cornea (320), and a portion of the front of the second sphere can be represented as the pupil (330). In the simplified eyeball model, gaze tracking can mean obtaining a vector from the center of the eyeball (311) toward the center point of the pupil (331), i.e., a gaze vector (340). Tracking a user's gaze can be implemented by tracking and measuring the movement of the eyeball. However, the specific gaze tracking method may vary depending on the applied eyeball model and gaze tracking algorithm.

[0082] FIG. 7a is a diagram illustrating a method for tracking a person's gaze using a simplified eye model, according to one embodiment of the present disclosure. FIG. 7b is a diagram illustrating information regarding the location of a glint on a 2D drawing.

[0083] Referring to FIGS. 7A and 7B , a wearable electronic device (200) can obtain eye data including information related to the user's eyes by emitting infrared light toward the user's eyes and receiving infrared light reflected from the user's eyeballs using camera modules (225, 226). The eye data obtained in the gaze tracking operation may refer to 2D data in which information about the eyeballs in 3D space (3D space) is projected onto a 2D plane (2D image space), and depth information (e.g., the distance between the camera modules (225, 226) and the user's eyeballs) may be obtained by analyzing a plurality of pieces of received eyeball data. In addition, actual eyeball information in 3D space may be obtained by backprojecting the 2D eyeball data. The eyeball data may refer to data including information related to the user's eyes and gaze obtained from the camera modules (225, 226). The eye data may include at least one of information about the location of a glint feature point of the user's eye, the size and location of the eye, the size and location of the pupil, the size and location of the iris, and the size and location of the cornea.

[0084] The glint feature points (hereinafter referred to as 'glints') of the user's eye are located on the cornea as points (g1, g2) where light is brightly reflected, and the number of glints can be determined in various ways depending on the situation (e.g., the characteristics of the user's eye, the location of the light source, the presence of an external light source, etc.). In Fig. 7b, glints can be depicted in the form of dark dots around the pupil (330). Fig. 7b illustrates six glints displayed around the pupil (330) as an example. The pupil is an area that absorbs light incident on the user's eye, and thus has a characteristic of no or little reflected light. Therefore, the pupil data value can have a U-shape with a large value at the edge of the eye and a small value at the center of the eye. The pupil is an area that absorbs light incident on the eye, and since there is little reflected light, the pupil data value is small, whereas the edge of the eye has the white of the eye, so most of the light incident on the eye is reflected, and thus the pupil data value may appear high. The user's gaze can be tracked using 2D data for pupil data values ​​and the depth information.

[0085] However, in the case of tracking the gaze using camera modules (225, 226) positioned outside the rim (223, 224) based on the virtual center point between the user's left and right eyes, as in the embodiments illustrated in FIGS. 4 and 5, the eye data is acquired through information about the gaze obtained by shooting from the lower left toward the user's eyes (e.g., shooting with the camera module (225)) and / or shooting from the lower right (e.g., shooting with the camera module (226)), so the data may be somewhat inaccurate. In addition, for example, in an operation to recognize an iris, the user must gaze at the camera modules (225, 226) positioned outside the rim (223, 224) based on the virtual center point between the left and right eyes to recognize the iris, which may cause the user to feel uncomfortable.

[0086] Below, various embodiments for solving these problems will be described.

[0087] FIG. 8 is a perspective view of a wearable electronic device according to one embodiment of the present disclosure. FIG. 9 is a diagram illustrating internal components of a wearable electronic device according to one embodiment of the present disclosure.

[0088] Referring to FIGS. 8 and 9, a wearable electronic device (400) includes a front surface (410) forming an exterior and a back surface (420) facing a part of the body including the user's eyes and / or face, and may include a plurality of components (e.g., at least one infrared light source, a lens module, a display, a camera module, and / or various sensor modules) in the front surface (410), the back surface (420), and the space between the front surface (410) and the back surface (420). In the wearable electronic device (400), at least one of the above components may be omitted, or one or more other components may be added. For example, the wearable electronic device (400) may further include an input module (e.g., a microphone) or a battery. According to one embodiment of the present disclosure, the wearable electronic device (400) of FIGS. 8 and 9 (or the wearable electronic device (500) of FIG. 14, the wearable electronic device (600) of FIG. 15, and the wearable electronic device (700) of FIG. 16, which will be described later) may be substantially the same as the electronic device (101) of FIG. 1 and the wearable electronic device (200) of FIGS. 2 to 4, and may be implemented to be wearable on a user's body.

[0089] A wearable electronic device (400) may include a main body (401) in which components such as a lens module (e.g., lens modules 221 and 222 of FIGS. 3 and 4) and a display (e.g., display (D) of FIG. 2) are arranged, and a wearing member (402, 403) extending from the main body (401) or rotatably coupled to the main body. According to one embodiment, the wearing member (402, 403) may be rotatably coupled to the main body (401) by a hinge structure. The main body (401) is supported on the user's face, and the wearing member (402, 403) is supported on the side of the user's head (e.g., ear), so that the user can stably wear the wearable electronic device (400).

[0090] A wearable electronic device (400) according to one embodiment of the present disclosure may include a lens module (421, 422) including at least one lens. The wearable electronic device (400) may include a display (431, 432) arranged to face a first surface (421a, 422a) of the lens module (421, 422) and configured to output visual information toward the lens module. The wearable electronic device (400) may include an infrared camera (425) for acquiring user information by infrared rays output from at least one infrared light source. Here, the infrared light source outputting the infrared rays may be the infrared light sources (227, 228) according to the embodiment illustrated in FIG. 5, but is not necessarily limited thereto. The number of infrared light sources and the positions of the infrared light sources may be set in various ways depending on the embodiment. A wearable electronic device (400) may pass infrared rays of a designated band incident through a second face (421b, 422b) of a lens module (421, 422), and may include an infrared pass filter (441, 442) disposed on a third face (421c, 422c) of the lens module (421, 422).

[0091] The lens module (421, 422) may include two lens modules corresponding to the user's two eyes. According to one embodiment, the lens module (421, 422) may include a first lens module (421) corresponding to the user's left eye (EL; eye left) and a second lens module (422) corresponding to the user's right eye (ER; eye right). The first lens module (421) and the second lens module (422) may have substantially the same configuration, except that they correspond to the user's left eye (EL) and right eye (ER), respectively. The first lens module (421) may include a first surface (421a) facing the front surface (410) of the wearable electronic device (400), and a second surface (421b) facing in the opposite direction to the first surface (421a). Additionally, the first lens module (421) may include a third face (421c) and a fourth face (421d) facing in a different direction from the first face (421a) and the second face (421b). The second lens module (422) may include a first face (422a) facing the front side (410) of the wearable electronic device (400) and a second face (422b) facing in a direction opposite to the first face (422a). Additionally, the second lens module (422) may include a third face (422c) and a fourth face (422d) facing in a different direction from the first face (422a) and the second face (422b).

[0092] The third surfaces (421c, 422c) of the first lens module (421) and the second lens module (422) may be surfaces closer to the center of the wearable electronic device, and the fourth surfaces (421d, 422d) may be surfaces closer to the wearing portion (402, 403) of the wearable electronic device. The third surfaces (421c, 422c) and / or the fourth surfaces (421d, 422d) may be perpendicular to the first surfaces (421a, 422a) and / or the second surfaces (421b, 422b), respectively, but are not necessarily limited thereto. In FIG. 9, the shapes of the first lens module (421) and the second lens module (422) are illustrated as having an approximately rectangular cross-sectional shape when viewed from above, but this is not necessarily the case. According to one embodiment, the first lens module (421) and the second lens module (422) may have a barrel structure corresponding to the size and shape of the lens(es) included therein. For example, as illustrated in FIG. 9, when the first lens module (421) and the second lens module (422) are arranged so that lenses having relatively smaller diameters are positioned closer to the displays (431, 432) and the lenses having relatively larger diameters are positioned closer to the user's eyes (EL, ER), the first lens module (421) and the second lens module (422) may have a lens barrel structure having a narrower diameter closer to the displays (431, 432) and a wider diameter closer to the user's eyes (EL, ER), respectively. According to one embodiment, the third surfaces (421c, 422c) and / or the fourth surfaces (421d, 422d) may be seamlessly connected to each other. Accordingly, the third side (421c, 422c) and / or the fourth side (421d, 422d) are depicted as facing different directions when the wearable electronic device is viewed from above as illustrated in FIG. 9, but may also be interpreted as parts facing different directions belonging to a single circle when the wearable electronic device is viewed from the front as illustrated in FIG. 5, for example.

[0093] As described above, the optical characteristics of the first lens module (421), including the number, arrangement, and shape of the lenses included in the first lens module (421), may be the same as the optical characteristics of the second lens module (422), including the number, arrangement, and shape of the lenses included in the second lens module (422). Hereinafter, in describing the lens modules (421, 422), the first lens module (421) may be used as an example, and the description of the first lens module (421) may be applied to the second lens module (422). In addition, the description of the second lens module (422) may be omitted for convenience.

[0094] The displays (431, 432) may include a first display (431) corresponding to the first lens module (421) and a second display (432) corresponding to the second lens module (422). The first display (431) and the second display (432) may be arranged to face the first surfaces (421a, 422a) of the first lens module (421) and the second lens module (422), respectively, and may output visual information toward the first lens module (421) and the second lens module (422). The display (431, 432) may include at least one of a liquid crystal display (LCD), a digital mirror device (DMD), a liquid crystal on silicon (LCoS), a light emitting diode (LED) on silicon (LEDoS), an organic light emitting diode (OLED), or a micro light emitting diode (micro LED).

[0095] An infrared camera (425) may be provided to track a user's gaze or recognize a user's iris. The infrared camera (425) may obtain information about a glint image or an iris image in which infrared light output from at least one infrared light source shines on the eye. In the embodiments of FIGS. 3 and 4 , two cameras were provided as an example to perform this function. However, in the embodiments of FIGS. 8 and 9 , only one camera may be provided between the first lens module (421) and the second lens module (422). Accordingly, the infrared camera (425) may be referred to as a 'single infrared camera' or a 'single gaze tracking camera'. However, the scope of the present disclosure is not limited to the embodiments illustrated in FIGS. 8 and 9 , and the number of infrared cameras (425) of the present disclosure may be provided in multiple numbers depending on the embodiment. In addition, the infrared camera (425) may be positioned at various locations between the first lens module (421) and the second lens module (422) so long as the infrared light that has passed through the first lens module (421) and the second lens module (422) can be incident thereon. For example, referring to FIGS. 8 and 9 together, the infrared camera (425) may be positioned approximately between the first lens module (421) and the second lens module (422) or between the first display (431) and the second display (432). According to one embodiment, the wearable electronic device (400) may additionally include a waveguide to prevent loss due to light leakage until the light that has passed through the first lens module (421) and the second lens module (422) reaches the infrared camera (425).

[0096] In the embodiments of FIGS. 8 and 9, the infrared camera (425) may be positioned toward the rear (420) of the wearable electronic device (400). According to one embodiment, infrared rays passing through the first lens module (421) and the second lens module (422) and the first infrared pass filter (441) and the second infrared pass filter (442) may be directly incident on the infrared camera (425).

[0097] The infrared pass filter (441, 442) may be configured to allow infrared rays of a designated band among the light incident through the second surface (421b, 422b) of the lens module (421, 422) to pass through, and allow the passed infrared rays to be incident on the infrared camera (425). Based on the direction of propagation of the infrared rays, the infrared pass filter (441, 442) may be placed between the lens module (421, 422) and the infrared camera (425). Since the infrared pass filter (441, 422) blocks the transmission of light other than infrared rays of a designated band, for example, visible light, the influence on visual information obtainable through the lens module (421, 422) can be minimized. According to one embodiment, light incident on the infrared camera (425) through the second surface (421b, 422b) is output from at least one infrared light source, is reflected to the user's eye, and is incident on the infrared camera (425), and may include information about the user's line of sight (e.g., glint) or information about the user's iris.

[0098] The infrared pass filters (441, 442) may include a first infrared pass filter (441) corresponding to the first lens module (421) and a second infrared pass filter (442) corresponding to the second lens module (422). Referring to FIG. 9, the first infrared pass filter (441) and the second infrared pass filter (442) may be disposed on the third surfaces (421c, 422c) of the first lens module (421) and the second lens module (422), respectively. The third surfaces (421c, 422c) of the first lens module (421) and the second lens module (422), respectively, are surfaces close to the center of the wearable electronic device (400), and the first infrared pass filter (441) and the second infrared pass filter (442) disposed thereon may be disposed at positions facing each other. According to one embodiment, the infrared pass filter (441, 442) may be disposed at a position corresponding to the lens closest to the user's eye among the plurality of lenses included in the lens module (421, 422) on the third face (421c, 422c) of the lens module (421, 422). For example, the infrared pass filter (441, 442) may be disposed adjacent to the lens located first from the user's eye (hereinafter referred to as the 'first lens') among the plurality of lenses included in the lens module (421, 422). Light incident through the second face (421b, 422b) of the lens module (421, 422) may be refracted while passing through the first lens, and the refracted light may filter out only infrared rays of a specified band while passing through the infrared pass filter (441, 442) and reach the infrared camera (425). According to one embodiment, since visual information may be lost and / or distorted when light reaching the infrared camera passes through multiple lenses, the loss and / or distortion of visual information can be reduced and / or prevented by placing the infrared pass filter (441, 442) at a position corresponding to the lens closest to the user's eye among the multiple lenses included in the lens module (421, 422).

[0099] Referring to FIG. 9, one infrared pass filter (441, 442) is provided on each side of the first lens module (421) and the second lens module (422), but the embodiment of the present disclosure is not necessarily limited thereto. According to one embodiment, the infrared pass filters (441, 442) may each include a plurality of pass filters. For example, the infrared pass filters (441, 442) may include a plurality of filters (not shown) for blocking diffuse reflection or external light to increase infrared transmission performance. Hereinafter, the infrared pass filter will be described in more detail with reference to FIGS. 10 to 12.

[0100] FIG. 10 is a diagram of an infrared pass filter according to an embodiment of the present disclosure. FIG. 11 is a graph showing optical characteristics of an infrared band-pass film according to a frequency band according to an embodiment of the present disclosure. FIG. 12 is a diagram showing the principle of wave destructive interference of an antireflection film according to an embodiment of the present disclosure.

[0101] Here, Fig. 10 can illustrate a cross-section of a laminated structure of an infrared pass filter.

[0102] In addition to the function of passing only infrared rays of a specified band, the infrared pass filter (441, 442) may further include an anti-reflection function. Accordingly, the infrared pass filter (441, 442) may further include an anti-reflection material.

[0103] For example, as illustrated in FIG. 10, the infrared pass filter (441, 442) may include a glass (441-1, 442-1), a first filter layer (441-2, 442-2) formed on one surface of the glass (441-1, 442-1), and a second filter layer (441-3, 442-3) formed on the other surface. According to one embodiment, referring to the direction of movement of infrared (IR), the first filter layer (441-2, 442-2) may be a portion closer to the third surface (421c, 422c) of the lens module (421, 422), and the second filter layer (441-2, 442-2) may be a portion closer to the infrared camera (425). For example, layers (441-2, 442-2) may face the third side (421c, 422c) of the lens module (421, 422).

[0104] According to one embodiment, the infrared bandpass material may be formed in the first filter layer (441-2, 442-2) or the second filter layer (441-3, 442-3). The anti-reflection material may be formed in the first filter layer (441-2, 442-2) or the second filter layer (441-3, 442-3). According to one embodiment, the infrared bandpass material may be formed in either the first filter layer (441-2, 442-2) or the second filter layer (441-3, 442-3), and the anti-reflection material may be formed in the first filter layer (441-2, 442-2) and the second filter layer (441-3, 442-3), respectively. According to one embodiment, the infrared bandpass material and the anti-reflection material may be each attached to glass in the form of a film or a tape. Alternatively, the infrared bandpass material and the anti-reflection material may be applied to the surface of the glass by various methods such as chemical vapor deposition or painting.

[0105] Referring to Fig. 11, the optical characteristics of an infrared band-pass material can be examined. An infrared pass filter, for example, can pass light above the near-infrared wavelength of approximately 700 nm by including an infrared band-pass material, but can block light below the near-infrared wavelength. By including an infrared pass filter with such characteristics, visible light leakage can be prevented.

[0106] Referring to Fig. 12, the optical characteristics when an anti-reflection material is used can be examined. Fig. 12 can show that when light is incident on a layer including an anti-reflection material, surface reflection and back reflection occur, and wave destructive interference occurs due to this. For example, the layer including an anti-reflection material can include a base (B) (or a substrate) and a coating layer (CL) on the upper surface of the base (B). When visible light is incident on the layer including the anti-reflection material, the light reflected from the coating layer and the light reflected from the base can interfere. According to Fig. 12, when a layer including an anti-reflection material is provided, the reflectivity for visible light can be significantly reduced due to wave destructive interference. An infrared pass filter can reduce and / or prevent deterioration of an image and / or video output from a display due to reflection of visible light from the inside and outside of the glass, for example, by including an anti-reflection material.

[0107] As described above, the infrared pass filter (441, 442) includes an infrared band pass material and / or an anti-reflection material, and on the other hand, by controlling the refractive index of the glass, etc., it can block light in the visible light band and secure a sufficient amount of infrared light reaching the infrared camera.

[0108] Next, the lens module of the present disclosure can be described in more detail with reference to FIGS. 13a to 13c.

[0109] FIG. 13A is a drawing of a lens module according to one embodiment of the present disclosure. FIG. 13B is a drawing of a lens module according to one embodiment of the present disclosure. FIG. 13C is a drawing of a lens module according to one embodiment of the present disclosure.

[0110] The lens module of the present disclosure may be applied to various lens modules. For example, the lens module of the wearable electronic device of the present disclosure (e.g., the wearable electronic device (400) of FIGS. 8 and 9) may include one of a pancake lens, a Fresnel lens, and a convex lens. For example, the embodiment of FIG. 9 and the embodiments of FIGS. 13A and 13B discussed above may be examples of pancake lenses, and FIG. 13C may be an example of a Fresnel lens (or convex lens).

[0111] Referring to FIGS. 13A and 13B , in one embodiment, in a usage environment where a display and a user's eyes are positioned at a relatively close distance, an optical system with a pancake lens structure can be useful for providing good image quality while using a limited number of lenses. The optical system with a pancake lens structure can implement an optical path with a sufficient length compared to the mechanical length (e.g., the total length of the lens) by reflecting the visual information output by the display at least twice on the path to the user's eyes. The pancake lens structure can provide good image quality while being compact. However, the repeated reflection structure may increase the refraction or scattering of light. For example, when the refraction or scattering of light increases, a deterioration in image quality may occur due to interference of the refracted or scattered light. The pancake lens structure may further include, for example, a lens assembly (e.g., a plurality of lenses (L1, L2, L3 or more)), at least one quarter wave plate (QWP), at least one reflective polarizer (RP) and / or at least one beam splitter.

[0112] In one embodiment, in one embodiment included in the pancake lens, at least one of the plurality of lenses (L1, L2, L3) may be movable to adjust the diopter to provide a vision correction function to the user. Alternatively, the arrangement of the plurality of lenses (L1, L2, L3) from the initial design may vary depending on the embodiment. In the embodiment of FIG. 13b, the plurality of lenses (L1, L2, L3) are shown arranged closer to the user's eyes than the positions of the plurality of lenses (L1, L2, L3) included in the embodiment of FIG. 13a. In one embodiment, the positions of the infrared pass filters (441, 442) of the present disclosure may also be formed differently depending on the positions of the plurality of lenses (L1, L2, L3).

[0113] According to one embodiment, the number and / or positions of the infrared pass filters (441, 442) are not limited to the above-described embodiments. The infrared pass filters (441, 442) are substantially composed of one infrared pass filter, and a single infrared pass filter may be placed in front of the infrared camera (425).

[0114] Referring to Fig. 13c, a Fresnel lens (or convex lens) may be composed of a single lens (L). Referring to Fig. 13c, due to the simple lens assembly structure, it may be easier to utilize the internal space of the lens module (421, 422).

[0115] With respect to the lens modules of FIGS. 13a to 13c, application examples thereof may be disclosed in the wearable electronic devices illustrated in FIGS. 14 to 16.

[0116] Referring again to FIGS. 8 and 9, a wearable electronic device (400) according to an embodiment of the present disclosure may include at least one infrared light source (e.g., infrared light sources 227 and 228 of FIG. 5 ). According to an embodiment, the infrared light sources may be disposed on a rim of the wearable electronic device (400). For example, the infrared light sources may be disposed on each of the first rim (423) and the second rim (424) of the wearable electronic device (400). In another example, a plurality of infrared light sources may be radially disposed on each of the first rim (423) and the second rim (424). However, the embodiment for the arrangement of the at least one infrared light source is not limited thereto. According to an embodiment, the infrared light source may be disposed at a location adjacent to a display of the wearable electronic device (400). For example, a plurality of infrared light sources may be arranged along the edges of the first display (431) and the second display (432) of the wearable electronic device (400). According to one embodiment, the infrared light sources may be arranged on the display of the wearable electronic device (400). For example, the infrared light sources may be arranged to replace at least a portion of each of the pixels included in the first display (431) and the second display (432) of the wearable electronic device (400). The arrangement of the infrared light sources will be described in detail below with reference to the embodiments of FIGS. 17 and 18.

[0117] FIG. 14 is a diagram illustrating internal components of a wearable electronic device according to one embodiment of the present disclosure.

[0118] A wearable electronic device (500) according to one embodiment of the present disclosure may include a lens module (521, 522) including at least one lens. The wearable electronic device (500) may include a display (531, 532) arranged to face a first surface (521a, 522a) of the lens module (521, 522) and configured to output visual information toward the lens module. The wearable electronic device (500) may include an infrared camera (525) for acquiring user information by infrared rays output from at least one infrared light source. The wearable electronic device (500) may pass infrared rays of a designated band incident through the second face (521b, 522b) of the lens module (521, 522), and may include an infrared pass filter (541, 52) disposed on the third face (521c, 522c) of the lens module (521, 522). For the wearable electronic device (500) of FIG. 14, the description of the wearable electronic device (400) of the embodiments of FIGS. 8 and 9 may be applied. Therefore, the description of each component of FIG. 14 may be omitted to the extent overlapping with the embodiments of FIGS. 8 and 9.

[0119] The lens module (521, 522) may include two lens modules corresponding to the user's two eyes. According to one embodiment, the lens module (521, 522) may include a first lens module (521) corresponding to the user's left eye (EL) and a second lens module (522) corresponding to the user's right eye (ER). The first lens module (521) may include a first side (521a) facing the front side (510) of the wearable electronic device (500) and a second side (521b) facing in the opposite direction to the first side (521a). In addition, the first lens module (521) may include a third side (521c) and a fourth side (521d) facing in a different direction from the first side (521a) and the second side (521b). The second lens module (522) may include a first side (522a) facing the front side (510) of the wearable electronic device (500) and a second side (522b) facing in an opposite direction to the first side (522a). In addition, the second lens module (522) may include a third side (522c) and a fourth side (522d) facing in a different direction from the first side (522a) and the second side (522b). The lens module (500) of FIG. 14 may include a pancake lens structure similar to the lens module (400) of FIG. 9.

[0120] The displays (531, 532) may include a first display (531) corresponding to the first lens module (521) and a second display (532) corresponding to the second lens module (522). The first display (531) and the second display (532) may be arranged to face the first surfaces (521a, 522a) of the first lens module (521) and the second lens module (522), respectively, and may output visual information toward the first lens module (521) and the second lens module (522).

[0121] The infrared camera (525) may be positioned approximately between the first lens module (521) and the second lens module (522). According to one embodiment, the wearable electronic device (500) may additionally include a waveguide to prevent loss due to light leakage until the light passing through the first lens module (521) and the second lens module (522) reaches the infrared camera (525). In the embodiment of FIG. 14, the infrared camera (525) may be positioned toward the front (510) of the wearable electronic device (500). According to one embodiment, infrared light passing through the first lens module (521) and the second lens module (522) and the first infrared pass filter (541) and the second infrared pass filter (542) may be incident on the infrared camera (525).

[0122] The infrared pass filter (541, 542) may be configured to allow infrared rays of a designated band among the light incident through the second surface (521b, 522b) of the lens module (521, 522) to pass through, and allow the passed infrared rays to be incident on the infrared camera (525). Based on the direction of propagation of the infrared rays, the infrared pass filter (541, 542) may be placed between the lens module (521, 522) and the infrared camera (525). Since the infrared pass filter (541, 542) blocks the transmission of light other than infrared rays of a designated band, for example, visible light, the influence on visual information obtainable through the lens module (521, 522) can be minimized. According to one embodiment, infrared rays of a designated band among the light incident through the second surface (521b, 522b) may be output from at least one infrared light source, pass through the lens module (521, 522), and then be reflected into the user's eye and re-incident into the lens module (521, 522). The infrared rays reflected into the user's eye and re-incident may include information about the user's line of sight (e.g., glint) or information about the user's iris.

[0123] The infrared pass filters (541, 542) may include a first infrared pass filter (541) corresponding to the first lens module (521) and a second infrared pass filter (542) corresponding to the second lens module (522). Referring to FIG. 14, the first infrared pass filter (541) and the second infrared pass filter (542) may be disposed on the third surfaces (521c, 522c) of the first lens module (521) and the second lens module (522), respectively. The third surfaces (521c, 522c) of the first lens module (521) and the second lens module (522), respectively, are surfaces close to the center of the wearable electronic device (500), and the first infrared pass filter (541) and the second infrared pass filter (542) disposed thereon may be disposed at positions facing each other. According to one embodiment, the infrared pass filter (541, 542) may be positioned on the third face (521c, 522c) of the lens module (521, 522) at a position corresponding to the first lens (e.g., the first lens) closest to the user's eye. In the embodiment of FIG. 14, light incident through the second face (521b, 522b) of the lens module (521, 522) may be refracted while passing through the first lens, and the refracted light may pass through the infrared pass filter (541, 542) so that only infrared rays of a designated band are filtered out and reach the infrared camera (525).

[0124] The wearable electronic device (500) of FIG. 14 may further include a reflective member (550) configured to refract the infrared rays and focus or guide them to the infrared camera. The reflective member (550) may be disposed outside the lens modules (521, 522). The reflective member (550) may be disposed in a space between the first lens module (521) and the second lens module (522). The reflective member (550) may refract and / or reflect infrared rays reaching the reflective member (550) and focus and / or guide them to the infrared camera (525). As illustrated in FIG. 14, infrared rays passing through the infrared pass filters (541, 542) may be refracted and / or reflected after reaching the reflective member (550) and then be incident on the infrared camera (525) located behind the reflective member (550).

[0125] FIG. 15 is a diagram illustrating internal components of a wearable electronic device according to one embodiment of the present disclosure.

[0126] A wearable electronic device (600) according to one embodiment of the present disclosure may include a lens module (621, 622) including at least one lens. The wearable electronic device (600) may include a display (631, 632) arranged to face a first surface (621a, 622a) of the lens module (621, 622) and configured to output visual information toward the lens module. The wearable electronic device (600) may include an infrared camera (625) for acquiring user information by infrared rays output from at least one infrared light source. The wearable electronic device (600) may pass infrared rays of a designated band incident through the second face (621b, 622b) of the lens module (621, 622), and may include an infrared pass filter (641, 642) arranged on the third face (621c, 622c) of the lens module (621, 622). For the wearable electronic device (600) of FIG. 15, the description of the wearable electronic device (400) of the embodiments of FIGS. 8 and 9 may be applied. Therefore, the description of each component may be omitted to the extent of overlap.

[0127] The lens module (621, 622) may include two lens modules corresponding to the user's two eyes. According to one embodiment, the lens module (621, 622) may include a first lens module (621) corresponding to the user's left eye (EL) and a second lens module (622) corresponding to the user's right eye (ER). The first lens module (621) may include a first side (621a) facing the front side (610) of the wearable electronic device (600) and a second side (621b) facing in the opposite direction to the first side (621a). In addition, the first lens module (621) may include a third side (621c) and a fourth side (621d) facing in a different direction from the first side (621a) and the second side (621b). The second lens module (622) may include a first side (622a) facing the front side (610) of the wearable electronic device (600) and a second side (622b) facing in an opposite direction to the first side (622a). In addition, the second lens module (622) may include a third side (622c) and a fourth side (622d) facing in a different direction from the first side (622a) and the second side (622b). The lens module (600) of FIG. 15 may include a pancake lens structure, similar to the lens module (400) of FIG. 9. However, the lens module (600) of FIG. 15 may include a pancake structure that is positioned relatively closer toward the user's eyes than the pancake lens structure positioned in the lens module (400) of FIG. 9.

[0128] The displays (631, 632) may include a first display (631) corresponding to the first lens module (621) and a second display (632) corresponding to the second lens module (622). The first display (631) and the second display (632) may be arranged to face the first surfaces (621a, 622a) of the first lens module (621) and the second lens module (622), respectively, and may output visual information toward the first lens module (621) and the second lens module (622).

[0129] The infrared camera (625) may be positioned approximately between the first lens module (621) and the second lens module (622). According to one embodiment, the wearable electronic device (600) may additionally include a waveguide to prevent loss due to light leakage until the light passing through the first lens module (621) and the second lens module (622) reaches the infrared camera (625). In the embodiment of FIG. 15, the infrared camera (625) may be positioned toward the front (610) of the wearable electronic device (600). According to one embodiment, infrared light passing through the first lens module (621) and the second lens module (622) and the first infrared pass filter (641) and the second infrared pass filter (642) may be incident on the infrared camera (625).

[0130] The infrared pass filter (641, 642) may be configured to allow infrared rays of a designated band among the light incident through the second surface (621b, 622b) of the lens module (621, 622) to pass through, and allow the passed infrared rays to be incident on the infrared camera (625). Based on the direction of propagation of the infrared rays, the infrared pass filter (641, 642) may be placed between the lens module (621, 622) and the infrared camera (625). Since the infrared pass filter (641, 622) blocks the transmission of light other than infrared rays of a designated band, for example, visible light, the influence on visual information obtainable through the lens module (621, 622) can be minimized. According to one embodiment, infrared rays of a designated band among the light incident through the second surface (621b, 622b) may be output from at least one infrared light source, pass through the lens module (621, 622), and then be reflected into the user's eye and re-incident into the lens module (621, 622). The infrared rays reflected into the user's eye and re-incident may include information about the user's line of sight (e.g., glint) or information about the user's iris.

[0131] The infrared pass filters (641, 642) may include a first infrared pass filter (641) corresponding to the first lens module (621) and a second infrared pass filter (642) corresponding to the second lens module (622). Referring to FIG. 15, the first infrared pass filter (641) and the second infrared pass filter (642) may be disposed on the third surfaces (621c, 622c) of the first lens module (621) and the second lens module (622), respectively. The third surfaces (621c, 622c) of the first lens module (621) and the second lens module (622), respectively, are surfaces close to the center of the wearable electronic device (600), and the first infrared pass filter (641) and the second infrared pass filter (642) disposed thereon may be disposed at positions facing each other. According to one embodiment, the infrared pass filter (641, 642) may be positioned on the third face (621c, 622c) of the lens module (621, 622) at a position close to the first face (621a, 622a) (or display (631, 632)). In the embodiment of FIG. 15, light incident through the second face (621b, 622b) may pass through a plurality of lenses included in the lens module and then reach a position adjacent to the first face (621a, 621b) of the lens module (621, 622).

[0132] The wearable electronic device (600) of FIG. 15 may further include a reflective member (650). The reflective member (650) may refract infrared rays that have reached a position adjacent to the first surface (621a, 621b) of the lens module and focus or guide the infrared rays to the infrared camera (625). The reflective member (650) may be arranged inside / outside the lens module (621, 622) to refract and / or reflect infrared rays and focus and / or guide the infrared rays to the infrared camera (625). In the embodiment illustrated in FIG. 15, the reflective member (650) may include a first reflective member (651) disposed inside the first lens module (621), a second reflective member (652) disposed inside the second lens module (622), a third reflective member (653) disposed outside the first lens module (621), and a fourth reflective member (654) disposed outside the second lens module (622). In one embodiment, the reflective member (650) may include at least a partially curved surface. Light incident through the second face (621b, 622b) of the lens module (621, 622) is refracted while passing through a plurality of lenses in the lens module (621, 622), and after reaching the reflective member (650) (e.g., the first reflective member (651) and the second reflective member (652)), is refracted and / or reflected, and can proceed toward the third face (621c, 622c) of the lens module (621, 622). In addition, light that has passed through the infrared pass filter (641, 642) can be refracted and / or reflected again after reaching the reflective member (650) (the third reflective member (653) and the fourth reflective member (654)), and can then be incident on the infrared camera (625).

[0133] FIG. 16 is a diagram illustrating internal components of a wearable electronic device according to one embodiment of the present disclosure.

[0134] A wearable electronic device (700) according to one embodiment of the present disclosure may include a lens module (721, 722) including at least one lens. The wearable electronic device (700) may include a display (731, 732) arranged to face a first surface (721, 722a) of the lens module (721, 722) and configured to output visual information toward the lens module. The wearable electronic device (700) may include an infrared camera (725) for acquiring user information by infrared rays output from at least one infrared light source. The wearable electronic device (700) may pass infrared rays of a designated band incident through the second face (721b, 722b) of the lens module (721, 722), and may include an infrared pass filter (741, 742) arranged on the third face (721c, 722c) of the lens module (721, 722). For the wearable electronic device (700) of FIG. 16, the description of the wearable electronic device (400) of the embodiments of FIGS. 8 and 9 may be applied. Therefore, the description of each component may be omitted to the extent of overlap.

[0135] The lens module (721, 722) may include two lens modules corresponding to the user's two eyes. According to one embodiment, the lens module (721, 722) may include a first lens module (721) corresponding to the user's left eye (EL) and a second lens module (722) corresponding to the user's right eye (ER). The first lens module (721) may include a first surface (721a) facing the front surface (710) of the wearable electronic device (700) and a second surface (721b) facing in the opposite direction to the first surface (721a). In addition, the first lens module (721) may include a third surface (721c) and a fourth surface (721d) facing in a different direction from the first surface (721a) and the second surface (721b). The second lens module (722) may include a first surface (722a) facing the front surface (710) of the wearable electronic device (700) and a second surface (722b) facing in an opposite direction to the first surface (722a). In addition, the second lens module (722) may include a third surface (722c) and a fourth surface (722d) facing in a different direction from the first surface (722a) and the second surface (722b). The lens module (700) of FIG. 16 may include a Fresnel lens (or convex) lens structure, unlike the lens module (400) of FIG. 9. For example, the lens module (700) of FIG. 16 may include one Fresnel lens (or convex) lens. The lens module (700) of FIG. 16 can play a role in uniformly spreading light when visual information output from the display (731, 732) is focused or guided to the user's eye (EL, ER) through a Fresnel (or convex) lens, or when light reflected from the user's eye is incident on the lens module (721, 722) and directed toward the first surface (721a, 722a).

[0136] The displays (731, 732) may include a first display (731) corresponding to the first lens module (721) and a second display (732) corresponding to the second lens module (722). The first display (731) and the second display (732) may be arranged to face the first surfaces (721a, 722a) of the first lens module (721) and the second lens module (722), respectively, and may output visual information toward the first lens module (721) and the second lens module (722).

[0137] The infrared camera (725) may be positioned approximately between the first lens module (721) and the second lens module (722). According to one embodiment, the wearable electronic device (700) may additionally include a waveguide to prevent loss due to light leakage until the light passing through the first lens module (721) and the second lens module (722) reaches the infrared camera (725). In the embodiment of FIG. 16, the infrared camera (725) may be positioned toward the front (710) of the wearable electronic device (700). According to one embodiment, infrared light passing through the first lens module (721) and the second lens module (722) and the first infrared pass filter (741) and the second infrared pass filter (742) may be incident on the infrared camera (725).

[0138] The infrared pass filter (741, 742) may be configured to allow infrared rays of a designated band among the light incident through the second surface (721b, 722b) of the lens module (721, 722) to pass through, and allow the passed infrared rays to be incident on the infrared camera (725). Based on the direction of propagation of the infrared rays, the infrared pass filter (741, 742) may be placed between the lens module (721, 722) and the infrared camera (725). Since the infrared pass filter (741, 722) blocks the transmission of light other than infrared rays of a designated band, for example, visible light, the influence on visual information obtainable through the lens module (721, 722) can be minimized. According to one embodiment, infrared rays of a designated band among the light incident through the second surface (721b, 722b) may be output from at least one infrared light source, pass through the lens module (721, 722), and then be reflected into the user's eye and re-incident into the lens module (721, 722). The infrared rays reflected into the user's eye and re-incident may include information about the user's line of sight (e.g., glint) or information about the user's iris.

[0139] The infrared pass filters (741, 742) may include a first infrared pass filter (741) corresponding to the first lens module (721) and a second infrared pass filter (742) corresponding to the second lens module (722). Referring to FIG. 16, the first infrared pass filter (741) and the second infrared pass filter (742) may be disposed on the third surfaces (721c, 722c) of the first lens module (721) and the second lens module (722), respectively. The third surfaces (721c, 722c) of the first lens module (721) and the second lens module (722), respectively, are surfaces close to the center of the wearable electronic device (700), and the first infrared pass filter (741) and the second infrared pass filter (742) disposed thereon may be disposed at positions facing each other. According to one embodiment, the infrared pass filter (741, 742) may be positioned at a position close to the first surface (721a, 722a) (or display (731, 732)) on the third surface (721c, 722c) of the lens module (721, 722). In the embodiment of FIG. 16, light incident through the second surface (721b, 722b) may pass through the Fresnel lens (or convex lens) included in the lens module and then reach a position adjacent to the first surface (721a, 721b) of the lens module (721, 722).

[0140] The wearable electronic device (700) of FIG. 16 may further include a reflective member (750). The reflective member (750) may refract infrared rays that have reached a position adjacent to the first surface (721a, 721b) of the lens module and focus or guide the infrared rays to the infrared camera (725). The reflective member (750) may be positioned inside / outside the lens module (721, 722) to refract and / or reflect infrared rays and focus and / or guide the infrared rays to the infrared camera (725). In the embodiment illustrated in FIG. 16, the reflective member (750) may include a first reflective member (751) disposed inside the first lens module (721), a second reflective member (752) disposed inside the second lens module (722), a third reflective member (753) disposed outside the first lens module (721), and a fourth reflective member (754) disposed outside the second lens module (722). In one embodiment, the reflective member (750) may include at least a partially curved surface. Light incident through the second face (721b, 722b) of the lens module (721, 722) is refracted while passing through the Fresnel lens (or convex lens) in the lens module (721, 722), and after reaching the reflective member (750) (e.g., the first reflective member (751) and the second reflective member (752)), is refracted and / or reflected, and can proceed toward the third face (721b, 722b) of the lens module (721, 722). In addition, light that has passed through the infrared pass filter (741, 742) can be refracted and / or reflected again after reaching the reflective member (750) (the third reflective member (753) and the fourth reflective member (754)), and can then be incident on the infrared camera (725).

[0141] FIG. 17 is a diagram illustrating a display according to one embodiment of the present disclosure. FIG. 18 is a diagram illustrating a display according to one embodiment of the present disclosure.

[0142] According to one embodiment, at least one infrared light source that may be included in the wearable electronic device may include at least one infrared light source that may be disposed on the rim or, alternatively, at least one infrared light source that may be disposed inside the wearable electronic device.

[0143] FIG. 17 may illustrate that at least one infrared light source (827) is disposed on a substrate (805) placed inside a wearable electronic device. At this time, a display (831) may be disposed on the substrate (805), and at least one infrared light source is depicted disposed around the display (831). In FIG. 17, at least six infrared light sources are depicted, but this is merely an example and is not limited thereto.

[0144] FIG. 18 may illustrate that at least one infrared light source (827) is arranged in a display (831). At this time, the display (831) may be composed of a plurality of pixel(s), and some of the plurality of pixel(s) may be replaced with at least one infrared light source (827). According to one embodiment, among the plurality of pixel(s) included in the display 831), pixel(s) arranged at the edge are pixels that do not transmit actual images or video, and may have a non-visible area (NVA) that does not transmit images. In addition, among the plurality of pixel(s) included in the display 831), another portion, that is, pixel(s) located in the center, are pixels that transmit actual images or video, and may have a visible area (VA) that transmits images. According to one embodiment, some of the pixel(s) arranged at the edge among the plurality of pixel(s) may be replaced with at least one infrared light source (827).

[0145] As illustrated in FIGS. 17 and 18, when at least one infrared light source is placed on or around the display (831), some of the infrared light output from the at least one infrared light source may be lost in the process of being refracted and / or reflected while passing through the lens(es) included in the lens module. Accordingly, the present disclosure implements an optimized optical design including lenses such as a pancake lens, a Fresnel lens, and a convex lens, thereby reducing and / or minimizing the light lost in the process of being refracted and / or reflected.

[0146] FIG. 19 is a schematic diagram illustrating a gaze tracking method using a wearable electronic device according to one embodiment of the present disclosure. FIG. 20 is a schematic diagram illustrating a gaze tracking method using a wearable electronic device according to one embodiment of the present disclosure.

[0147] The present disclosure may provide various embodiments for tracking a user's gaze and / or recognizing an iris using a wearable electronic device. As an example of a method for tracking gaze, the present disclosure may provide a method for tracking gaze using a single camera, as in the embodiment described above with reference to FIG. 8 and subsequent drawings, unlike the two camera modules used to track gaze in FIGS. 3 and 4.

[0148] Taking Fig. 19 as an example, glint images of a user's eyes can be acquired using a single infrared camera. At this time, one acquired frame (frame 1) may include two glint images of the left eye and the right eye. For example, a processor included in a wearable electronic device (e.g., processor (120) of Fig. 1 or an image signal processor) may divide the two glint images included in one frame (frame 1) into two frames (frame 1-1, frame 1-2) corresponding to the user's left eye and right eye, respectively. Then, eye data for the user's left eye and right eye, respectively, can be extracted using the two divided frames (frame 1-1, frame 1-2). A specific gaze tracking method may be applied to the embodiments of Figs. 6a to 7b above.

[0149] Taking Fig. 20 as an example, a single infrared camera can be used to acquire glint images of a user's eyes, and two glint images of the left eye and the right eye can be acquired alternately. For example, a processor included in a wearable electronic device (e.g., the processor (120) of Fig. 1 or an image signal processor) can acquire a glint image of the left eye in the first frame (frame 1), and a glint image of the right eye in the second frame (frame 2). Then, the glint image of the left eye can be acquired again in the third frame (frame 3). Here, the order in which the left eye or the right eye is captured first can be set in various ways depending on the embodiment. The embodiment of Fig. 20 can extract eye data for the left eye and the right eye using a single infrared camera by alternately capturing the left eye and the right eye at predetermined time intervals.

[0150] According to the gaze tracking method of FIGS. 19 and 20, glint images for the user's left and right eyes can be acquired using a single infrared camera, similar to tracking gaze using two infrared cameras. Furthermore, according to the present disclosure, using a single infrared camera can have the effect of reducing the amount of current consumed compared to tracking gaze using two infrared cameras.

[0151] Electronic devices according to embodiments of the present disclosure may take various forms. Electronic devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, wearable devices, or home appliances. Electronic devices according to embodiments of the present disclosure are not limited to the aforementioned devices.

[0152] The embodiments of the present disclosure and the terminology used therein are not intended to limit the technical features described in this document to specific embodiments, but should be understood to encompass various modifications, equivalents, or substitutes of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the items, unless the context clearly dictates otherwise. In this document, phrases such as "A or B," "at least one of A and B," "at least one of A or B," "A, B, or C," "at least one of A, B, and C," and "at least one of A, B, or C" each include any one of the items listed together in the corresponding phrase, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used merely to distinguish the corresponding component from other corresponding components, and do not limit the corresponding components in any other respect (e.g., importance or order). When a component (e.g., a first component) is referred to as being "coupled" or "connected" to another component (e.g., a second component), with or without the terms "functionally" or "communicatively," it is understood that the component can be connected to the other component directly (e.g., wired), wirelessly, or via a third component.

[0153] The term "module" used in the embodiments of the present disclosure may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integral component, or a minimum unit or part of such a component that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0154] Embodiments of the present disclosure may be implemented as software (e.g., a program) including one or more instructions stored in a storage medium (e.g., built-in memory or external memory) readable by a machine (e.g., an electronic device). For example, a processor (e.g., a processor) of the machine (e.g., an electronic device) may call at least one instruction among the one or more instructions stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the at least one instruction called. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, "non-transitory" only means that the storage medium is a tangible device and does not contain a signal (e.g., electromagnetic waves), and this term does not distinguish between cases where data is stored semi-permanently and cases where it is stored temporarily in the storage medium.

[0155] According to one embodiment, a method according to the embodiment(s) of the present disclosure may be provided as a computer program product. The computer program product may be traded between sellers and buyers as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read only memory (CD-ROM)) or may be provided through an application store (e.g., Play Store). TM ) or directly between two user devices (e.g., smartphones), online distribution (e.g., downloading or uploading). In the case of online distribution, at least a portion of the computer program product may be at least temporarily stored or temporarily created in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

[0156] According to one embodiment, each component (e.g., a module or a program) of the above-described components may include one or more entities, and some of the entities may be separated and placed in other components. According to one embodiment, one or more components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., a module or a program) may be integrated into a single component. In such a case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. According to one embodiment, the operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

[0157] According to one embodiment of the present disclosure, a wearable electronic device (101; 200; 400; 500; 600; 700) may be provided. The wearable electronic device may include at least one infrared light source (227; 228; 827); a lens module (221; 222; 421; 422; 521; 522; 621; 622; 721; 722) including at least one lens; a display (D; 431; 432; 531; 532; 631; 632; 731; 732) arranged to face a first surface (421a; 422a; 521a; 522a; 621a; 622a; 721a; 722a) of the lens module and configured to output visual information toward the lens module; An infrared camera (425; 525; 625; 725) for obtaining user information by infrared rays output from at least one infrared light source; and an infrared pass filter (441; 442; 541; 542; 641; 642; 741; 742) disposed on a third face (421c; 422c; 521c; 522c; 621c; 622c; 721c; 722c) of the lens module, which passes infrared rays of a designated band incident through the second face (421b; 422b; 521b; 522b; 621b; 622b; 721b; 722b) of the lens module.

[0158] According to one embodiment, the lens module may include a first lens module (221; 421; 521; 621; 721) and a second lens module (222; 422; 522; 622; 722) corresponding to the user's two eyes.

[0159] According to one embodiment, the display may include a first display (D; 431; 531; 631; 731) corresponding to the first lens module and a second display (D; 432; 532; 632; 732) corresponding to the second lens module.

[0160] According to one embodiment, the infrared pass filter may include a first infrared pass filter (441; 541; 641; 741) corresponding to the first lens module and a second infrared pass filter (442; 542; 642; 742) corresponding to the second lens module.

[0161] According to one embodiment, the infrared camera may be positioned between the first lens module and the second lens module.

[0162] According to one embodiment, the first infrared pass filter and the second infrared pass filter may be arranged to face each other.

[0163] According to one embodiment, the present invention may further include a reflective member (550; 650; 750) configured to refract the infrared rays and focus or guide them to the infrared camera.

[0164] According to one embodiment, the reflective member may be disposed inside and / or outside the lens module.

[0165] In one embodiment, the reflective member may have at least a partially refracted surface.

[0166] According to one embodiment, the lens module may be one of a pancake lens, a Fresnel lens, and a convex lens.

[0167] In one embodiment, the infrared camera may be positioned toward the rear (420; 520; 620; 720) of the wearable electronic device.

[0168] According to one embodiment, the infrared pass filter may be positioned on the third surface (421c; 422c; 521c; 522c; 621c; 622c; 721c; 722c) of the lens module, at a position corresponding to the lens closest to the user's eye among the plurality of lenses included in the lens module.

[0169] According to one embodiment, the infrared pass filter may further include an anti-reflection function.

[0170] According to one embodiment, the infrared pass filter may further include glass (441-1; 441-2); an infrared bandpass material formed on at least one surface of the glass; and an antireflection material formed on one surface and the other surface of the glass.

[0171] According to one embodiment, the infrared camera may be positioned toward the front (410; 510; 610; 710) of the wearable electronic device.

[0172] According to one embodiment, the infrared pass filter may be positioned on the third side of the lens module, adjacent to the first side.

[0173] In one embodiment, the wearable electronic device can track the user's gaze or recognize the iris using the infrared camera.

[0174] According to one embodiment of the present disclosure, a wearable electronic device (101; 200; 400; 500; 600; 700) may be provided. The wearable electronic device may include at least one infrared light source (227; 228; 827); a first lens module (221; 421; 521; 621; 721) and a second lens module (222; 422; 522; 622; 722) including at least one lens; a first display (431; 531; 631; 731) arranged to face a first surface (421a; 521a; 621a; 721a) of the first lens module and configured to output visual information toward the first lens module; A second display (432; 532; 632; 732) arranged to face the first face (422a; 522a; 622a; 722a) of the second lens module and configured to output visual information toward the first lens module; a single infrared camera (425; 525; 625; 725) for acquiring user information by infrared light output from the at least one infrared light source; a first infrared pass filter arranged on the third face (421c; 521c; 621c; 721c) of the first lens module, the first infrared light passing through the second face (421b; 521b; 621b; 721b) of the first lens module; And it may include a second infrared pass filter (441;442;541;542;641;642;741;742) arranged on a third surface (422c;522c;622c;722c) of the second lens module, which passes infrared rays of a designated band incident through the second surface (422b;522b;622b;722b) of the second lens module. The single infrared camera may be positioned between the third surface of the first lens module and the third surface of the second lens module.

[0175] According to one embodiment, the infrared camera may be positioned between the first lens module and the second lens module.

[0176] According to one embodiment, it may include a reflective member (550; 650; 750) configured to refract the infrared rays and focus or guide them to the infrared camera.

[0177] According to one embodiment, the lens module may be one of a pancake lens, a Fresnel lens, and a convex lens.

[0178] In one embodiment, the infrared camera may be positioned toward the front or back of the wearable electronic device.

[0179] While this disclosure has been described by way of example and example, it should be understood that the example is intended to be illustrative and not limiting. It will be apparent to those skilled in the art that various changes in form and detail may be made without departing from the overall scope of this disclosure, including the appended claims and their equivalents.

Claims

1. In wearable electronic devices (101; 200; 400; 500; 600; 700), At least one infrared light source (227; 228; 827); A lens module comprising at least one lens (221;222;421;422;521;522;621;622;721;722); A display (D; 431; 432; 531; 532; 631; 632; 731; 732) arranged to face the first surface (421a; 422a; 521a; 522a; 621a; 622a; 721a; 722a) of the lens module and configured to output visual information toward the lens module; An infrared camera (425; 525; 625; 725) for obtaining user information by infrared light output from at least one of the above infrared light sources; and A lens module having a second surface (421b;422b;521b;522b;621b;622b;721b;722b) for passing infrared rays of a designated band incident thereon, and including an infrared pass filter (441;442;541;542;641;642;741;742) disposed on a third surface (421c;422c;521c;522c;621c;622c;721c;722c) of the lens module. Wearable electronic devices.

2. In paragraph 1, The above lens module includes a first lens module (221;421;521;621;721) and a second lens module (222;422;522;622;722) corresponding to the user's two eyes, The display includes a first display (D; 431; 531; 631; 731) corresponding to the first lens module and a second display (D; 432; 532; 632; 732) corresponding to the second lens module. An electronic device comprising a first infrared pass filter (441; 541; 641; 741) corresponding to the first lens module and a second infrared pass filter (442; 542; 642; 742) corresponding to the second lens module.

3. In paragraph 2, The above infrared camera is an electronic device disposed between the first lens module and the second lens module.

4. In paragraph 2, A wearable electronic device wherein the first infrared pass filter and the second infrared pass filter are positioned facing each other.

5. In any one of paragraphs 1 to 4, A wearable electronic device further comprising a reflective member (550; 650; 750) configured to refract the infrared rays and focus or guide them to the infrared camera.

6. In paragraph 5, A wearable electronic device wherein the reflective member is disposed inside and / or outside the lens module.

7. In paragraph 5, A wearable electronic device wherein the reflective member has at least a partially refracted surface.

8. In any one of paragraphs 1 to 7, A wearable electronic device wherein the lens module is one of a pancake lens, a Fresnel lens, and a convex lens.

9. In any one of paragraphs 1 to 8, A wearable electronic device wherein the above infrared camera is positioned toward the rear (420; 520; 620; 720) of the wearable electronic device.

10. In any one of paragraphs 1 to 9, The above infrared pass filter is positioned on the third surface (421c; 422c; 521c; 522c; 621c; 622c; 721c; 722c) of the lens module, in a position corresponding to the lens closest to the user's eye among the plurality of lenses included in the lens module.

11. In any one of paragraphs 1 to 10, A wearable electronic device wherein the above infrared pass filter further includes an anti-reflection function.

12. In paragraph 11, Infrared pass filter Glass (441-1; 441-2); An infrared band pass material formed on at least one surface of the above glass, and A wearable electronic device further comprising an anti-reflective material formed on one surface and the other surface of the glass.

13. In any one of paragraphs 1 to 12, A wearable electronic device wherein the above infrared camera is positioned toward the front (410; 510; 610; 710) of the wearable electronic device.

14. In any one of paragraphs 1 to 13, A wearable electronic device wherein the above infrared pass filter is positioned on the third surface of the lens module, adjacent to the first surface.

15. In any one of paragraphs 1 to 14, The above wearable electronic device is a wearable electronic device that tracks the user's gaze or recognizes the iris using the infrared camera.

Images